Arman Hossain built GuppyLM to prove you don't need a PhD or massive GPU cluster to understand how language models work. The project is an 8.7 million parameter transformer that speaks in short, lowercase sentences about aquatic life. It's intentionally simple. Vanilla architecture, no Grouped Query Attention or Rotary Positional Embeddings. Just the basics so you can see how every piece fits together.
The whole pipeline runs in Google Colab. Hossain gives you synthetic data generation scripts, tokenizer training, model architecture, and the training loop. Five minutes on a T4 GPU gets you a working model. Ask it about life's meaning and you'll hear "food. the answer is always food." The 128-token context window limits conversations to single turns. Fits a fish pretty well.
Projects like this strip away the complexity around LLMs. Andrej Karpathy's nanoGPT and Ronen Eldan's TinyStories already showed that small models can teach big lessons. GuppyLM gives you the full stack from raw data to chatbot. If you've wanted to peek inside the black box, Hossain just handed you the flashlight.