GuppyLM is a 9 million parameter language model that does one thing honestly: it pretends to be a fish. The model speaks in short, lowercase sentences about water temperature, bubbles, food, and tank life. Ask it about the meaning of life and it responds "food. the answer is always food." Ask for a joke? "what did the fish say when it hit the wall. dam." Five minutes on a Google Colab T4 GPU. That's all it takes to train from scratch. Developer arman-bd built this to strip away every modern optimization. No Grouped Query Attention. No Rotary Position Embeddings. No SwiGLU activations. Just a vanilla transformer with 6 layers, 384 hidden dimensions, and learned position embeddings. The training data is 60,000 synthetic single-turn conversations across 60 topics, all generated from templates. BPE tokenizer. 4,096 vocab size. Everything runs in a single Colab notebook. Build even a tiny model and you're forced to understand how random tokens become coherent output. Hacker News commenters noted this. You see the full pipeline: data generation, tokenizer training, model architecture, training loop, inference. Coding agents often require similar orchestration to manage these stages effectively. The weights are tied with embeddings. Context maxes out at 128 tokens. Small enough to run in a browser. Available on Hugging Face under MIT license, with separate notebooks for training your own or just chatting with the pre-trained fish.
GuppyLM: A tiny fish-brain LLM that teaches transformers
GuppyLM is a ~9M parameter educational language model that speaks like a fish. Built from scratch using a vanilla transformer architecture, it trains on 60K synthetic conversations in 5 minutes on Google Colab. The project aims to explain how LLMs work by showing the complete pipeline: data generation, tokenizer, model architecture, training loop, and inference.
