Language Model Builder: Train a Small LLM From Scratch on Your Mac
On July 21, 2026, developer Felix Rieseberg released Language Model Builder — a free native macOS app that walks you through building a small language model from scratch on your own hardware. It pairs an interactive textbook on tokenization, embeddings, attention, and transformers with a real training pipeline: pre-training, supervised fine-tuning, and direct preference optimization, all running locally on Apple Silicon via Apple’s MLX framework. Version 1.1.0 landed the very next day.
Intermediate
A Textbook That Actually Trains Something
Most “learn how LLMs work” resources stop at explanation. Language Model Builder is structured as a fifteen-step course that ends with a model you can talk to. The sidebar moves through three phases: Foundations (what a language model is, tokenization, embeddings), How models work (anatomy of a transformer, training data, how models learn, how models learn behavior), and Build your model — project setup, pre-training data, pre-training, sampling, fine-tuning data, supervised fine-tuning, direct preference optimization, and finally chatting with the result.
The prose and the machinery sit side by side. On the DPO page, for example, the explanation of preference learning runs in the right-hand pane while a “preference ballot” in the left pane asks your own supervised fine-tuning checkpoint the same question twice and asks you to pick the better answer. Four saved pairs unlock a DPO run — a deliberately low bar the app describes as “a minimum for seeing the machinery work, not a substantial preference dataset.”
What You Can Actually Train
The app ships a catalog of pre-training corpora rather than making you hunt for data. TinyStories 2 (2.2 GB of synthetic short stories, licensed CDLA Sharing 1.0) is the recommended starting point, alongside WikiText-2 Raw, Simple English Wikipedia, GoodWiki, arXiv Abstracts, Cosmopedia, Tiny Shakespeare, and a “Add your own” option. Fine-tuning sets include Everyday Conversations, No Robots, Dolly, and GSM8K.
Scale is modest by design. The demo run pictured above trains a ~25.1M-parameter model with a 10K-vocabulary BPE tokenizer over a 547M-token corpus at roughly 16,000 tokens per second. Rieseberg says default settings produce “a model that writes coherent, grammatical multi-paragraph text in as little as a day,” and that a MacBook Pro M5 Max can reach GPT-2-small class — roughly 100–150M parameters on a few billion tokens — in about a week. Training checkpoints save as they go, so runs can be paused and resumed, or kept around for comparison.
X-Ray Mode
The payoff screen is the chat interface, which renders every generated token as a shaded chip — stronger tint means the model sampled it with higher probability. Click any token and the app shows what else was possible and at what odds. In the example below, the model picked “of” at 31.9% over “and” at 31.3%, “was” at 17.0%, and “to” at 8.5%. A transcript pane on the right shows the raw chat template with its <|user|> and <|assistant|> markers, plus context usage and sampler settings.
It is a small feature that does a lot of pedagogical work: sampling stops being an abstraction once you can see that two candidate tokens were separated by half a percentage point.
Local, Free, and Deliberately Limited
Nothing leaves the machine. There is no account, no cloud component, and no bill — models, datasets, and training history all stay local, and outputs are written in the standard safetensors format so trained models open in other tools. Asked why the app is free, Rieseberg answers plainly: “Because I like building things and this app was fun to build. I am fortunate enough that I don’t need to sell it.”
The constraints are the flip side of the MLX dependency: Apple Silicon and macOS 15 or later, with no Windows, Linux, or Intel Mac builds planned. Version 1.1.0, published on July 22, added fine-tuning for existing open-weight base models including Qwen 2.5, dataset mixing, and the ability to build fine-tuning datasets from your own iMessage conversations — complete with group chats and speaker labels. The 26.7 MB installer had drawn roughly 165 downloads within a day of the Show HN post.
What This Means
Language Model Builder is not competing with production training stacks, and it does not pretend to. A 25M-parameter model trained on children’s stories will not do anything useful. What it does is collapse the distance between reading about pre-training and watching a loss curve flatten on your own laptop — and then between reading about RLHF-style alignment and clicking through preference pairs that visibly change how your model answers.
For students and instructors, that matters more than capability. The full base → SFT → DPO pipeline is the same shape used by frontier labs, just scaled down until a single machine can run it end to end in a weekend. Tools like Unsloth Studio have made local fine-tuning approachable; this goes one step further back, to the point where the model doesn’t exist yet.
Related Coverage
- Unsloth Studio: Open-Source No-Code UI for Local LLM Training and Inference — a comparable no-code local training dashboard, aimed at fine-tuning existing models
- Reverse Engineering Apple’s Neural Engine to Train Transformers on M4 — another effort to push transformer training onto Apple Silicon
- Apple Open-Sources Its Foundation Models Framework, Adds Claude and Gemini — background on Apple’s on-device ML stack
沪公网安备31011502017015号