nanochat
Andrej Karpathy · Other · Chatbot
nanochat is an open-source, full-stack LLM training framework from Andrej Karpathy that builds a ChatGPT-style language model from scratch. It covers the whole pipeline, from tokenization and pretraining to finetuning, evaluation, and inference, in a single codebase of roughly 8,000 lines. You rent one GPU node, run a script, and a few hours later you can chat with the model you just trained through a web interface or the command line.
About nanochat
What Is nanochat
nanochat is the simplest experimental setup for training large language models. It's built for a single GPU node, the code is minimal and hackable, and it walks through every major LLM stage: tokenization, pretraining, midtraining, supervised finetuning, optional reinforcement learning, evaluation, and inference. Karpathy calls it the best ChatGPT that $100 can buy.
The point isn't to compete with frontier models. A nanochat model trained for a few hours is a small, curious thing that can hold basic conversations, tell stories, and answer simple questions. What you get instead is a complete, readable training pipeline you can actually understand and change. That's why it's shaping up to be the capstone project for Karpathy's LLM101n course.
The honest limit: this is a learning and research tool, not a production model. Karpathy himself doesn't recommend finetuning it on your own writing. A model this small will imitate surface style, not grasp the deeper reasoning a top-tier LLM has. If you want that, finetune a larger open model like Llama 3 or use retrieval with a bigger system.
Getting Started
- Spin up a cloud GPU server with a single 8XH100 node (market price is around $24 per hour).
- Clone the repo from GitHub and install the
uvproject manager if you don't already have it. - Run the
runs/speedrun.shscript, which handles data prep, training, and finetuning end to end. - Wait a few hours. The script trains a GPT-2-grade model and reports its scores.
- Start the web server or the CLI and chat with your freshly trained model.
Product Information
A quick look at nanochat's pricing, supported platforms, and performance.
Best for
The users, tasks, and scenarios where this tool fits best.
Users
- Machine learning students
- Researchers and hobbyists
- Small teams exploring custom models
Tasks
- Training a GPT-2-class language model
- Studying the LLM pipeline
- Benchmarking training tweaks
Scenarios
- Running your first end-to-end training job
- Teaching a class or running a workshop
- Weekend experimentation
Key features
One Script, Full Pipeline
The nanochat speedrun ties the whole process together on a fresh server. It prepares data, trains a tokenizer, pretrains the transformer, runs midtraining and finetuning, and produces a model you can talk to. One command replaces the usual patchwork of separate tools, which is the whole point of keeping everything in a single repo rather than scattering it across projects.
The Depth Dial
nanochat is set up to train a whole family of compute-optimal models by changing one setting: --depth, the number of layers in the transformer. Everything else, including width, attention heads, learning rate schedules, and training horizon, gets calculated automatically. Want GPT-2 grade? Set depth to 26. That's roughly where GPT-2 capability lands.
Rust Tokenizer and Clean Data Prep
The tokenizer is written from scratch in Rust for speed, and the data pipeline repacks a subset of the FineWeb-Edu dataset into compact Parquet files for streaming. It's the unglamorous plumbing. But it's the part most tutorials skip, and it's here in full.
Inference Engine with a Web UI
Training ends with something you can actually use. nanochat ships an efficient inference engine with KV caching, plus both a CLI and a ChatGPT-style web interface. There's also a lightweight Python sandbox for tool calls, so the model can try to run code. That last part is a nice touch.
Optional RL Stage
Beyond supervised finetuning, you can run an optional reinforcement learning stage using the GRPO algorithm on the GSM8K math dataset. It's a bonus stage rather than a core requirement, but it shows how a small model can be pushed on gradeable tasks, which is genuinely useful if your research touches on reasoning or math.
Markdown Report Card
After training, nanochat generates a single Markdown report card that summarizes the run. It reads like a scorecard, covering model size, training time, and results across benchmarks such as MMLU, ARC-Easy, and GSM8K. That makes comparisons easy. Keep it around.
MIT-Licensed and Hackable
The entire project is open under the MIT license and runs to about 8,000 lines, mostly Python with a little Rust. The code is meant to be read, forked, and modified. Karpathy wrote most of it by hand, and the structure stays close to the metal, which means there's little hidden magic to trip you up when you start poking around.
Pros and cons
Pros
- Covers the entire LLM pipeline, from tokenizer to chat UI, in one small repo.
- Trains a GPT-2-class model for roughly $100 in GPU rental, far below what it cost in 2019.
- Only needs a single GPU node, so you skip distributed-training complexity.
- The MIT license and readable code make it easy to fork and adapt for your own experiments.
- Keeps an active speedrun leaderboard that shows how fast others have pushed the same task.
Cons
- It isn't a production model. The output is a small model for learning, not a rival to GPT-4 or Claude, so don't expect polished answers.
- You still need real GPU hardware or a cloud account; there's no way to run the training on a laptop or a phone.
- Finetuning it to match a personal writing style doesn't work well, because the base model is too small to absorb more than surface patterns.
- Setup assumes some comfort with Linux, Python, and the command line, which shuts out non-technical users.
Frequently asked questions
nanochat is used to train and run a small ChatGPT-style language model from scratch. People use it to learn how LLMs work end to end, to test training ideas cheaply, and to build a working chatbot from a single script.
Related content
Explore related tools, skills, and articles for nanochat.
nanochat Alternatives
BinkBink
BinkBink · OtherBinkBink is a free online game platform and AI game maker that lets anyone turn a short text description into a playable browser game. You can jump into hundreds of community-made games. Or describe your own idea and play it in seconds, then share it with friends. Want to create your own game? You don't need to code. No engine setup, no download, no hassle.
Audiogen
Audiogen Inc. · OtherAudiogen is an AI music generator built by Audiogen Inc., a small research team that spent about 2.5 years training its own generative music model and designing a web interface around it. Instead of a plain text box, this AI music tool turns the timeline into a beginner-friendly Generative Audio Workstation, or GAW, where inpainting, extending, remixing and stem editing work more like painting on a canvas. The product is still in beta, so access runs through a waitlist or an invite. Paid plans aren't published yet.
Aiml API
AIMLAPI OÜ · OtherAiml API is a unified AI model API that puts more than 1000 models from OpenAI, Google, Anthropic, and others behind one endpoint and one bill. You write code against a single OpenAI-compatible schema, then switch between chat, image, video, and audio models by changing a model string. It suits developers and small teams who want multi-model access without juggling a dozen separate provider accounts, and it removes the usual billing headache that comes with testing several vendors. One key covers it all.