Soup CLI
MePlay, Inc. · Coding
Soup CLI is an open-source, command-line tool for fine-tuning and post-training large language models. As an LLM fine-tuning tool, it turns the usual setup work into one workflow: you write a single YAML config, run `soup train`, and Soup handles GPU detection, batch sizing, and quantization on its own. The project is built for people who want to fine-tune models locally instead of renting cloud GPUs or wrestling with training scripts, and its headline feature, layer streaming, lets an 8B model train on a laptop GPU with only 4 GB of VRAM.
About Soup CLI
What Is Soup CLI
Soup CLI (also published as the soup-cli Python package) is a CLI-first stack that covers the whole post-training loop: data cleaning, method selection, config generation, evaluation, and checkpoint gating. It's maintained by MePlay, Inc. and released under the Apache-2.0 license, so you can read the source, fork it, or ship it inside your own pipeline.
The main problem it solves is friction. Anyone who has trained a model knows the drill: pick hardware, fight dependency conflicts, guess at a batch size, and debug a crash at step 400. Soup compresses that into a config file it writes for you. Why does that matter? Because the setup, not the training itself, is where most beginner fine-tunes die. It's aimed at developers, researchers, and small teams who want a working fine-tune without becoming infrastructure engineers.
The biggest limitation is hardware. Training still wants a CUDA GPU, and running a real 7B to 8B job on a normal machine means accepting QLoRA and quantized weights rather than full-precision training. That's the trade-off. Layer streaming, the feature that makes tiny GPUs viable, is still labeled BETA and has to be turned on by hand.
Getting Started
- Install the package. Use
pipx install "soup-cli[train]"oruv tool install "soup-cli[train]". The baresoup-cliinstall is a light CLI without PyTorch, so you need the[train]extra to fine-tune. - Generate a config. Run
soup init --template chatto create a startersoup.yamlfor a chat-style fine-tune. - Point it at your data and base model. Edit the YAML to set your dataset path and the model you want to adapt.
- Start training. Run
soup trainand let it detect your GPU and pick the settings. For big models on small cards, setstream_layers: truefirst. - Serve or export the result. Use
soup serveto expose the tuned model behind an OpenAI-compatible API endpoint.
Product Information
A quick look at Soup CLI's pricing, supported platforms, and performance.
Best for
The users, tasks, and scenarios where this tool fits best.
Users
- Solo developers and hobbyists
- ML engineers and researchers
- Small startups
Tasks
- Supervised fine-tuning
- Preference optimization
- Local experimentation
Scenarios
- A laptop-only workflow
- Private fine-tuning
- Turning a tuned model into a service
Key features
One Config, One Command
The core idea is that a single YAML file describes the whole job. You run soup init to generate a starting config, edit a few fields, then run soup train. Soup reads the config, checks your GPU, and picks a batch size and quantization setting that fit. For anyone who's maintained a folder of training shell scripts, that's the appeal: the config is the documentation.
Layer Streaming for Tiny GPUs
Layer streaming is the feature that gets the most attention. Instead of loading the entire frozen base model into VRAM, Soup keeps it out of memory and feeds the GPU one decoder layer at a time. On an RTX 3050 Laptop with 4 GB, the project reports an 8B model running at 3.32 GB peak memory. Big result. The catch is that it's opt-in and still BETA, and the throughput numbers come from a specific older version, so treat the speed as a floor rather than a promise.
Built-In Evaluation and Checkpoint Gating
Soup doesn't stop at training. It derives evaluations from your setup and gates checkpoints against them, so a run that regresses gets flagged instead of silently saved. For teams that train often, this matters more than raw speed. You want a pipeline that tells you when a checkpoint is worse than the last one.
OpenAI-Compatible Serving
The soup serve command turns a trained model into an API service with one command. It exposes the standard /v1/chat/completions, /v1/models, and /health endpoints, supports streaming responses, and can switch to vLLM or SGLang backends for higher throughput. LoRA adapters can be served directly, with the base model resolved from the adapter config, so you don't have to merge weights by hand first.
Local-First, No Cloud Required
Everything is designed to run on hardware you already own. No cloud required. There's no mandatory account, no hosted dashboard, and no data leaving your machine unless you set it up to. For teams handling sensitive data, that's often the deciding factor over a managed training service.
Data Tooling and Inspection Commands
The light soup-cli install still includes soup data ... and inspection commands that work without PyTorch. That means you can clean and inspect your dataset on a machine that has no GPU at all. Prep now, train later.
Pros and cons
Pros
- Open source under Apache-2.0, so there's no vendor lock-in and the source is fully auditable.
- A single YAML config replaces most of the scripting and setup that fine-tuning usually demands.
- Layer streaming makes genuinely large models trainable on small, consumer GPUs.
- Serving and evaluation are part of the same tool, not separate utilities you have to wire together.
- Works fully offline, which keeps data and weights on your own hardware.
Cons
- Layer streaming, the standout feature, is still BETA and must be enabled manually, so it isn't a set-and-forget choice.
- Best performance assumes a CUDA GPU; Apple Silicon and CPU support is experimental and slow.
- The split install can confuse newcomers, since a plain `pip install soup-cli` won't let you train until you add the `[train]` extra.
- Documentation is dense and version-specific, so advice from older guides may not match the current release.
Frequently asked questions
It's an open-source command-line tool for fine-tuning and post-training large language models. You define a job in one YAML file and run soup train, and it handles GPU detection, quantization, and batching for you.
Related content
Explore related tools, skills, and articles for Soup CLI.
Soup CLI Alternatives
Base Code
Base44 (Wix) · Coding · ProductivityBase Code is a shared cloud development environment from Base44. Teammates describe changes to an existing GitHub codebase in chat, preview them live and send them back as pull requests.
Google EmbeddingGemma 2
Google DeepMind · Writing · Image · CodingGoogle EmbeddingGemma 2 is a 740M-parameter open model that turns text, code, images, audio, and video frames into vectors in one 768-dimension space, running on phones and laptops.
GitGlow
GitGlow · Coding · ProductivityGitGlow is a local-first desktop Git client that lets independent developers review their own code and their AI agents' changes in one place before shipping.