Most companies that want a custom AI model end up in cloud pricing hell. You need a GPU instance. You need SSH access to a remote box. You need someone who can configure quantization, batch size, and memory layout without breaking the training run. Then you get a $400 cloud bill for a weekend of experiments and nothing deployable to show for it.
That assumption is wrong in 2026.
A solo founder built a tool that runs the entire fine-tuning pipeline on a 4 GB laptop GPU, at 119 tokens per second, on a model with 8 billion parameters. One YAML config. One command. No cloud required. The numbers are measured, not claimed, and the paper is published.
The Tool: Soup
Soup is an open-source CLI for fine-tuning and post-training large language models. The pitch: "fine-tune and post-train LLMs in one command. No SSH, no config hell."
Free. Apache-2.0. Available at github.com/MakazhanAlpamys/Soup.
The core feature is called layer streaming. Standard fine-tuning loads the entire model into VRAM at once. An 8B model needs roughly 16 GB of VRAM at minimum with normal approaches. Layer streaming keeps the frozen base model in host RAM and feeds it to the GPU one decoder layer at a time. Only the adapter (the small trainable layer) lives in VRAM. The result: Llama-3.1-8B-Instruct trains at 3.32 GB peak on an RTX 3050 Laptop 4 GB, at 119.6 tokens per second. That result was reproduced independently on an H100 at 113.00 tok/s at the same 3.32 GB peak. The underlying paper is published on Zenodo.
Layer streaming is opt-in and still marked BETA. Set stream_layers: true in your config to enable it. Without it, you still get QLoRA fine-tuning with the normal VRAM requirements listed below.
| VRAM Available | Max Model Size (QLoRA 4-bit) | Examples |
|---|---|---|
| 4 GB (with layer streaming) | 8B | Llama-3.1-8B, Mistral-7B |
| 8 GB | ~7B | Llama-3.1-8B, Mistral-7B |
| 16 GB | ~14B | Phi-4-14B, Qwen2.5-14B |
| 24 GB | ~34B | CodeLlama-34B, Yi-1.5-34B |
| 48 GB | ~70B | Llama-3.3-70B |
Soup works with any text-generation model on HuggingFace that loads through AutoModelForCausalLM. Over 100 models ship as ready-made recipes (soup recipes list). Supported training tasks include SFT, DPO, ORPO, GRPO, KTO, SimPO, IPO, and BCO.
Setup: Three Steps to Your First Run
Python 3.10, 3.11, or 3.12 required. Python 3.13+ is not supported yet.
Step 1. Install
Copy this.
pip install "soup-cli[train]"
Use double quotes. Single quotes break in cmd.exe and PowerShell. The [train] extra pulls in PyTorch, Transformers, PEFT, TRL, and the datasets library. Without it, you get the CLI and data tools but not the training stack.
Step 2. Create a config
Copy this.
soup init --template chat
This generates a soup.yaml file. Run soup init without a template for an interactive wizard instead. Templates available: chat, code, tool-calling, medical, reasoning, vision, kto, orpo, simpo, and more.
Edit the generated config to point at your data and pick a base model:
Copy this.
base: meta-llama/Llama-3.1-8B-Instruct
task: sft
data:
train: ./data/train.jsonl
format: alpaca
val_split: 0.1
training:
epochs: 3
lr: 2e-5
batch_size: auto
lora:
r: 64
alpha: 16
quantization: 4bit
stream_layers: true # enable for 4 GB cards
seed: 1234
output: ./output
Run soup doctor before training if you want to verify your GPU, system resources, and dependencies in one pass. It catches problems before a full run.
Step 3. Train
Copy this.
soup train --config soup.yaml
Batch size, GPU detection, and quantization are handled automatically. Your adapter weights land in ./output.
Done. The model is trained.
After Training: The Full Lifecycle
Soup covers the pipeline past the training step:
Copy this.
soup chat --model ./output # interactive test soup merge --adapter ./output # merge LoRA into base soup export --model ./output --format gguf --quant q4_k_m # export for Ollama / llama.cpp soup serve --model ./output # OpenAI-compatible API server soup eval benchmark --model ./output # evaluate against standard benchmarks soup push --model ./output --repo you/my-model # push to HuggingFace
Export targets include GGUF, ONNX, TensorRT, AWQ, and GPTQ.
What You Can Actually Do with a Custom Model
Training your own model produces a business asset. Here are use cases that map to operations problems, not toy demos.
Customer support trained on your actual docs. Take your help center content, product FAQs, email templates, and resolved ticket history. Train on that. The result handles routine support at the level of someone who has read every document, not just matched a keyword.
Internal knowledge retrieval. Search surfaces keywords. A fine-tuned model surfaces answers. Train on your internal wikis, SOPs, and reports. Your team asks in plain language and gets direct, contextual responses rather than a list of links.
Sales conversation work. Train on your best call transcripts and closed deals. The model learns your actual playbook. Not a generic script pulled from a marketing textbook.
Proposal and quote drafting. Fine-tune on your approved proposals and scope documents. Feed it a new client brief. The draft matches your real format and pricing logic. Your team edits rather than writes from scratch.
Compliance and policy review. For regulated industries, a model trained on your specific compliance framework responds differently from a general-purpose one. It flags deviations against your actual rules, not a generic interpretation.
The cloud alternative costs $0.01 to $0.12 per 1,000 tokens at inference, compounding indefinitely. A local fine-tuned model running on your own hardware costs the electricity.
The Limits. Read These Before You Start.
Layer streaming is BETA. The 119.6 tok/s benchmark was measured on v0.72.2, before a correctness repair in v0.73.0 that cost approximately 4.8% throughput at 32B. The 4 GB card has not been re-run since that repair. The README is clear about what was measured and when.
Layer streaming costs time, not memory. DPO with layer streaming reads the layer stack 1.52 times per step compared to SFT. Memory usage stays flat. Run time increases.
Python 3.13+ is not supported. CI does not test it. PyTorch wheels on 3.13+ have crashed at runtime. Use 3.10, 3.11, or 3.12.
Apple Silicon users: check your quantization. Older versions silently downgraded quantization: 4bit to none on Apple Silicon because the device detection did not recognize MLX. This was fixed. If you ran an earlier version on an M-series Mac, your quantization setting may not have applied. Re-run with an updated install.
Your base model must work with AutoModelForCausalLM. If it does not, Soup will not load it.
CPU training is experimental. Quantization is disabled automatically on CPU. Training will complete, but very slowly. Treat it as a test path, not a production one.
Data quality controls output quality. Soup handles the infrastructure. Your training data determines the model's behavior. A clean 500-example dataset usually outperforms a noisy 10,000-example one. Audit your data before your first real run.
Multi-GPU and 8B+ validation require hardware the maintainer does not have. The project runs on a single 4 GB laptop. Some features affecting larger setups are gated behind help wanted issues pending hardware contributions. Know what's been tested at what scale.
One Action
Run soup doctor on the machine you plan to use.
Copy this.
pip install soup-cli soup doctor
It checks your GPU, system resources, dependencies, and installed version in one pass. That output tells you whether your hardware is a viable starting point before you write a single YAML line or download a base model.
If soup doctor comes back clean, you are two commands away from your first training run.