A developer ran a 744-billion-parameter AI model on a consumer machine with no GPU. The project is called Colibrì. The model is GLM-5.2. Both are free.
The project is brand new, the code is open source, and the implications are significant: every API call you make to a cloud provider is a cost you could avoid with the right local setup.
What This Actually Means
The standard assumption in AI infrastructure is that large models require large GPUs. A 744B model would normally need a server rack with tens of thousands of dollars in hardware.
Colibrì breaks that assumption by not loading the full model at once.
GLM-5.2 is a Mixture-of-Experts architecture. Per token, only about 40 billion of the 744 billion parameters activate. The "active" portion of the model, the dense layer handling attention and embeddings, occupies about 9.9GB in RAM at int4 quantization.
Colibrì keeps that 9.9GB portion in RAM and streams the 21,504 expert modules from disk on demand. Think of it the way Netflix streams video: instead of downloading the entire library before you watch anything, you pull only what you need, exactly when you need it.
The disk holds 370GB of quantized model weights. The RAM needs to hold roughly 9.9GB plus the expert cache. A machine with 16GB RAM can run it. 25GB gives you better cache performance.
No GPU involved. Pure C. Zero external dependencies.
What You Need Before You Start
Before running Colibrì, confirm you have:
- At least 16GB of system RAM (25GB or more recommended for smoother performance)
- 370GB of free disk space for the model weights
- A machine running macOS or Linux
- GCC and OpenMP available (the setup script checks for both and stops if either is missing)
- A reasonably fast SSD. A slow spinning drive will produce slow token generation due to the streaming architecture.
How to Set It Up
Step 1. Clone the repository.
Copy this.
git clone https://github.com/alexandreofbh/colibri_GLM-5.2 cd colibri_GLM-5.2/c ./setup.sh
The setup script checks for required dependencies, builds the binary, and runs self-tests. If it exits cleanly, the binary is ready.
Step 2. Download the model weights. The model is 370GB in int4 format. Download instructions are in the repository. This is the longest step: budget time based on your connection speed.
Step 3. Run it. Set the model path in your shell environment:
Copy this.
export COLI_MODEL=/path/to/GLM-5.2-colibri-int4
Then start an offline chat:
Copy this.
./coli chat
Or spin up an OpenAI-compatible local API:
Copy this.
COLI_API_KEY=local-secret ./coli serve --host 127.0.0.1 --port 8000
The serve command turns your local machine into an OpenAI endpoint. Point any OpenAI-compatible client at http://127.0.0.1:8000 and use your COLI_API_KEY as the API key. Change one line in your existing code and the token bill goes to zero.
You can also run a system readiness check before the first chat:
Copy this.
./coli doctor
It reports whether your RAM, disk speed, and dependencies are suitable for the model.
Done.
What It Costs Per Token
Zero. Nothing leaves your machine. The model runs entirely on local hardware. There are no API calls, no rate limits, no billing.
The cost is the hardware you already own plus the electricity to run it.
What You Get Back
- A 744B-parameter model running fully offline on consumer hardware
- An OpenAI-compatible local endpoint for drop-in replacement of cloud API calls
- Zero token cost per query after setup
- Full privacy: no data leaves the machine
- Engine-neutral deployment: the same code that calls OpenAI or Claude API works with one endpoint change
Honest: The Real Requirements
This is not a setup for any laptop. The minimum 16GB RAM requirement rules out older MacBook Pros and any machine that shares RAM with a GPU. A MacBook Pro with 24GB unified memory or a Mac Mini with 16GB should handle it. A 32GB machine gives you comfortable headroom.
The 370GB model download is a real commitment. You need both the disk space and the patience for a large file transfer.
Token generation speed depends heavily on SSD read speed. An NVMe SSD produces acceptable performance. A slower drive will noticeably limit throughput.
GLM-5.2 is a strong open model, not a guaranteed match for the frontier commercial models in every task. Test it on your specific use cases before committing infrastructure around it.
One Action This Week
Run ./coli doctor before you do anything else. It tells you in 30 seconds whether your machine meets the requirements. If it passes, the model download is the next step.
If it fails on RAM, you know the minimum hardware upgrade to make this viable. That is a useful number to have before deciding whether local inference is the right path for your business.
The repository is linked at the bottom of this page.