Colibri Runs GLM-5.2 Locally on 25GB. Here's the Catch.
A single-file C engine is streaming a 744B-parameter model off an NVMe drive on hardware you already own. I read the repo like an SRE — the receipts are real, and so is the speed tax.
- Colibri runs GLM-5.2 — a 744B-parameter MoE model — on a machine with ~25GB of RAM by streaming its 19,456 routed experts from a ~372GB NVMe container. No GPU, no Python at runtime.
- Author-reported speeds: 0.05-0.1 tok/s cold on the 25GB dev box, ~1.8 tok/s warm on a 128GB CPU desktop, 5.8-6.8 tok/s decode on 6x RTX 5090.
- The MTP speculative-decoding head must be int8 — the original HF mirror ships int4 heads that collapse to 0-4% draft acceptance. Use the mateogrgic mirror.
- Repo went from created (July 1) to 936-point Show HN (July 9) to 17,831 GitHub stars by July 22. Apache 2.0 engine, MIT weights.
Colibri is a pure-C, Apache 2.0 inference engine that runs GLM-5.2, Z.ai's 744-billion-parameter MIT-licensed MoE model, locally on a consumer machine with roughly 25GB of RAM and 400GB of fast NVMe. It keeps the ~17B dense parameters resident at int4 and streams the 19,456 routed experts from disk, with no GPU and no Python at runtime.
Can you run GLM-5.2 locally — all 744 billion parameters — on a machine with 25GB of RAM and no GPU? Colibri says yes, and the receipts in the README hold up. It’s a pure-C engine that keeps the dense ~17B parameters in RAM at int4 (~9.9GB) and streams the rest off your NVMe drive. No Python at runtime. No BLAS. No CUDA.
Let me be straight with y’all: I have not run this on my own metal yet. I don’t have 400GB of NVMe sitting free. This is me reading the repo like an SRE, not faking a benchmark. But I’ve been the loudest GLM 5.2 evangelist on this site since the Fable blackout, when I picked it up through Cline for $2 and watched it do savage pentest work on my own apps. I said then we’d all end up “running back to torrents… on home hardware because the government doesn’t want us to run the latest models.” Colibri is that prediction with a Show HN post.
How does Colibri fit a 744B model in 25GB of RAM?
Sparsity does the work. GLM-5.2 activates only ~40B of its 744B parameters per token, and only ~11GB of that actually changes token-to-token — the routed experts. Colibri keeps the stable dense part resident and streams the 19,456 routed experts (75 MoE layers x 256, plus the MTP head, ~19MB each at int4) from a ~372GB disk container on demand.
That’s the whole trick, and it’s a beautiful one. The router says which experts it needs; the engine pages them in like a database working set. One C file (~6,700 lines as of v1.1.0) plus small headers. The weights themselves are MIT from Z.ai, engine Apache 2.0. Started as a one-person project on a 12-core laptop. Repo created July 1… 936 points on Hacker News July 9… 17,831 stars by July 22. Crazy work.
How fast is GLM-5.2 running locally with Colibri?
Slow at the floor, usable at the ceiling — and the README is refreshingly honest about the ladder. These are the author’s self-reported numbers, not mine and not independent: 0.05-0.1 tok/s cold on his 25GB dev box, up to 5.8-6.8 tok/s decode across six RTX 5090s.
| Hardware | Author-reported speed | Where it loses |
|---|---|---|
| 25GB RAM, 12-core CPU (cold) | 0.05-0.1 tok/s | Interactive anything. That’s ~3,000 tokens in an 8-hour overnight run. |
| 128GB CPU-only desktop (warm) | ~1.8 tok/s | Still no chat pacing — but 50k+ tokens overnight is real batch work. |
| Single RTX 5070 Ti laptop-class box | 1.07 tok/s | GPU barely helps until experts get residency. |
| 6x RTX 5090, full residency | 5.8-6.8 tok/s | If you own six 5090s, you have other options. |
Compare the other local path: Unsloth’s official GLM-5.2 doc puts its 1-bit quant at 223GB of combined RAM+VRAM, 2-bit at 245GB. Different lane entirely — Unsloth wants you rich in memory; Colibri wants you rich in patience.
What’s the catch when you download GLM-5.2 for Colibri?
The MTP head. Colibri’s speculative decoding needs the multi-token-prediction head at int8 — int4 MTP heads collapse to 0-4% draft acceptance, which kills the 2.2-2.8 tokens-per-forward payoff. The README warns the original mirror ships broken int4 heads. Download mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp — ~372GB, keep 400GB free on fast NVMe.
Peep game: that one quantization detail is the difference between speculative decoding paying and doing nothing. Read the README before you burn a weekend of bandwidth.
Should you run GLM-5.2 locally with Colibri?
If you want a frontier-class coder answering in real time, no — my open-weight coding models guide covers API routes that cost pennies. And if streaming experts off NVMe is more pain than you signed up for, my August local LLM rankings cover what actually fits in consumer VRAM. If you want the strongest MIT-licensed model on Earth running on hardware nobody can revoke, throttle, or export-control… this is the door, and it’s Apache 2.0.
I got my start in Ops. Availability you own beats speed you rent.
The hummingbird is slow. It’s also yours.
#TheAIMogul
Bottom lineColibri is the most important local-AI repo of the summer: it proves the strongest MIT-licensed open-weight model can run on hardware you already own, if you'll trade speed for sovereignty. Download it for overnight batch work and the principle — not for interactive chat.