What Colibrì is (how it runs on 25 GB)
Colibrì's memory layout (only a small part of the 744B sits in RAM)
Colibrì is an inference engine, published on GitHub by open-source developer JustVugg, for running a 744B-parameter AI model called GLM-5.2 on an ordinary PC. GLM-5.2 is normally a model on the scale of dedicated servers and stacks of expensive GPUs, so Colibrì drew attention for making it run on "an ordinary PC with about 25 GB of RAM." Let us start with what it is and why it works.
The definition (a pure-C inference engine)
Colibrì is an inference engine that runs GLM-5.2 — a 744B-parameter MoE (mixture of experts, an AI whose interior is split into many specialist parts) — on a PC with about 25 GB of RAM. Colibrì's defining trait is that it drives a huge model with a single program written in C, using no GPU and no dedicated libraries.
"Run GLM-5.2 (744B-parameter MoE) on a consumer machine with ~25 GB of RAM — in pure C, with zero dependencies, by streaming experts from disk." — from the repository summary
The program itself is a single C file, c/glm.c of about 2,400 lines, plus small headers. It needs no external matrix library (BLAS), no Python at runtime, and no GPU. Typical software for running generative AI locally depends on Python and large libraries, whereas Colibrì is an extremely lightweight build that carries none of them.
"The engine is a single C file (`c/glm.c`, ~2,400 lines) plus small headers. No BLAS, no Python at runtime, no GPU required" — from the "The idea" section
Why 744B runs on 25 GB of RAM
The thing most people want to understand here is why a 744B model runs on a PC with so little RAM. The answer is that GLM-5.2 has an MoE (mixture-of-experts) structure. In MoE, the whole model is huge, but only a fraction of it actually works each time a single word (token) is produced. GLM-5.2 uses only about 40B of its 744B parameters per token, and only about 11 GB of that swaps in and out each time.
"A 744B Mixture-of-Experts model activates only ~40B parameters per token — and only ~11 GB of those change from token to token (the routed experts)." — from the "The idea" section
Colibrì exploits this property. It keeps only the shared core used by every token (about 17B parameters) resident in RAM, compressed to about 9.9 GB. The remaining 21,504 expert parts (about 370 GB total) stay on disk, and only the ones needed are read in on demand. This approach — not loading everything into memory, but streaming in just what is used from disk — is the heart of running a giant model on a small amount of RAM.
"the dense part (attention, shared experts, embeddings — ~17B params) stays resident in RAM at int4 (~9.9 GB); the 21,504 routed experts ... live on disk (~370 GB) and are streamed on demand" — from the "The idea" section
Colibrì's required specs and speed
Inference speed by environment (community measurements, tokens per second)
Bar length is tokens generated per second. Figures are from the measurement table in the official repository. Results vary widely with disk speed.
Before you try Colibrì, it helps to know the PC specs you need and the speed you will actually get. Because its way of running rests on reading from disk, speed depends heavily on disk performance.
Required hardware (RAM, SSD, CPU)
Colibrì runs on Windows 11, macOS, and Linux (or WSL2). What it asks for is at least 16 GB of RAM and a fast SSD (NVMe) to hold the roughly 370 GB model. No GPU is needed: with at least 16 GB of RAM and a fast SSD that can hold the ~370 GB model, it runs on the CPU alone.
"The engine needs: Linux (or WSL2), macOS, or Windows 11 natively (MinGW-w64); gcc with OpenMP, AVX2, ≥16 GB RAM, and the ~370 GB int4 model on a local NVMe" — from the requirements
One thing to watch is where the model lives. At about 370 GB it needs an SSD with generous free space. And because the engine reads from it constantly while running, it must sit on a fast SSD directly attached to the PC — not a slow external drive or a network location.
Measured inference speed (0.05–1 tokens per second)
Speed varies greatly by environment. On the author's slow test setup it runs at 0.05–0.1 tokens per second; on a machine with a fast SSD and plenty of RAM it climbs to around 1 token per second. Getting a few hundred characters of reply takes several to a dozen-plus minutes — this is not the snappy pace of a conversation.
The author does not hide the slowness. The official repository states plainly that "this is not fast." On that basis, it positions the value not in speed but in the fact that "a top-tier giant model answers correctly on a machine that costs less than the cooling fan of an expensive GPU like an H100."
"This is not fast. It is a 744B frontier-class model answering correctly on a machine that costs less than one H100 fan." — from "Honest numbers"
From experience running local LLMs, this speed range is less "a tool for talking in real time" and more "an experimental rig for trying a giant model on your own hardware, without network calls, even if it takes time." For practical uses of running AI locally in general, see also the practical guide to local LLMs.
How to use Colibrì and what to watch for
The rough steps to try Colibrì
./coli chatFinally, let us go over the flow of actually trying Colibrì and the caveats worth knowing before you install it. Understanding the nature of its speed and storage will save you needless trouble.
How to use it (trying it with a converted model)
You can convert the model yourself, but the easy way to try it is to obtain GLM-5.2 data already converted for Colibrì (int4 format, about 370 GB) and use that. Place it on a directly attached SSD, build the engine, point it at the model location, and you can start a conversation with ./coli chat. It also supports the standard chat communication interface (OpenAI-compatible), so you can connect to it from existing tools.
GLM-5.2 itself is a large-scale model in the GLM family published by China's Z.ai. The model and Colibrì are separate things; Colibrì is simply "the engine for running that model on a small PC."
Caveats to check before installing
There are mainly three things to check before installing. The main points to note are: "the speed is too slow for conversation," "you need a directly attached SSD that can hold about 370 GB," and "the drive heats up during long sessions." As above, the speed is not suited to real-time conversation. The storage, too, is impractical on a slow or networked drive, so a fast SSD is a prerequisite.
One more point: during long continuous use, the drive gets hot because it is read constantly. The official repository notes that while the reads themselves barely wear the SSD, prolonged heavy use calls for attention to heat and drive temperature. As for licensing, the engine is Apache 2.0, and the model (GLM-5.2) weights are published by Z.ai under MIT. If it fits your purpose, the realistic way to start is small, with a pre-converted model.
When you want a local AI to read a long document, converting it to Markdown first keeps the structure of headings and tables intact and makes it easier to handle. To clean up a web page into that form, the following tool helps.
