Best Local LLMs, August 2026: What Actually Runs On Your GPU
Forget the leaderboard screenshots. Here's what open-weight model fits in 16GB, what needs 24GB, and what needs a rack — with the quant files and VRAM math to prove it.
- Qwen3.6-27B and the just-shipped Qwen3.8-27B (Aug 14) are the sweet spot for one consumer GPU — Q4_K_M quants run in ~17-19GB, so a single RTX 4090 or 3090 handles it.
- gpt-oss-20b (OpenAI, Apache 2.0) needs only 16GB and gpt-oss-120b fits one 80GB card — still the easiest on-ramp if you don't own a 4090.
- GLM-5.2 (744B MoE, MIT) and Kimi K2.6 (1T MoE) are genuinely frontier-adjacent, but neither fits on a single consumer GPU — GLM-5.2's smallest usable quant needs ~241GB, Kimi K2.6 needs 350GB+, both requiring CPU offload or multi-GPU rigs.
- DeepSeek-V4-Flash (284B total, 13B active, MIT) is the practical middle ground for anyone with a multi-GPU workstation but not a server room.
As of mid-August 2026, the best local LLMs by hardware tier are: Qwen3.8-27B or gpt-oss-20b for a single 16-24GB consumer GPU, DeepSeek-V4-Flash (13B active params) for multi-GPU workstations, and GLM-5.2 or Kimi K2.6 for anyone with 240GB+ of pooled VRAM/RAM. Match the model to your hardware first, capability second.
I get some version of “just run it locally” in my DMs every time a model drops. Half the time the person asking doesn’t own the hardware for it. So here’s the roundup I actually wanted: not “is this model good,” but what does it cost you in VRAM to find out.
This is the practical companion to the open-source vs. frontier capability gap piece — that one covers how close open-weight gets to Fable 5 and GPT-5.6 on benchmarks. This one covers whether the model fits the card you own.
The lineup, by what it actually takes to run
| Model | Size | Min hardware (real) | Best for |
|---|---|---|---|
| gpt-oss-20b (OpenAI, Apache 2.0) | 20B, MoE | ~16GB VRAM — one consumer GPU | Laptops, older cards, quick local iteration |
| Qwen3.8-27B (Aug 14, 2026, Apache 2.0) | 27B dense | ~17GB VRAM at Q4_K_M — RTX 4090/3090 | Single-GPU daily driver, coding + general use |
| Qwen3.6-27B (Apache 2.0) | 27B dense | ~18GB total at Q4_K_M/UD-Q4_K_XL | Same tier as 3.8, still solid if you’re already on it |
| gpt-oss-120b (OpenAI, Apache 2.0) | 120B, MoE | One 80GB GPU (H100/A100-class) | Workstation-tier reasoning without a cluster |
| DeepSeek-V4-Flash (MIT) | 284B total / 13B active, MoE | Multi-GPU workstation; 13B active keeps inference light | Best MoE without a server room |
| GLM-5.2 (MIT) | 744B total, MoE | ~241GB VRAM/RAM at Unsloth dynamic 2-bit | Long-horizon coding, 1M-token context, if you can pool memory |
| Kimi K2.6 (Modified MIT) | 1T total / 32B active, MoE | 350GB+ even at aggressive 2-bit — no single consumer box | Agentic + multimodal work, datacenter or heavy CPU-offload rigs only |
Every VRAM figure above is from actual quant releases (mostly Unsloth’s GGUF builds), not a vendor’s marketing page — model creators rarely publish “here’s what fits on a 4090,” so that number comes from the community’s quant math after release.
The single-GPU tier: this is where most people should stop
If you own one card, GLM-5.2 and Kimi K2.6 aren’t in your consideration set — not because they’re bad, because they’re architecturally not for you. Qwen3.8-27B just shipped August 14 as a dense 27B model, Apache 2.0, and Q4_K_M lands around 17GB. That’s a comfortable fit on a 24GB RTX 4090 or 3090 with room for a real context window.
gpt-oss-20b is the other honest answer. OpenAI shipped it in August 2025 and it’s still relevant a year later precisely because 16GB is such a low bar — a laptop 4080 or aging 3080 handles it, no offload gymnastics. It won’t out-argue a 744B MoE on hard reasoning, but for daily coding help the gap is smaller than the parameter count suggests.
The MoE middle: DeepSeek-V4-Flash earns its keep
DeepSeek-V4-Flash is the model I’d point a small team toward with a real GPU budget, no data center. 284B total parameters, only 13B active per token, MIT, 1M-token context by default. “Active parameters” is what matters for speed — inference behaves closer to a 13B model than a 284B one, even though you still store the full weight set. That’s the MoE trade: cheap to run, expensive to hold.
The frontier-adjacent tier: real, but not for your desk
GLM-5.2 and Kimi K2.6 are where “open-weight” and “runs at home” stop meaning the same thing. GLM-5.2’s smallest usable quant — Unsloth’s dynamic 2-bit — still needs roughly 241GB of combined VRAM and RAM. Kimi K2.6, a full trillion parameters, needs 350GB+ even that aggressively quantized, and no single consumer card holds it. Both are real options if you’re pooling memory across GPUs with llama.cpp or KTransformers CPU offload, or renting datacenter capacity by the hour. Neither is a weekend download-and-go project.
Where local still loses to hosted
Be honest about the trade. None of these — not even GLM-5.2 at full precision — beats Fable 5 or Opus 4.8 on the hardest reasoning and agentic benchmarks, and running big MoEs locally means eating the hardware cost yourself instead of paying per-token. If your workload is spiky, a hosted API beats a 240GB rig sitting idle most of the week. Local wins on cost-at-volume, privacy, and not being at the mercy of export-control drama — not on capability per dollar for occasional use.
Match the model to the card you own, not the one in the benchmark screenshot. That’s the whole exercise.
#TheAIMogul
Bottom lineIf you own one GPU, run Qwen3.8-27B or gpt-oss-20b and stop shopping. If you're building a rig, DeepSeek-V4-Flash's 13B active params make it the best MoE that doesn't need a data center. GLM-5.2 and Kimi K2.6 are the real deal, but 'open-weight' doesn't mean 'runs on your desk' — check the quant math before you buy hardware.