LLM Engineering •
Run Kimi K3 Locally: 2.8T Params From 4 SSDs at 1 Tok/s
Run Kimi K3 locally on a MacBook by streaming 1.45TB of experts from four SSDs — the 1 tok/s number, and why doubling drives doesn't double speed.
7 min read
Read more →Run Kimi K3 locally on a MacBook by streaming 1.45TB of experts from four SSDs — the 1 tok/s number, and why doubling drives doesn't double speed.
LLM inference in macOS VMs collapses to 12.63 tok/s because the guest reports GPU family 5 and llama.cpp drops its matrix kernels. The check, and its limits.