A 2026 tutorial walks through running GGUF models locally with llama.cpp on CPU and GPU. Georgi Gerganov's C++ port of the Llama architecture is now the default way to run quantized models without cloud APIs for local workflows. ExLlamaV2 wins on raw speed for NVIDIA hardware. Optimized CUDA kernels help there. But llama.cpp holds its own on CPU-only machines and Apple Silicon with Metal support. The real advantage is partial offloading. It splits model layers between GPU VRAM and system RAM. A 13B parameter model that needs 8GB of VRAM? Offload what fits. The rest runs on system RAM. Pure GPU loaders choke here. GGUF is now the community standard for quantized model distribution. It beats GPTQ and AWQ on metadata handling and cross-platform support. Ollama builds on llama.cpp to simplify things. But the core engine remains the flexible choice across Windows, Linux, and mobile. Partial offloading is why llama.cpp stays relevant. It runs models your hardware can't handle.