Ollama

Ollama is an open-source tool that lets developers and users run large language models locally on their own hardware with a single command. It provides a simple CLI, a REST API compatible with the OpenAI API format, and a curated model library covering hundreds of models including Llama, Mistral, Gemma, Qwen, and more. Ollama supports GPU acceleration via NVIDIA CUDA, AMD ROCm, and Apple Metal, making local inference fast and accessible across macOS, Linux, and Windows.