What is ollama Ollama is a tool used to run large language models locally. Instead of relying on cloud providers, developers can download and run models like Llama, Mistral, Gemma, and others directly on their own systems. It exposes an API server that allows applications to: load AI models, generate responses, create custom models, and share or push models to remote registries. Its API is a local HTTP REST API, usually running on: