runtime · Ollama
Ollama
Ollama runs open large language models locally from the command line and exposes them through an OpenAI-compatible API on localhost:11434. Free and open source, it handles model download, quantization, and GPU acceleration for you, and runs as a long-lived service — a private endpoint your scripts, apps, or whole lab can call without sending data to the cloud or paying per token. It carries a curated model registry plus any GGUF model. For a biology student it is the way to give analysis scripts a local LLM: point your Python at localhost and have it summarize results or draft figure captions, fully offline.
Docker image ollama/ollama:latest · Original container source
What it can help with
- local llm
- openai-compatible api
- offline inference
- gpu acceleration
- model quantization
- command-line interface