AnythingLLM — Quick-reference

Private, all-in-one ChatGPT for your documents · open-source (MIT) · local or your own cloud key · desktop app + self-hosted Docker server

What it is (vs Ollama / LM Studio / Jan)

Those tools run models. AnythingLLM adds document-RAG and AI agents on top: drop in PDFs, Word docs, CSVs, codebases, or whole folders and chat with them — with in-chat citations — in a ChatGPT-style UI. Point it at a local model and nothing leaves your machine; documents are stored and processed locally by default. Made by Mintplex Labs.

Two ways to run it

Build	Who for	Has
One-click desktop app	Single user	Mac / Windows / Linux installer · workspaces · document chat · agents
Self-hosted Docker server	Teams / lab	Multi-user · developer API · OpenAI-compatible endpoint

The OpenAI-compatible API and multi-user features are in the Docker server build, not the desktop app.

Desktop quick actions

Setup

Install & pick a provider

Download from anythingllm.com → run installer → on first run choose an LLM provider: built-in local, Ollama / LM Studio, or a cloud key (OpenAI, Anthropic, Azure, AWS, Gemini).

Sidebar

New Workspace

Click New Workspace. Each workspace containerises its own documents into a separate thread so context stays clean.

Workspace

Upload documents

Drop in PDFs, Word docs, CSVs, codebases, or a whole folder. AnythingLLM reads and indexes them locally.

Chat

Show Citations

Under any answer, open Show Citations to see the source documents it was drawn from — verify before you trust.

Chat

New Thread

Start a fresh thread inside a workspace for a distinct line of questioning; documents stay shared across threads.

Message bar

Invoke an agent

Type @agent at the start of a message to run a built-in AI agent with customisable skills.

Two chat modes (per workspace)

Mode	Behaviour
Query	Answers strictly from your documents; says so when it has no relevant context. Use when you need answers you can cite.
Conversation	Blends the model's general knowledge with your documents. Use as a brainstorming partner that also knows your files.

Agents & Community Hub

Run an agent in chat

@agent summarise the newest document
in this workspace and list its three
key claims.

Community Hub

Import and share agent skills, system prompts, and slash commands. Set a workspace system prompt to shape behaviour (e.g. "always answer from the uploaded documents and cite them").

Self-hosted Docker server

Run the official mintplexlabs/anythingllm image on a spare machine for a shared, multi-user private knowledge base. See docs.anythingllm.com for the current run command, volume mount, and port mapping — then open the web UI it serves, pick a provider, and create workspaces exactly as in the desktop app.

Step	Where
Run the container	Docker host — image `mintplexlabs/anythingllm`
First-run setup	Web UI → choose LLM provider → create workspace
Add users	Server settings — multi-user, each with their own login
Generate an API key	Server settings → developer API
Call the OpenAI-compatible endpoint	Swap base URL + key in existing OpenAI client code

OpenAI-compatible API (Docker build)

The server build exposes an OpenAI-compatible endpoint so existing OpenAI client code can target your private, document-grounded workspace by swapping only the base_url and key. Confirm the exact endpoint path and request shape in the docs before you wire anything up: docs.anythingllm.com/features/api.

# pip install openai — point the client at your AnythingLLM server.
# Check docs.anythingllm.com/features/api for the exact base URL path.
from openai import OpenAI

client = OpenAI(
    base_url="http://<your-server>/api/v1/openai",
    api_key="<your-anythingllm-api-key>"
)

response = client.chat.completions.create(
    model="<your-workspace>",
    messages=[
        {"role": "user", "content": "Summarise the uploaded papers' main findings in 3 bullets."}
    ]
)
print(response.choices[0].message.content)

When to reach for AnythingLLM

Situation	Use
Chat with your own PDFs / docs and get citations	AnythingLLM workspace
Sensitive or unpublished documents — nothing should leave the machine	AnythingLLM + a local model
A whole lab needs one shared private knowledge base	AnythingLLM Docker server (multi-user)
Existing OpenAI code should answer from your documents	Docker server's OpenAI-compatible API
You just want to browse and download raw models	Ollama or LM Studio go deeper