Heidelberg AI Bench

AnythingLLM, by Mintplex Labs, is an all-in-one private AI workspace. You drop in PDFs, Word docs, CSVs, or a whole folder, and chat with them in a familiar ChatGPT-style interface that shows citations back to your source files. Its angle versus a plain chatbot is document RAG plus built-in AI agents bundled together. It is provider-agnostic: run a fully local model so nothing leaves your machine, or bring your own cloud key from OpenAI, Anthropic, Azure, AWS, or Gemini. It ships as a one-click desktop app for Mac, Windows, and Linux, and as a Docker server you can self-host as a shared team endpoint. It is open source under the MIT license.

Yes, the self-hosted server exposes a full developer API. The docs state you can use it to manage, update, embed, and even chat with your workspaces. You generate an API key inside the app, then call endpoints with that key in the request header; accounts with the right access level can create and delete keys on the fly. The complete, browsable endpoint reference for your specific instance is served at /api/docs on the running server. The server build also exposes an OpenAI-compatible endpoint at /api/v1/openai, so existing OpenAI client code can target your private, document-grounded model by swapping the base URL. Keep your API key secret, since anyone holding it can use your instance.

Ollama and LM Studio are model runners: their job is to download, manage, and serve local LLMs, and they go deeper on browsing and one-click model downloads. AnythingLLM sits a layer above that. It is not primarily about managing raw models; it turns a model, local or cloud, into a private assistant over your own knowledge, with document RAG, citations, workspaces, and AI agents built in. In fact AnythingLLM can use Ollama or LM Studio as its underlying model provider. So a common setup is to run models in Ollama or LM Studio and use AnythingLLM as the document-chat and workspace layer on top. Pick AnythingLLM when the goal is chatting with your documents, not just running a model.

AnythingLLM is free and open source under the MIT license. Both the one-click desktop app and the self-hosted Docker server cost nothing to download and run, and the desktop app needs no account. The only thing you might pay for is an optional cloud LLM provider: if you add a key for OpenAI, Anthropic, Gemini, or similar, you pay whatever that provider charges per token. If you instead use a fully local model (the built-in option, or via Ollama or LM Studio), there is no usage cost at all. There is no AnythingLLM subscription or per-seat fee for running it yourself.

The fastest path is the desktop app. Download the one-click installer for Mac, Windows, or Linux from anythingllm.com and run it; no account is required. On first run, pick an LLM provider. Choosing the built-in local provider downloads a small model so nothing leaves your machine, which is the zero-setup option; you can instead point it at Ollama, LM Studio, or a cloud key. Then click New Workspace in the left sidebar, name it, and upload a few PDFs. AnythingLLM reads and indexes them locally. Ask a question grounded in those documents and you have a private document chat running in minutes.

It depends on the model you pick. When you point AnythingLLM at a local model, whether the built-in one or via Ollama or LM Studio, nothing leaves your machine: your documents are stored and processed locally by default, and the desktop app is built as a single-player application offering full privacy. If you instead add a cloud key for OpenAI, Anthropic, or Gemini, then your prompts and the relevant document text are sent to that provider to generate answers, just as with any cloud LLM. So privacy is a choice you control per workspace by choosing local versus cloud. For sensitive material, use a local model.

A workspace containerises a set of documents into its own chat thread, so the context for one project never bleeds into another. Instead of one giant pile of files where the model has to guess which of fifty papers you mean, you give each project its own workspace: one for a thesis chapter, one for a side project, one for course reading. Each answers only from its own files, which keeps answers focused. Within a workspace you can also start separate threads for distinct lines of questioning while the documents stay shared. Think of workspaces as top-level folders and threads as conversations inside them.

A Docker container is an isolated process that runs like a regular program but has its own filesystem and network view, while sharing the host’s Linux kernel. This isolation gives fast startup and low overhead compared to a full virtual machine.

Docker first checks for a local copy of the requested image; if it’s missing, Docker automatically downloads (pulls) it from Docker Hub or another registry before starting the container. This ensures the command always has what it needs.

Running `docker run hello-world` prints “Hello from Docker!” and then exits; seeing this output confirms the container launched successfully and demonstrates Docker’s quick start‑up time.

An image is a read‑only snapshot of a filesystem that contains everything needed to run an application. When you start a container, Docker adds a writable layer on top of that image, creating a temporary runtime instance.

Yes, each time you start a container from an image it gets its own isolated filesystem and resources, so you can have several independent instances running side by side.

Use `docker rm -f <container-name>` to force‑remove the container; this deletes its writable layer and frees the resources, ensuring it no longer appears in `docker ps`.

LM Studio is a free desktop application that lets you download and run AI language models directly on your own computer — no internet needed once a model is downloaded. It provides a friendly graphical interface similar to ChatGPT, but everything runs locally. It works on Mac (Apple Silicon), Windows, and Linux.

Because LM Studio's local server mimics the OpenAI API, many third-party tools connect to it out of the box — for example Obsidian, AnythingLLM, Continue.dev, Open WebUI, and n8n. LM Studio also supports the MCP protocol for connecting AI agents to external data sources.

LM Studio is a graphical desktop app — you click buttons, browse models visually, and chat in a built-in interface, friendlier for beginners with no coding. Ollama is command-line first and better for scripting or automation. Both use the same underlying engine (llama.cpp), so quality and speed are comparable; the difference is interface and workflow. A common pattern is LM Studio for exploring models and Ollama for integrating into code.

Yes, after the initial download. You need internet once to install LM Studio and once to download each model. After that, the app and all your models run entirely offline — useful in settings with restricted internet, or when you want guaranteed privacy.

The number (7B, 13B) refers to billions of parameters — a rough measure of capacity. Bigger models generally give smarter answers but need more RAM and run slower. For most everyday tasks on a typical laptop, a 7B or 8B model at Q4_K_M quantization is a practical starting point. LM Studio's Discover tab warns you if a model is likely too large for your hardware.

Slow output usually means the model is too large for your GPU and is spilling into RAM — try a smaller model or a lower quantization (Q4 instead of Q8). Strange or repetitive output is often the wrong chat template; LM Studio normally auto-detects it, but if it looks broken, check the correct template is selected in the model settings. Out-of-memory errors are solved by a smaller model variant.

Yes. Since mid-2025, LM Studio is free for both personal and workplace/commercial use, so you can run it in a lab or office without a license fee. The models themselves are governed by their own licenses (most popular ones permit research use freely). Organizations needing team-sharing or enterprise controls have optional paid tiers, but they're not required for standard use.