Run AI on your own machine — nothing leaves it
Run AI models on your own computer so your data never leaves it. Ollama is a local API + CLI, LM Studio is a polished desktop app, Jan is an offline-first, open-source ChatGPT alternative, and AnythingLLM turns any local (or cloud) model into a private assistant that chats with your own documents. → Pick by what you want: script it from the terminal → Ollama; a friendly desktop app with a local server your other tools can hit → LM Studio; the simplest one-click chat to get started → Jan; chat over your own files with citations → AnythingLLM.
Install Ollama for a command‑line API or LM Studio for a desktop app, then download a model and start it on your own computer so all processing stays in‑house.
Jan provides the simplest one‑click interface: install it, download a local model, and you can start chatting instantly without configuring servers or APIs.
Yes—by running Ollama, LM Studio, Jan, or AnythingLLM on your machine with the network off, all inference happens locally and no information leaves your device.
You learn how to check your computer’s RAM and GPU specifications to decide which model size it can handle.
Determine the suitable model tier for your machine based on its memory resources
List your total RAM, whether you have a discrete GPU (include its VRAM) or note unified memory on Apple Silicon, then state the highest model tier you can run and pick one specific model from that tier.Paste this into lmstudio’s chat input box. After submitting, verify that the RAM and GPU details match what you saw in Task Manager/About This Mac/Linux before confirming the suggested model.
free -h (Linux) to note total RAMnvidia-smi (NVIDIA) or recognise Apple‑Silicon uses unified memory to identify GPU capabilityJan is a desktop app that runs open‑weight LLMs entirely offline, providing a ChatGPT‑style interface without any cloud connection.
Launch a locally running chat model using Jan within 20 minutes
Explain RNA‑seq in two sentences for a biologist who has never heard of it.In Jan, click New Chat, choose the model you just downloaded, then paste the prompt into the input box and hit Enter. Watch that the response appears in the chat pane—no network activity indicator should show any external traffic.
Explain RNA‑seq in two sentences for a biologist who has never heard of it.LM Studio provides a point‑and‑click UI for downloading and running quantised GGUF models entirely on your machine.
Get a local model answering questions in under 20 minutes
Explain RNA‑seq in two sentences for a biologist who has never heard of it.In LM Studio, go to the Chat tab, select your downloaded model from the top dropdown, paste the prompt into the message box and hit Enter. Watch that a reply appears locally (you’ll see the text appear) and no network activity is triggered.
lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF, select the Q4_K_M variant and press DownloadOllama is an open‑source tool that runs LLMs locally via simple commands, handling download, quantisation and optional GPU acceleration.
Answer a question using a locally run LLM in under 20 minutes
ollama run llama3Open a terminal window and paste the command exactly as shown, then press Enter. Watch the console output – it will show the model downloading and finally present a >>> chat prompt indicating the local model is ready.
curl -fsSL https://ollama.com/install.sh | sh on Linuxollama run llama3 to download and start an interactive session>>> prompt, type your question, e.g., Explain RNA‑seq in two sentences for a biologist who has never heard of it./bye or Ctrl+DJan's whole point is privacy: because the model runs locally, you can use it on unpublished results, draft manuscripts, or other sensitive material that you would never paste into a cloud chatbot. The most convincing way to see this is to turn your network off entirely and watch Jan keep working. This is the tool for the work you cannot send to the cloud — it runs on a plane, in a no-internet lab, and never transmits your data.
Do this first Run a private AI assistant on your computer
Generate a response to confidential content without any network connection
Tighten this paragraph and flag anything that is unclear or unsupported:
We collected 200 blood samples from patients undergoing chemotherapy at three hospitals between January 2022 and June 2023. RNA was extracted using the RNeasy Mini Kit (Qiagen) and sequenced on an Illumina NovaSeq 6000. Differential expression analysis was performed with DESeq2, adjusting for age, sex, and treatment regimen.Paste the full text into Jan’s chat window after you’ve disabled Wi‑Fi or unplugged Ethernet. Send it and watch that a response appears even though the network indicator shows no connection.
One of LM Studio's most useful features is loading several models at the same time and sending the same prompt to all of them at once. For researchers this is genuinely powerful: you can test how different models handle a sensitive or technical question before committing to one. Everything stays offline — the comparison runs entirely on your machine.
Do this first Run a local model with LM Studio
Ask multiple models the same question and watch them disagree
I have 12 RNA‑seq samples across 3 conditions. What statistical approach should I use to find differentially expressed genes, and what are the main pitfalls?In LM Studio, go to the Chat tab, enable Multi‑model chat/Compare, add two models, then paste the prompt into the input box and hit Enter. Watch that both model panes display answers side‑by‑side so you can spot agreements and differences.
Ollama is CLI-first, and almost everything you need is a few short commands. ollama pull downloads a model without starting a chat; ollama list shows what you have; ollama ps shows what is currently loaded in RAM; ollama stop frees that RAM; and ollama rm deletes a model from disk. Knowing these five turns Ollama from a chat toy into a managed local model library you can keep tidy on a laptop with limited disk space.
Do this first Run your first local model
Download, list, load, unload and delete models using Ollama’s CLI
ollama pull qwen2.5 && ollama list && (gnome-terminal -- bash -c "ollama run llama3; exec bash") && sleep 5 && ollama ps && ollama stop llama3 && ollama ps && ollama rm qwen2.5 && ollama listPaste the whole line into a terminal session. It will pull a model, list your library, open a new terminal to load another model, then show RAM usage, stop it, and finally delete the first model. Watch for each command’s output to confirm the state changes.
ollama pull qwen2.5 to download a model without starting a chatollama list to see installed models with sizes and datesollama run llama3 in another terminal, then run ollama ps to view loaded models in RAMollama stop llama3 and verify removal via ollama psollama rm and confirm its absence with ollama listJan can connect to cloud providers via API keys, letting you choose between local and remote models per thread while keeping the same UI.
Do this first Run a private AI assistant on your computer
Run one thread on a cloud provider and another on a local model within the same Jan app
Critique the experimental design in this abstract and suggest two controls I may be missing.Paste the text into the message field of a newly created Jan thread (after selecting a cloud model in the Model Picker). Ensure you’ve saved your API key in Settings → Provider‑Models first; watch that the response comes from the chosen cloud model, not the local one.
Critique the experimental design in this abstract and suggest two controls I may be missing.LM Studio can expose any loaded model as an OpenAI‑compatible endpoint at http://localhost:1234/v1, enabling existing tools to call it locally.
Do this first Compare model responses in a local chat
Start the LM Studio server and query it with curl or Python
curl http://localhost:1234/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF",
"messages": [{"role": "user", "content": "Name three open-access genomics databases in one sentence each."}]
}'Paste the command into a terminal after you have started the server via Developer → Start Server (green dot should appear). Verify that the response is a JSON object with a choices field; if not, check the model name and server status.
localhost:1234 — point curl, Python, or any OpenAI-SDK code at it with no API key. Credit: lmstudio.ai/docs ↗localhost:1234 in the status barlms server start in a terminal and confirm status with lms server statushttp://localhost:1234/v1/chat/completionsopenai package, setting base_url='http://localhost:1234/v1/' and call the same modellms server stopOllama runs a background server at http://localhost:11434 offering both native and OpenAI‑compatible APIs, so any client can query a local model by changing the base URL.
Do this first Manage local AI models from the command line
Run your local model from curl and Python just like the OpenAI API
python - <<'PY'
from openai import OpenAI
client = OpenAI(base_url='http://localhost:11434/v1/', api_key='ollama')
response = client.chat.completions.create(
model='llama3',
messages=[{'role': 'user', 'content': 'Summarise the central dogma in 30 words.'}]
)
print(response.choices[0].message.content)
PYPaste this into a terminal (or a Python REPL) after installing the openai package. Verify that the printed line contains a concise summary—if you see an error about connection, ensure Ollama is running with ollama list.
ollama list (run ollama serve if needed)http://localhost:11434/api/chat with appropriate JSON payloadbase_url='http://localhost:11434/v1/' and call the modelJan can launch a local OpenAI‑compatible API server at http://localhost:1337, allowing external scripts to call the loaded model as if it were a cloud service.
Do this first Run a private AI assistant on your computer
Expose the loaded model on localhost and query it with curl or Python
curl http://localhost:1337/v1/chat/completions -H 'Content-Type: application/json' -d '{"messages":[{"role":"user","content":"Name three open-access genomics databases in one sentence each."}]}'Paste the command into a terminal window while Jan’s local API server is enabled (Settings → Local API Server). Watch for a JSON response containing a choices array with the generated text.
http://localhost:1337openai package is installed, execute the supplied Python script to obtain a responseHermes is a personal AI assistant that can be pointed at a local LM Studio server, making all its reasoning run entirely on your device.
Do this first Run a local model as an API server
Configure Hermes to connect to the LM Studio server with two settings
lms server start && lms models listPaste this into your system terminal while LM Studio is installed. Ensure the output shows the server running and a model listed—if not, load a model in LM Studio before proceeding.
lms server start and ensure a model is loaded~/.hermes/config.yaml in any editoryaml
provider: custom
base_url: "http://localhost:1234/v1"
``
and save the file~/.hermes/.env to contain OPENAI_API_KEY=local if not presenthermes --tui in a terminal (or launch Hermes Desktop) and ask “What model are you running on?”provider: custom and a localhost base_url makes Hermes operate entirely on your machineHermes can also use Ollama’s OpenAI‑compatible endpoint, allowing the assistant to run fully offline via a local Ollama model.
Do this first Query a private model using the OpenAI API format
Point Hermes at your Ollama server with two config lines
hermes --tuiPaste this into a terminal window where you have Hermes installed. When the TUI launches, type What model are you running on? and verify that Hermes reports the Ollama model (e.g., llama3) instead of a cloud provider.
ollama list and verify a model such as llama3 is available~/.hermes/config.yaml in an editor and add:
``yaml
provider: custom
base_url: "http://localhost:11434/v1/"
``
Save the file~/.hermes/.env contains a placeholder like OPENAI_API_KEY=ollamahermes --tui or via the Desktop applocalhost with another LAN IP in the base_url to reach a remote Ollama instanceJan supports custom Assistants, Projects and MCP integration, letting you create specialised personas, group related threads and enable tool‑calling while staying fully offline.
Do this first Run a private AI assistant on your computer
Create a specialised assistant, organise its threads into a project, and activate MCP for external tools
Create a new custom Assistant called BioReview with the system instruction: You are a careful bioinformatics methods reviewer. Always ask for the organism, sample size, and sequencing platform before advising.Enter this text in Jan’s Assistant creation screen and click Save. After saving, start a thread using the newly created BioReview assistant to confirm it asks the required clarification questions.
Best viewed on desktop — tap Enlarge to read the numbered controls.
LM Studio’s CLI (lms) can run headlessly and bind the server to any network interface, enabling LAN‑wide private model access.
Do this first Run a local model as an API server
Start LM Studio from the command line and make it reachable on your LAN
lms daemon up && sleep 2 && curl -s http://$(hostname -I | awk '{print $1}'):1234/v1/modelsPaste the command into a terminal on the machine where LM Studio is installed. After it runs, look for a JSON response containing a "data" field—if you see that, the always‑on daemon is serving correctly.
localhost to your LAN IP — your whole lab can now point at one shared server. Credit: lmstudio.ai/docs ↗llmster daemon keeps serving your model even with the LM Studio window closed. Credit: lmstudio.ai/docs ↗lms --version to confirm the CLI is installedlms server start then run lms load lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF to launch and load a modellocalhost to your machine’s LAN IP (or 0.0.0.0) and restart the server so other devices can reach http://:1234/v1 lms server stop when finishedcurl -fsSL https://lmstudio.ai/install.sh | bash and run lms daemon up to keep the service running in backgroundOllama can bind to all interfaces using the OLLAMA_HOST environment variable, turning a single machine into a shared private AI server for the whole lab.
Do this first Query a private model using the OpenAI API format
Make an Ollama server reachable from every workstation on your LAN
curl http://192.168.1.42:11434/api/chat -d '{"model": "llama3", "messages": [{"role": "user", "content": "Say hello"}], "stream": false}'Run this command in a terminal on any other computer on the same LAN (not the host). Replace 192.168.1.42 with your server’s IP. Watch for a JSON response containing a message field—if you see it, the shared Ollama endpoint is working.
ollama pull llama3 to download a model0.0.0.0:11434; on macOS use launchctl setenv OLLAMA_HOST "0.0.0.0:11434", on Linux edit the service with systemctl edit ollama.service adding Environment="OLLAMA_HOST=0.0.0.0:11434", on Windows add it to your account variables, then restart Ollamaifconfig, ip addr or ipconfig)curl http://:11434/api/chat -d '{"model":"llama3","messages":[{"role":"user","content":"Say hello"}],"stream":false}' 24h) to keep the model loaded between requestsThe Developer tab lists resident models and lets you tweak settings like Context Length, GPU Offload and Parallel slots, showing their impact on memory and speed.
Do this first Run a local model as an API server
Load several models simultaneously and see how every knob in the Developer panel affects resource use
Best viewed on desktop — tap Enlarge to read the numbered controls.
Parallel is per model — 1 for the 120B, 4 for the cheap ones. The right column is the selected model's load-time configuration, and nearly every row there trades memory for speed or quality. 130 outcomes in all — one per recipe below.
Ollama HR / People +1Need to strip personal data from HR or patient files before upload
Sensitive records are de-identified on the machine, satisfying data-governance requirements without involving a third-party API.
Ollama Founder +1I need an offline code reviewer
A free, always-available code reviewer that works on proprietary code offline, with no API subscription needed.
LM Studio Finance +1Need a fast local contract review with no cloud upload
A first-pass legal screen is done in minutes on sensitive contract text, without uploading confidential terms to a cloud AI service.
Meeting transcript full of filler and repeats
Clean, actionable meeting notes are ready in under a minute, with confidential strategic discussions processed entirely offline.
Need fast support answers that stay private
First-draft support answers in seconds that are anchored to real documentation, with no per-token cost and no customer data sent to a cloud API.
Ollama Sales +1Need a call recap without sending notes to the cloud
A clean call summary and a follow-up draft are ready before the rep closes their notes, with no prospect data reaching a cloud AI service.
Need a concise account brief from raw CRM notes
Reps walk into calls with a clean, accurate account brief drafted in seconds, without routing sensitive pipeline data through an external AI endpoint.
Ollama Finance +1Need to check expense claims without sharing employee data
Expense pre-screening is automated without sharing employee financial data with a cloud AI, and borderline claims are surfaced for human review before reimbursement.
Ollama HR / PeopleFinding biased language in a job posting as it goes live
Bias in job descriptions is caught and corrected before publication, improving applicant diversity without sharing unpublished internal role details with a cloud service.
LM Studio HR / PeopleNeed a quick, secure interview debrief
Structured debrief summaries are ready before the panel meeting, with candidate data kept on-device and compliant with HR data-handling policies.
Need a role‑specific offboarding list that stays private
A complete, role-specific offboarding checklist is ready in one prompt, reducing the risk of missed steps and keeping departure details private.
LM Studio SupportSupport tickets could hide escalation risks
High-risk tickets are surfaced to senior agents immediately, reducing time-to-escalation, with no customer message content leaving the local network.
Raw bug tickets from customers
Engineering receives a clean, structured bug report instead of a raw customer message, cutting triage time and avoiding the need to re-contact the customer for basic details.
LM Studio SupportDraft reply may break tone guide or policy
Agents catch policy-violating or off-brand replies before sending, reducing the risk of customer complaints arising from poorly worded responses, at zero per-check cost.
Need to reply to a new ticket fast
First responses are drafted in under 30 seconds, maintaining fast initial contact times while keeping customer message content on the local machine.
Ollama SupportNon‑English support tickets get auto‑translated and labeled
Non-English tickets are translated and routed to the right queue in seconds at zero per-call cost, with customer message content never leaving the local environment.
Ollama SalesWhen my competitive notes are scattered
A ready-to-use battlecard is drafted from raw internal notes in minutes, with no competitive strategy or pricing intelligence shared with an external AI service.
Ollama RoboticsNeed a safety review of firmware on‑site
A safety-focused code review covering the firmware's most risk-prone paths is available before every release, with no proprietary embedded code leaving the building or touching an external AI service.
Ollama HR / PeopleNeed to hide personal info in CVs
Anonymised candidate documents are produced on-device before any cloud-tool upload, reducing privacy risk and supporting compliance with data-minimisation principles. The human reviewer always makes the final assessment.
Ollama PhysicianNeed to share patient notes safely
Notes are safe to share for case discussion, audit, or a cloud tool afterward, with PHI handled entirely on-device and no diagnostic decision made by the model itself.
LM Studio PhysicianA structured draft note is ready for the chart within minutes of the consultation, with patient audio and transcript processed entirely offline and every draft reviewed and signed off by the physician before filing.
Ollama Small biz +2Survey responses need categories
Thousands of rows classified overnight on a laptop for free, with no data ever leaving the machine.
Need to group employee survey comments and spot urgent issues
HR gets a thematic summary of sensitive employee feedback without exposing staff comments to a cloud service, satisfying data-handling commitments made to employees.
Ollama Support +1Want to triage tickets without leaking PII
Tickets are pre-triaged and routed before a human even opens the queue, with no customer PII leaving the local network.
Ollama SupportNeed a weekly sentiment overview of support tickets
A sentiment heat-map of the week is ready in minutes without customer conversation data leaving the organisation, enabling queue-quality reviews at zero per-token cost.
Ollama SupportNeed to understand common CSAT comments
A ranked list of support experience themes (e.g. "wait time", "agent knowledge", "resolution quality") is ready in minutes, with customer feedback data processed entirely locally.
Ollama Finance +2Need a board‑pack summary while offline
Decision-relevant summaries are ready before landing, with zero data leakage risk and no dependency on airport Wi-Fi.
LM Studio HR / People +1Staff want quick policy answers
Routine HR queries are answered instantly from the exact handbook text pinned in the prompt, without involving a SaaS chatbot that would hold proprietary policy text on external servers.
I need a private summary of my personal notes
A private thinking partner that handles years of personal notes without any of that content going to an external server or training dataset, and without the setup steps the other local tools ask for.
Can’t find company policy answers fast
Staff get instant, cited answers from company docs without a SaaS chatbot ever holding the proprietary text on external servers.
Support agents need quick private answers from our KB
New support agents ramp faster using a private, citeable knowledge base that keeps sensitive customer and product data entirely in-house.
Can't send compliance docs to the cloud
Compliance queries are answered in seconds with direct citations, reducing the risk of relying on memory or an outdated spreadsheet, with no proprietary filings exposed to a cloud API.
New hires keep asking the same onboarding questions
New hires get instant answers to onboarding questions around the clock without HR fielding repeat enquiries, and proprietary process docs stay within the building.
Need policy answers without exposing docs
Managers get accurate, cited policy answers immediately rather than waiting for an HR email, and the policy documents remain within the organisations network.
On‑call engineers stuck digging through wikis
On-call engineers resolve incidents faster by querying a cited internal knowledge base instead of digging through scattered wikis, with no internal system architecture or past incident data leaving the local network.
Can't tell if a bug is fixed in a given version
Support agents answer version-specific questions accurately with source citations, without searching wikis manually or asking engineering, and with internal release notes staying on-premise.
Can't locate the sales talk track fast
Reps surface the right talk tracks and objection responses in seconds with source citations, without proprietary sales methodology leaving the internal network.
LM Studio Investor +1Can't let deal data leave your PC
A clean, structured briefing document is ready before the investment committee meeting, with no deal-sensitive or material non-public information reaching a cloud AI provider.
Need protocol help and note drafts without internet
The prosthetist has a knowledgeable assistant available throughout the clinic day regardless of connectivity, with all patient session context and device-specific information processed on-device and never transmitted to an external server.
LM Studio ScientistNeed to query de‑identified patient notes on your laptop
Ethics-compliant analysis of identifiable data is possible on a standard laptop, even before IRB-approved anonymisation is complete.
Raw field observations in remote area with no internet
Research work continues productively in remote locations with no connectivity, and all raw observations stay on the device.
Want to search your PDFs locally and get cited answers
A grounded, citeable assistant over sensitive or licensed papers, with zero risk of the PDFs reaching a cloud provider or training set.
Ollama Finance +2Need a variance commentary without cloud exposure
A first-draft variance commentary ready in minutes, with sensitive financials processed entirely locally and no data shared with any cloud provider.
LM Studio Finance +1Verify payroll for hidden errors while keeping salaries private
Potential payroll errors are caught pre-approval in minutes, with no employee compensation data sent to a cloud AI provider.
Find which accounts have manual journal entries
Auditors navigate large workpaper sets in natural language with cited source references, without any confidential financial data leaving the internal network.
Ollama HR / People +1Need to find salary band gaps locally
A prioritised list of pay-equity gaps is produced in seconds without exposing salary data to a cloud AI, ready to inform the next compensation review cycle.
Ollama HR / PeopleNeed to score anonymous résumés with evidence quotes
CVs are screened consistently against the same criteria for every candidate, with all application data processed locally to protect candidate privacy.
LM Studio Sales +1Want to flag risky deals from your notes without sending data out
Deals at risk are identified before the pipeline review meeting, giving the manager data to challenge or support rep confidence scores without leaking deal details to a cloud AI.
LM Studio RoboticsWant to label EMG fitting sessions on the workstation
Structured session labels and summaries are ready for the clinical database in minutes, with patient EMG and fitting data processed entirely on the workbench machine — no cloud endpoint, no data-processor agreement needed beyond the clinic walls.
LM Studio HR / PeopleNeed private, on‑device CV briefs for each candidate
A structured briefing on each candidate is ready before the longlist review call, with no personal data or CV content sent to a cloud AI service, keeping the process privacy-law friendly.
LM Studio ScientistNeed to write grant sections while offline
Grant drafts progress during travel or remote fieldwork with no connectivity dependency and no risk of unpublished ideas reaching a cloud provider.
Need SEO‑friendly product copy from bullet specs
A full batch of product descriptions is written at zero ongoing cost, with no dependency on cloud AI pricing or availability.
LM Studio Creator +2Rewrite unreleased titles offline
You can riff on an unannounced launch or a sensitive draft freely, offline, comparing tone across models with no subscription meter running.
LM Studio Sales +2Draft a confidential bid section without sending any client data
A complete first-draft proposal is produced offline, with no client names, pricing, or competitive strategy shared with an external AI provider.
Need a board‑ready cash‑flow narrative but can’t share data
A polished management commentary is ready for the board pack in minutes, with sensitive forward-looking cash figures processed locally and not exposed to any external service.
LM Studio Finance +2Need a first‑draft investment memo but can’t send data off‑site
A structured first-draft memo is ready for committee review without any target-company data, valuation, or deal terms reaching a cloud AI provider.
Bullet‑point notes for a review
A polished, fair-sounding performance review narrative is drafted in minutes, with employee-specific data processed entirely offline.
LM Studio HR / PeopleNeed a quick, legally‑safe grievance reply
A legally careful, professionally toned grievance response is drafted in minutes without sensitive dispute details leaving the device or the HR team.
Need a fast personal cold‑outreach email that never leaves my computer
Personalised first-touch emails are drafted in under a minute per prospect, with no contact data or research notes leaving the local device.
Ollama Investor +1Raw diligence notes need memo sections
IC memo sections are drafted from raw notes in minutes, with confidential valuation assumptions, deal terms, and target-company financials processed entirely locally.
Ollama EveryoneNeed fast responses from an AI model
Run a model locally on your GPU via Ollama
Ollama EveryoneOllama adds a lightweight Go layer that exposes an OpenAI-compatible API
Ollama EveryoneThe 37 % slowdown in Ollama is due to its cache not persisting across turns
Ollama EveryoneOllama is the preferred local target for editor integrations like VS Code Copilot
Ollama EveryoneRunning AI models on my own computer
Set up Ollama to run local LLMs on your machine
Ollama EveryoneNeed a local AI model
Pull a specific model into Ollama for later use
You can retrieve answers drawn directly from your uploaded PDFs without any cloud request
A single workspace lets you ask questions that require synthesis across several documents
Need answers that come just from my uploaded documents
In Query mode the model will refuse to answer when no relevant document exists, preventing hallucinations
Want answers that draw from my docs and the model’s own knowledge
Chat mode lets the model supplement missing information with its general knowledge, useful for brainstorming
Unsure if you need citable answers or a creative chat
Setting a default chat mode aligns the tool with whether you need citable rigor or creative assistance
Ensures that each workspace only answers from the documents you intended for it
Shows that identical questions produce different answers when asked in separate workspaces
Provides a quick overview of projects (workspaces) and their conversations (threads)
My research folders are a mess
A deliberate naming and rule system prevents future mis-uploads and saves time
You can ask the agent to enumerate all uploaded documents instantly
The agent can condense one or more documents into short, readable summaries
The agent can fetch a web page and return the most relevant points
I need to see numbers from my file as a graph
You can ask the agent to create a chart based on numbers found in your uploads
Repeating a multi‑step workflow over and over
You can create a single, reusable `@agent` command that performs a common multi-step task consistently
Need to set up a new AI instance
Your instance is ready to answer queries using the language model you choose
Need teammates to access shared knowledge base
Multiple people can log into the same private knowledge base
Need a scriptable way for my app to talk to my workspace
You obtain a secret token that lets scripts call the developer API
You confirm the API returns answers based on your workspace documents
Want a chatbot on your website
You can drop a ready-made chat interface onto external webpages
Want to save a web page or snippet to your notes
You can push any page or selected text directly into your knowledge base, even behind logins
Need to query your private docs from a script
You can automate queries to your private knowledge base from any command line or script
Can’t upload files one by one
You can open the Data Connectors interface to see a card for each connector and start importing content without manual file uploads
I only want part of a documentation site
You can pull a specific portion of a docs site into a workspace by setting crawl depth and link limits
You can ask a question that draws on several imported sources and see exactly which connector supplied each fact
Want an up‑to‑date, searchable hub for a project
By populating one workspace with several connectors you create an up-to-date, searchable knowledge hub for any project
Want to add web search without setting up any keys
You can start using web search instantly without configuring an API key
The agent can answer questions with live data and include source citations
Let an AI assistant run queries on your database without risk
You can let the agent run SELECT queries against your database safely
Want weekly order totals for last month
The agent can translate natural-language questions into SELECT statements and return results
Need a way to keep the agent from changing data
Using a read-only database user prevents the agent from modifying data
Need a reusable named workflow for your agent
Creates a named, describable flow that the agent can later invoke by name
Want to pull web page or API data as the first step
Allows the flow to fetch raw data from a web page or API as its first step
Need to save the LLM’s final answer
Saves or returns the final LLM output so the agent can deliver it to the chat
Need the flow always on for agents
Persisting the canvas registers the flow as a skill that can be called instantly
Run a saved workflow from chat
Executes the entire block chain with a single chat command, returning the final output
A clear name/description lets the agent pick the right flow when multiple exist
Need a ready‑made skill or flow
The hub lets you bring in pre-built agent skills, flows, prompts, or slash commands with a single import string
Hub imports get blocked in my Docker container
Self-hosted Docker installations block hub imports until an environment variable is set, protecting against untrusted code
Old Docker packages still installed
Removing any previously installed Docker components prevents conflicts with the new installation
System lacks required packages for Docker
Installing ca-certificates, curl and gnupg ensures the system can fetch and verify Docker's repository securely
Apt can’t verify Docker packages
Adding Docker's official GPG key lets apt verify packages from Docker's repo
No official Docker package source
Configuring the official Docker apt source enables installation of the latest stable engine
Want Docker on your machine
Installing docker-ce, its CLI, containerd and related plugins gives you a fully functional Docker runtime
Running the official hello-world container confirms that Docker Engine is correctly installed and can pull/run images
Anyone who can access /var/run/docker.sock effectively has full root privileges on the host
Membership in the docker group grants the same power as root because it allows access to the Docker socket
Need to run Docker without typing sudo
Adding your account to the docker group lets you run Docker commands without sudo
Group membership changed but session still shows old rights
You must start a new login session or use newgrp for the group membership to take effect
Running a Docker command without sudo confirms the group membership is active
If Docker still fails, confirming the docker group exists helps diagnose the issue
Know which architecture your Mac uses to anticipate Docker image compatibility
Need Docker Desktop installer for macOS
Obtain the official Docker Desktop DMG that matches your macOS version
Docker isn’t in your Applications folder
Place Docker Desktop into the standard macOS Applications folder for easy launching
First run asks for your macOS password to install helper tools
Allow Docker to install its privileged helper tools so it can manage containers
Confirm Docker Desktop is running and ready to accept commands
Ensure the Docker command-line interface is installed and reachable from Terminal
Need an Intel‑only Docker image on Apple Silicon
Override Docker's default ARM64 runtime to pull/run an image built for Intel CPUs
Docker won’t launch after a macOS update
Ensures Docker can start by granting it the required virtualization permission
Docker Desktop won’t start after an OS upgrade
Restores missing or corrupted files that can block startup after an OS upgrade
Slow file sharing on Apple Silicon builds
Improves build performance by embedding files directly into the image instead of mounting them at runtime
Docker volume mounts are slow on Apple Silicon
Specifies a concrete host location, helping Docker resolve the mount faster on Apple Silicon
Identifies which local process is using a Docker-exposed port so you can free it
Docker can’t start because a port is busy
Frees the occupied port so Docker can bind to it
Default Docker port is taken
Allows containers to run on an alternative host port when the default is taken
You can quickly tell if WSL2 is already set up on your machine
Need a Linux environment on Windows
Installs the Linux kernel, required Windows features, and a default Ubuntu distro
Confirms that the installed distro is running under WSL2
First time opening Ubuntu in WSL2 and need a Linux login
You need a Linux username/password to log into the WSL2 distro
Docker Desktop fails to run
Docker Desktop requires Intel VT-x or AMD-V to run the WSL2 VM
You can confirm Docker Desktop is running and see its version and storage driver
An empty table proves the Docker client can talk to the engine and that no containers are currently active
Successfully pulling the image confirms network access and that the Docker daemon can retrieve layers from Docker Hub
Running the test container shows that Docker can create, start, and clean up a container in one step
Understanding ephemerality prevents confusion when `docker ps` shows no containers after a run
My WSL2 uses too much memory
You can prevent WSL2 from consuming too much RAM on low-memory machines
You can visually confirm that a Linux VM is running under Windows
You can see which Linux distributions are active and their version/status
You can quickly assess how much storage Docker images, containers, and volumes are using
You can see real-time CPU, memory, network, and I/O usage for each running container
You can explain how Docker Desktop, WSL2, and the Docker Engine interact
Understand the trade-off between a full-featured Docker Desktop UI and a lightweight, free WSL2-native Docker engine
Learn where to find Windows-level error logs that explain why Docker Desktop fails to start
Docker Desktop won’t start because of a Hyper‑V conflict
Resolve Docker Desktop startup failures caused by Hyper-V conflicts by disabling and re-enabling the hypervisor
Containers can’t reach the internet
Fix container internet connectivity by changing Docker Desktop's network mode
Containers failing to resolve external sites
Bypassing Docker's default DNS resolves internet-access failures inside containers
Identify which Windows process is occupying a Docker-mapped port so you can free it or change the mapping
A Docker port is blocked by a Windows process
Stop the offending Windows process so Docker can bind to its required port
LM Studio EveryoneKnowing your total RAM lets you choose a model tier that fits comfortably
LM Studio EveryoneMapping your free RAM (or VRAM) to a model tier ensures the model loads with headroom and runs acceptably
Patience ensures the full model is present before you try to run it
You identify the smallest model that still meets your task requirements
You know the local model is ready, so offline work will succeed
Jan can operate fully offline, guaranteeing no data leaves your machine
Edit confidential text without sending it
You can safely improve confidential text without any risk of transmission
The response arrives instantly, proving the model is executing on your machine
You gain confidence that offline Jan protects unpublished material
Selecting a real confidential workflow makes the offline habit meaningful
Need to run a private workflow with no internet
Executing the whole private workflow without internet shows practical limits and strengths
Understanding where the local model suffices helps decide when to upgrade
Unsure when to keep data offline
A clear guideline ensures you consistently protect sensitive work
No credentials for cloud AI
You can connect Jan to a cloud model by supplying your own API key for the chosen provider
Want separate AI models for each chat
You can select a cloud model for any individual thread without affecting other threads
Sending a challenging query to a cloud model can yield richer, more advanced answers
Need a thread for sensitive work
You can revert to a local model for sensitive work, keeping data on your machine
A simple curl command proves the endpoint returns a valid OpenAI-style JSON response
Want to use the OpenAI Python SDK against a local model
Using the official `openai` SDK you can call your local model just like a cloud model
Need to stop the local API server
Disabling the server stops any further requests to your private endpoint
Want to query a local LLM with just one import
A one-function wrapper lets any script call the local model with a single import
The assistant automatically follows its saved instruction for every new thread
Want to keep a custom assistant for later
Saving the assistant under a name lets you retrieve it for future work instantly
Running real work inputs shows whether the assistant follows its instruction and where it needs refinement
LM Studio EveryoneCan't find a ready‑to‑use AI model
You obtain the model files locally with a single click
LM Studio EveryoneWant to talk to a local AI model
You can interact with the local model just like an online LLM, but everything stays on your machine
LM Studio EveryoneYou know your data never leaves the computer, preserving privacy
You can only compare models if you have at least two of them stored locally
Send one prompt to two AI assistants
Activating the Multi-model chat feature lets you send one prompt to multiple models at once
Sending the same question to both models lets you directly compare their reasoning and output
All processing stays on-device, so confidential data never gets transmitted
Want a quick side‑by‑side model comparison
Summarising strengths, weaknesses, and trust level for each model helps you decide which to use
LM Studio EveryoneA quick curl request proves the local endpoint returns a valid OpenAI-style response
LM Studio EveryoneNeed to call a local model from existing OpenAI‑Python code
Using the OpenAI Python SDK shows that existing code can talk to your local model unchanged
LM Studio EveryoneServer keeps running after I’m done
Stopping the server frees resources and prevents stray network listeners
LM Studio EveryoneCalling a local language model like OpenAI’s API
A single reusable function lets any script call your local LM Studio model with the same interface as OpenAI's SDK
AI app is calling a cloud model but you need it to use your local LM Studio
Changing two lines tells Hermes to send requests to your local LM Studio instance instead of a cloud API
Need a text‑based prompt interface
Launching Hermes with `--tui` opens an interactive terminal UI ready to accept prompts
LM Studio only runs on one PC
Changing `localhost` to another machine's IP lets Hermes talk to an LM Studio server on your network
LM Studio EveryoneYou can quickly check whether the LM Studio API is up and which model is active
LM Studio EveryoneNeed to stop a headless server without leaving stray processes
You can gracefully terminate the headless server without leaving stray processes
LM Studio EveryoneWant my AI app to stay running across reboots
The `llmster` background service keeps LM Studio running across reboots and without the GUI
LM Studio EveryoneWant to start a local LLM server with one click
A single shell file can start the server, load a model, and announce the API URL
LM Studio EveryoneA simple HTTP request confirms other machines can reach your LM Studio endpoint
Ollama EveryoneYou see how well the local model understands domain-specific prompts
Ollama EveryoneYou obtain a comparable answer to evaluate quality and speed
Ollama EveryoneYou know how much storage each model consumes to inform RAM/VRAM decisions
Ollama EveryoneYou settle on the most efficient model for your regular tasks
Ollama EveryoneYou can quickly gauge which model better suits a specific task by comparing their answers
Ollama EveryoneWant to query your own LLM in Python with a single import
Encapsulating the OpenAI-compatible client in a single helper lets any script query your local model with one import and one function call
Want your AI assistant to use a local model
Setting `provider: custom` and the local Ollama base URL makes Hermes use your own model instead of a cloud service
Launching Hermes in TUI mode lets you verify that it is now talking to the local Ollama model
Ollama EveryoneKnowing the host's LAN IP lets peers point their scripts to the shared endpoint
Ollama EveryoneA successful JSON reply confirms the remote Ollama server is reachable and functional
Ollama EveryoneA successful curl from another machine confirms the whole setup works end-to-end
AnythingLLM, by Mintplex Labs, is an all-in-one private AI workspace. You drop in PDFs, Word docs, CSVs, or a whole folder, and chat with them in a familiar ChatGPT-style interface that shows citations back to your source files. Its angle versus a plain chatbot is document RAG plus built-in AI agents bundled together. It is provider-agnostic: run a fully local model so nothing leaves your machine, or bring your own cloud key from OpenAI, Anthropic, Azure, AWS, or Gemini. It ships as a one-click desktop app for Mac, Windows, and Linux, and as a Docker server you can self-host as a shared team endpoint. It is open source under the MIT license.
Yes, the self-hosted server exposes a full developer API. The docs state you can use it to manage, update, embed, and even chat with your workspaces. You generate an API key inside the app, then call endpoints with that key in the request header; accounts with the right access level can create and delete keys on the fly. The complete, browsable endpoint reference for your specific instance is served at /api/docs on the running server. The server build also exposes an OpenAI-compatible endpoint at /api/v1/openai, so existing OpenAI client code can target your private, document-grounded model by swapping the base URL. Keep your API key secret, since anyone holding it can use your instance.
Ollama and LM Studio are model runners: their job is to download, manage, and serve local LLMs, and they go deeper on browsing and one-click model downloads. AnythingLLM sits a layer above that. It is not primarily about managing raw models; it turns a model, local or cloud, into a private assistant over your own knowledge, with document RAG, citations, workspaces, and AI agents built in. In fact AnythingLLM can use Ollama or LM Studio as its underlying model provider. So a common setup is to run models in Ollama or LM Studio and use AnythingLLM as the document-chat and workspace layer on top. Pick AnythingLLM when the goal is chatting with your documents, not just running a model.
AnythingLLM is free and open source under the MIT license. Both the one-click desktop app and the self-hosted Docker server cost nothing to download and run, and the desktop app needs no account. The only thing you might pay for is an optional cloud LLM provider: if you add a key for OpenAI, Anthropic, Gemini, or similar, you pay whatever that provider charges per token. If you instead use a fully local model (the built-in option, or via Ollama or LM Studio), there is no usage cost at all. There is no AnythingLLM subscription or per-seat fee for running it yourself.
The fastest path is the desktop app. Download the one-click installer for Mac, Windows, or Linux from anythingllm.com and run it; no account is required. On first run, pick an LLM provider. Choosing the built-in local provider downloads a small model so nothing leaves your machine, which is the zero-setup option; you can instead point it at Ollama, LM Studio, or a cloud key. Then click New Workspace in the left sidebar, name it, and upload a few PDFs. AnythingLLM reads and indexes them locally. Ask a question grounded in those documents and you have a private document chat running in minutes.
It depends on the model you pick. When you point AnythingLLM at a local model, whether the built-in one or via Ollama or LM Studio, nothing leaves your machine: your documents are stored and processed locally by default, and the desktop app is built as a single-player application offering full privacy. If you instead add a cloud key for OpenAI, Anthropic, or Gemini, then your prompts and the relevant document text are sent to that provider to generate answers, just as with any cloud LLM. So privacy is a choice you control per workspace by choosing local versus cloud. For sensitive material, use a local model.
A workspace containerises a set of documents into its own chat thread, so the context for one project never bleeds into another. Instead of one giant pile of files where the model has to guess which of fifty papers you mean, you give each project its own workspace: one for a thesis chapter, one for a side project, one for course reading. Each answers only from its own files, which keeps answers focused. Within a workspace you can also start separate threads for distinct lines of questioning while the documents stay shared. Think of workspaces as top-level folders and threads as conversations inside them.
Looking for ready‑made agent prompts and commands
Discover how to access community-contributed agents, system prompts, and slash commands to extend your assistant
Want to tap big cloud AI models
Learn how Jan can use powerful cloud LLMs via API keys
Other apps can’t talk to my assistant
You can start a built-in API server from Jan's Settings to let other programs on the same machine talk to Jan automatically
Large model responses are slow
You can run Jan on CPU or use a GPU to speed up responses
Need to set up a new app on Windows, macOS or Linux
You can set up Jan quickly on Windows, macOS, or Linux
Want an AI model you can explore and download
Learn how to obtain and start using an AI model in Jan
Have a .gguf model file on your computer
You can load a locally stored .gguf model into Jan
Jan is a free desktop application that lets you run AI language models (like Llama, Mistral, or Qwen) directly on your own computer, so you can chat with an AI without sending anything to the internet. It is made by Menlo Research and works similarly to ChatGPT, but everything runs locally. You download it once, install it like any app, and use it without a subscription or account.
All three are free tools for running local AI models — the main differences are audience and openness. Ollama is command-line focused and best for developers. LM Studio has a polished interface but its app code is closed-source. Jan sits in the middle: a graphical chat interface resembling ChatGPT and fully open-source, making it a good fit for people who want a familiar UI and want to verify the software. Performance differences are small.
Jan's local models use the GGUF format, a file type designed to package AI models efficiently for consumer hardware. GGUF models come in quantized versions (Q4, Q8, etc.) — lower numbers are smaller and faster but slightly less accurate; higher numbers are larger and more precise. Q4_K_M is a good balance. On Apple Silicon Macs, Jan also supports the MLX format for extra speed.
When you run a local model in Jan, everything happens on your own computer — your conversations are never sent to any server, there is no telemetry by default, and no account is required. After the initial model download, Jan works with no internet connection. Your data is stored in a local folder on your machine and goes nowhere else.
A rough guide: 8 GB RAM runs small models up to about 3B for basic chat; 16 GB handles 7B–8B models well (the sweet spot for quality vs speed on most laptops); 32 GB or more is needed for 13B+ models. Jan's Hub flags if a model likely won't fit, so you can check before downloading.
Ollama EveryoneOllama is a free, open-source tool that lets you download and run AI language models entirely on your own computer with a single command. It works like a package manager for AI models — you pull a model and start chatting, with no cloud account or API key. Once downloaded, the model runs locally, so your conversations never leave your machine.
Ollama EveryoneOllama itself is terminal-only — there's no built-in graphical chat window. However, free companion apps like Open WebUI give you a ChatGPT-style browser interface that connects to Ollama running on your machine. Once both are running, you chat through a browser tab just like ChatGPT.
Ollama EveryoneOpen WebUI is the most popular free graphical front-end for Ollama. It runs as a local web app you open in your browser and gives you a full chat interface, conversation history, and model switching — no coding required. Install it (a desktop app exists), make sure Ollama is running, and Open WebUI connects to it automatically on localhost.
Ollama EveryoneYes — after the initial model download, Ollama requires zero internet. You can pull a model at home, take your laptop somewhere with no Wi-Fi, and it still works normally. This makes it suitable for sensitive academic work or travel.
Ollama EveryoneLM Studio has a full graphical desktop app — you browse and download models with clicks, no terminal needed. Ollama is terminal-based and more lightweight, but better for integrating with other apps. For a complete beginner, LM Studio is the easier start; if you're comfortable with a terminal and want to connect AI to other tools, Ollama is the better choice. Many people use both.
Ollama EveryoneRun 'ollama ps' in the terminal while a model is loaded. The 'Processor' column shows 100% GPU (fully accelerated), 100% CPU (no GPU), or a split if the model is too large for VRAM and offloaded some layers to RAM. If you expected GPU but see CPU, update your GPU drivers.
Ollama EveryoneYes. Ollama is fully free and open-source with no subscriptions, usage limits, or API costs for running models locally, and no account is required to download and use it. An optional paid cloud tier exists for accessing larger hosted models, but local use is entirely free.
Ollama EveryoneOllama works on a regular laptop without a GPU — it automatically falls back to CPU-only mode, which is usable but slower (a few tokens per second on a small model). For comfortable daily use, a GPU with several GB of VRAM, or an Apple Silicon Mac, is recommended, but not required to try it.
Ollama EveryoneAs a rough guide, a 7–8B model needs around 8 GB of RAM (or ~4–6 GB VRAM with a GPU); a ~13–14B model needs around 16 GB RAM or more VRAM; and a 70B model needs very large amounts of VRAM, out of reach for most laptops. If you have only 4–6 GB RAM, start with a small 2B–3B model.
Ollama EveryoneOn Windows or Mac, go to ollama.com/download, grab the installer, and run it. On Linux, paste one line into the terminal: curl -fsSL https://ollama.com/install.sh | sh. After installation, open a terminal and type 'ollama run llama3.2' to download and launch your first model.
Ollama EveryoneOpen a terminal and type 'ollama run llama3.2'. Ollama downloads the model if you don't already have it, then drops you into an interactive chat prompt. Type your question, press Enter, and the model replies in the terminal. To stop, type '/bye' or press Ctrl+D.
Ollama EveryoneOllama stores models in a hidden '.ollama/models' folder in your home directory by default; you can change the location with the OLLAMA_MODELS environment variable. Sizes vary: a 7B model is roughly 4–5 GB, a 14B around 8–9 GB, and a 70B can exceed 40 GB. Use 'ollama list' to see what's installed and 'ollama rm <model>' to delete one.
Ollama EveryoneYes, local Ollama is genuinely private. Once you pull a model it runs entirely offline — your prompts and the model's answers never leave your computer. The official FAQ states Ollama runs locally and does not see your prompts or data when run locally. The only internet traffic is the initial model download. Avoid model tags labelled '-cloud', which run on external servers.
Ollama EveryoneNeed an AI that runs on your own computer
Install Ollama on your laptop, download a suitable model like Llama 3, and verify it runs via the local API on port 11434. This provides the AI brain locally without any cloud dependency.
Keep all your data private while getting AI answers from it
Configure AnythingLLM to use Ollama as the LLM provider, its built‑in embedder, and LanceDB as the vector store. All components stay on your laptop, giving you a fully offline retrieval‑augmented generation system.
Document searches give vague results
Adjust chunk size, overlap, similarity threshold, and max snippets in AnythingLLM to balance precision, context, and speed for your specific documents. Smaller chunks give precise matches; higher similarity thresholds filter out irrelevant results.
Ollama EveryoneNeed a local LLM for your AI agents
Ollama provides an easy way to run large language models locally, eliminating external API latency and cost. By pulling Llama 3.1 you get a capable open‑source model that Agent Span can call directly.
Need a local AI chat workspace
After installing AnythingLLM you click Get Started, choose an LLM provider (e.g., local model or NVIDIA NIM), select the model, and name your workspace. This initializes the app with your chosen backend and creates the main dashboard for chatting and agent work.
Want your chatbot to make visual charts
Agent skills are modular capabilities that can be toggled in the Settings → Agent Skills panel. Enabling a skill like “Generate Charts” adds the function to any agent you create, letting it output visual data without external tools.
Need info from a huge PDF
RAG (Retrieval‑Augmented Generation) lets the agent embed a document, then answer questions by searching its vector store. Uploading a 137‑page PDF and tagging the agent with “rag” enables fast, citation‑backed answers.
Need current facts for your query
The Web Search skill connects the agent to DuckDuckGo (or other providers). When enabled, the agent can fetch up‑to‑date information from the internet and incorporate it into its response.
Need a quick time‑management web app
With an agent that has code generation enabled, you can describe a small application and the model will output ready‑to‑run source files. This demonstrates using local LLMs for rapid prototyping.
The same set on /recipes, filtered by tool and role.
Watch this if you're worried your hardware isn't good enough. It picks realistic models for modest machines.
ProgrammingKnowledge is a long-running, reliable tutorial channel. Best no-command-line intro to running models on a laptop.
Straight from the maker. Watch to see where AnythingLLM is heading and how local AI stops being desk-bound.
The freshest plain install walkthrough. Pair with Tim's video if you want the install spelled out before the deeper tour.
The hands-on 'chat with your own PDFs, fully private' build. Pair it with the Ollama videos if you haven't set up a local model yet.
Dr Walker explains the why (data privacy) as well as the how — the right framing for using local AI in research.
The tightest 'what is this and how do I start' video. Watch first to see the whole tool before going deep on RAG.
The complete hands-on walkthrough. Watch after Dr Walker's framing to get Jan fully configured.
Tech With Tim is a top, no-fluff coding educator. The fastest path from 'never run a local model' to 'I have a private LLM on my machine'.
AnythingLLM, by Mintplex Labs, is an all-in-one private AI workspace. You drop in PDFs, Word docs, CSVs, or a whole folder, and chat with them in a familiar ChatGPT-style interface that shows citations back to your source files. Its angle versus a plain chatbot is document RAG plus built-in AI agents bundled together. It is provider-agnostic: run a fully local model so nothing leaves your machine, or bring your own cloud key from OpenAI, Anthropic, Azure, AWS, or Gemini. It ships as a one-click desktop app for Mac, Windows, and Linux, and as a Docker server you can self-host as a shared team endpoint. It is open source under the MIT license.
Yes, the self-hosted server exposes a full developer API. The docs state you can use it to manage, update, embed, and even chat with your workspaces. You generate an API key inside the app, then call endpoints with that key in the request header; accounts with the right access level can create and delete keys on the fly. The complete, browsable endpoint reference for your specific instance is served at /api/docs on the running server. The server build also exposes an OpenAI-compatible endpoint at /api/v1/openai, so existing OpenAI client code can target your private, document-grounded model by swapping the base URL. Keep your API key secret, since anyone holding it can use your instance.
Ollama and LM Studio are model runners: their job is to download, manage, and serve local LLMs, and they go deeper on browsing and one-click model downloads. AnythingLLM sits a layer above that. It is not primarily about managing raw models; it turns a model, local or cloud, into a private assistant over your own knowledge, with document RAG, citations, workspaces, and AI agents built in. In fact AnythingLLM can use Ollama or LM Studio as its underlying model provider. So a common setup is to run models in Ollama or LM Studio and use AnythingLLM as the document-chat and workspace layer on top. Pick AnythingLLM when the goal is chatting with your documents, not just running a model.
AnythingLLM is free and open source under the MIT license. Both the one-click desktop app and the self-hosted Docker server cost nothing to download and run, and the desktop app needs no account. The only thing you might pay for is an optional cloud LLM provider: if you add a key for OpenAI, Anthropic, Gemini, or similar, you pay whatever that provider charges per token. If you instead use a fully local model (the built-in option, or via Ollama or LM Studio), there is no usage cost at all. There is no AnythingLLM subscription or per-seat fee for running it yourself.
The fastest path is the desktop app. Download the one-click installer for Mac, Windows, or Linux from anythingllm.com and run it; no account is required. On first run, pick an LLM provider. Choosing the built-in local provider downloads a small model so nothing leaves your machine, which is the zero-setup option; you can instead point it at Ollama, LM Studio, or a cloud key. Then click New Workspace in the left sidebar, name it, and upload a few PDFs. AnythingLLM reads and indexes them locally. Ask a question grounded in those documents and you have a private document chat running in minutes.
It depends on the model you pick. When you point AnythingLLM at a local model, whether the built-in one or via Ollama or LM Studio, nothing leaves your machine: your documents are stored and processed locally by default, and the desktop app is built as a single-player application offering full privacy. If you instead add a cloud key for OpenAI, Anthropic, or Gemini, then your prompts and the relevant document text are sent to that provider to generate answers, just as with any cloud LLM. So privacy is a choice you control per workspace by choosing local versus cloud. For sensitive material, use a local model.
A workspace containerises a set of documents into its own chat thread, so the context for one project never bleeds into another. Instead of one giant pile of files where the model has to guess which of fifty papers you mean, you give each project its own workspace: one for a thesis chapter, one for a side project, one for course reading. Each answers only from its own files, which keeps answers focused. Within a workspace you can also start separate threads for distinct lines of questioning while the documents stay shared. Think of workspaces as top-level folders and threads as conversations inside them.
Jan is a free desktop application that lets you run AI language models (like Llama, Mistral, or Qwen) directly on your own computer, so you can chat with an AI without sending anything to the internet. It is made by Menlo Research and works similarly to ChatGPT, but everything runs locally. You download it once, install it like any app, and use it without a subscription or account.
All three are free tools for running local AI models — the main differences are audience and openness. Ollama is command-line focused and best for developers. LM Studio has a polished interface but its app code is closed-source. Jan sits in the middle: a graphical chat interface resembling ChatGPT and fully open-source, making it a good fit for people who want a familiar UI and want to verify the software. Performance differences are small.
Jan's local models use the GGUF format, a file type designed to package AI models efficiently for consumer hardware. GGUF models come in quantized versions (Q4, Q8, etc.) — lower numbers are smaller and faster but slightly less accurate; higher numbers are larger and more precise. Q4_K_M is a good balance. On Apple Silicon Macs, Jan also supports the MLX format for extra speed.
When you run a local model in Jan, everything happens on your own computer — your conversations are never sent to any server, there is no telemetry by default, and no account is required. After the initial model download, Jan works with no internet connection. Your data is stored in a local folder on your machine and goes nowhere else.
A rough guide: 8 GB RAM runs small models up to about 3B for basic chat; 16 GB handles 7B–8B models well (the sweet spot for quality vs speed on most laptops); 32 GB or more is needed for 13B+ models. Jan's Hub flags if a model likely won't fit, so you can check before downloading.
LM Studio is a free desktop application that lets you download and run AI language models directly on your own computer — no internet needed once a model is downloaded. It provides a friendly graphical interface similar to ChatGPT, but everything runs locally. It works on Mac (Apple Silicon), Windows, and Linux.
Because LM Studio's local server mimics the OpenAI API, many third-party tools connect to it out of the box — for example Obsidian, AnythingLLM, Continue.dev, Open WebUI, and n8n. LM Studio also supports the MCP protocol for connecting AI agents to external data sources.
LM Studio is a graphical desktop app — you click buttons, browse models visually, and chat in a built-in interface, friendlier for beginners with no coding. Ollama is command-line first and better for scripting or automation. Both use the same underlying engine (llama.cpp), so quality and speed are comparable; the difference is interface and workflow. A common pattern is LM Studio for exploring models and Ollama for integrating into code.
Yes, after the initial download. You need internet once to install LM Studio and once to download each model. After that, the app and all your models run entirely offline — useful in settings with restricted internet, or when you want guaranteed privacy.
The number (7B, 13B) refers to billions of parameters — a rough measure of capacity. Bigger models generally give smarter answers but need more RAM and run slower. For most everyday tasks on a typical laptop, a 7B or 8B model at Q4_K_M quantization is a practical starting point. LM Studio's Discover tab warns you if a model is likely too large for your hardware.
Slow output usually means the model is too large for your GPU and is spilling into RAM — try a smaller model or a lower quantization (Q4 instead of Q8). Strange or repetitive output is often the wrong chat template; LM Studio normally auto-detects it, but if it looks broken, check the correct template is selected in the model settings. Out-of-memory errors are solved by a smaller model variant.
Yes. Since mid-2025, LM Studio is free for both personal and workplace/commercial use, so you can run it in a lab or office without a license fee. The models themselves are governed by their own licenses (most popular ones permit research use freely). Organizations needing team-sharing or enterprise controls have optional paid tiers, but they're not required for standard use.
Yes. As of mid-2025, LM Studio is free for both personal use and use at work, with no commercial license required. Optional paid Teams/Enterprise tiers exist for organizations needing private collaboration or SSO, but the core desktop app is free, and the models it runs are free and open-source.
+ 19 more in the library.
Hermes
ollama runollama pullollama listollama psollama stopollama rmollama servehermes setuphermes --tuisystemctl edit ollama.service/bye>>>OLLAMA_HOSTOLLAMA_KEEP_ALIVEhttp://localhost:11434/api/chat/v1/~/.hermes/config.yaml~/.hermes/.envprovider: custom[Service]llama3qwen2.5gemmamistralopenaipip install openaiapi_keybase_urllocalhost"stream": falseifconfigip addripconfiglmslms server startlms server stoplms server statuslms --versionllmsterhermes setuphermes --tuichmod +x start-local-llm.shpip install openaihttp://localhost:1234/v1localhost:1234localhosthttp://<your-ip>:1234/v10.0.0.0openaiapi_keybase_urlconfig.yaml~/.hermes/config.yaml~/.hermes/.envhttp://localhost:1337localhost:1337modelopenaipip install openaiapi_keyhttp://localhost:1337/v1base_urlcurl@agentmintplexlabs/anythingllm/api/docsrequestsworkspace@agentdocker runShow CitationsAsk, share, or report — over on the Heidelberg AI community forum.