Where the AI comes from

2.1 One gateway, one name, every app

LiteLLM is a small server that speaks the OpenAI API and forwards to whatever actually answers. On the bench it runs on port 4000 and every app points at it.

Understand why one switch can change the AI for five apps at once.

in a terminal, with the key from the .env in your HeidelbergBench folder. It lists what the gateway serves.

1. Two names, always. default is what n8n, AnythingLLM and the rest ask for. hermes exists so the agent can use a different model without disturbing anything else.
2. The apps are configured once. Their .env says LLM_BASE_URL=http://litellm:4000/v1. Nothing in them names a model provider.
3. So switching is one file. The dashboard rewrites litellm/config.yaml and restarts the gateway — a few seconds, and no app notices more than a blip.

• You'll see A JSON list with two entries: default and hermes.
• Takeaway Configure the apps once against a name; move the name. That is what a gateway buys you.

2.2 The four options, and what each one costs you

Do this first One gateway, one name, every app

Pick a provider for a real task instead of always taking the default.

1. The bench model (qwen2.5:3b, on your disk). Free, offline, private. Small: it summarises, extracts, drafts and demonstrates. It will not reason through a hard problem.
2. LM Studio, on your machine or a lab machine. Free, and as big as the hardware allows. The bench asks the address what it serves rather than guessing a model name.
3. Requesty or EUrouter. Frontier models, processed in the EU, paid per token. This is the answer when quality matters and the data is sensitive.
4. OpenRouter. The widest selection and usually the cheapest, but routing leaves the EU unless you are on its enterprise plan.

• You'll see The dashboard names your choice under "Which AI answers", and the model line changes on every app tile.
• Takeaway Small-and-local, big-and-yours, big-and-EU, big-and-global. Almost every real choice is one of those four.
• Check A seminar analyses interview transcripts that may not leave the university. Which two options remain?
• Cost Local: electricity. EU routers: provider price plus roughly 5%. Nothing is charged without a key you pasted yourself.

2.3 Point the bench at a model on another machine

A laptop is a weak place to run a model and a fine place to run apps. If there is a workstation, a lab machine or a GPU box on your network, the bench can use its model while everything else stays on your laptop.

Do this first The four options, and what each one costs you

Use a stronger model without paying anyone, by borrowing a machine you already have.

1. On the other machine: run LM Studio, load a model, and turn on its local server. Note the address it prints, usually http://:1234/v1.
2. On the bench: Change under "Which AI answers", choose LM Studio, and give that address. The setup asks it what models it serves and takes the first one — a name nobody has to type correctly.
3. If it is the same machine, the address from inside a container is http://host.docker.internal:1234/v1, not localhost — localhost inside a container is the container.
4. Check it took. The tiles now say that model, and one question in "Ask the course" comes back noticeably faster or better.

• You'll see Every app tile shows the new model name within a few seconds of the switch.
• Takeaway The gateway does not care where the model runs. Neither should your apps.