Heidelberg AI Curriculum

39tools in the catalogue

AI Bench availability is not confirmed for these catalogue entries.

XTTS-v2

Models · Self-hosted

Free

Good for

Synthesize speech in multiple languages from text using a voice reference.
Run the model locally through the Coqui TTS Python library.
Use optional GPU acceleration when generating speech interactively or at scale.

Watch out

The Coqui Public Model License restricts commercial use of generated audio.

Official site

Whisper (open source)

Models · Self-hosted

FreeOpen source

Good for

Transcribe lectures, interviews, and recordings with an openly released speech model.
Run whisper.cpp on CPUs or Apple Silicon, or use faster-whisper with a GPU.
Keep transcription local when your workflow requires offline processing.

Watch out

You install and operate command-line software rather than a polished transcription app.

Official site

Wan 2.2

Models · Self-hosted

Free

Good for

Generate short video clips from text or animate an existing image.
Download the open model and run it through a ComfyUI workflow.
Choose self-hosting when learning the pipeline or producing many clips.

Watch out

Comfortable local use requires a GPU with substantial VRAM.

Official site

Google Veo

Models · Cloud

Commercial

Good for

Generate video scenes from text prompts using Google's cloud model.
Include generated audio when a scene needs sound or dialogue.
Define shots, rights, captions, and review steps before publishing results.

Watch out

Availability and output limits vary by product surface and may change.

Official site

Sora 2

Models · Cloud

Commercial

Good for

Describe a scene and generate a short cinematic video from text.
Use consistent faces and clothing across cuts for connected visual explanations.
Generate through the prompt interface or an API when available.

Watch out

Sora does not generate audio, and the API has a stated sunset risk.

Official site

SadTalker

Models · Self-hosted

FreeOpen source

Good for

Animate a still image into a talking head from an audio clip.
Generate audio-synced, 3D-aware head motion on your own machine.
Use the open-source pipeline when you can manage Python and model setup.

Watch out

Self-hosting requires setup and a capable GPU for usable speed.

Official site

Qwen3.6 (35B-A3B)

Models · Self-hosted

Free

Good for

Run an open mixture-of-experts model with coding and multilingual capabilities.
Use its long context window for substantial documents or codebases.
Apple MLX support makes it a practical local option on suitable Macs.

Watch out

Its model size still requires substantial memory for local inference.

Official site

Phi-4-mini

Models · Self-hosted

Free

Good for

Run a compact local model for summaries, drafts, classification, and simple code.
Use its long context window to process sizeable documents locally.
Choose it for quick, scoped work on modest laptop hardware.

Watch out

Its small size limits performance on difficult reasoning and niche knowledge tasks.

Official site

OpenAI TTS

Models · Cloud

BYOK

Good for

Turn written content into spoken narration with OpenAI TTS models.
Add speech output to chatbots and accessibility experiences.
Pair text generation and transcription within OpenAI workflows.

Watch out

The available voices may not suit projects needing cloning or especially realistic delivery.

Official site

OpenAI Realtime API

Models · Cloud

Commercial

Good for

Stream microphone audio and receive spoken responses in one API.
Build interruptible voice conversations with function calling.
Let agents look up information or trigger actions while speaking.

Watch out

Long conversations incur usage charges, so estimate audio duration before implementation.

Official site

Nano Banana Pro (Gemini 3 Pro Image)

Models · Cloud

Commercial

Good for

Generate images with text, data labels, and multimodal instructions.
Create search-grounded visuals when current facts matter.
Access the model through Gemini or Vertex AI services.

Watch out

Access and product naming are changing during Google's Imagen migration; verify the current endpoint.

Official site

Moshi (Kyutai)

Models · Self-hosted

Free

Good for

Build speech-native conversations that listen and speak simultaneously.
Support interruptions and overlapping speech in voice interactions.
Self-host the model when your hardware supports real-time use.

Watch out

Real-time operation requires a capable GPU and research-oriented deployment work.

Official site

Ministral 3

Models · Self-hosted

Free

Good for

Run small open-weight models on laptops or edge hardware.
Process images and long context locally when configured accordingly.
Choose an Apache-licensed coding sibling for software work.

Watch out

Small models suit scoped tasks; test harder reasoning tasks before relying on them.

Official site

Mistral Medium 3.5

Models · Cloud

Commercial

Good for

Process text and images with a large-context multimodal model.
Use it for coding and agentic application workflows.
Access hosted inference or self-host the available weights.

Watch out

Verify the provider, region, contract, and licence for your chosen deployment path.

Official site

MiniMax-M2

Models · Self-hosted

Commercial

Good for

Use a model focused on coding and agentic tool chains.
Run multi-file code, execution, and repair workflows.
Choose self-hosting or the MiniMax API for model access.

Watch out

Check the exact model version and licence before self-hosting or commercial use.

Official site

LTX-Video

Models · Self-hosted

Free

Good for

Generate video from text or images through ComfyUI workflows.
Explore model-specific audio-video generation paths.
Pin releases before planning hardware or production workflows.

Watch out

Release-specific hardware, resolutions, clip lengths, licences, and nodes require verification.

Official site

Llama 4 Scout

Models · Self-hosted

Free

Good for

Run a multimodal model for broad tasks with a large context window.
Use quantization to target a single high-end GPU.
Plan deployments around Meta's Community Licence conditions.

Watch out

Quantized local use still requires substantial hardware, and the licence has deployment conditions.

Official site

LatentSync

Models · Self-hosted

FreeOpen source

Good for

Align supplied audio with mouth motion in existing face video.
Replace dialogue audio for permitted video projects.
Review generated motion before sharing or publishing.

Watch out

You must supply permitted footage and voices, then verify releases and hardware requirements.

Official site

Kokoro 82M (open source)

Models · Self-hosted

FreeOpen source

Good for

Generate speech locally with a lightweight Apache-2.0 model.
Run Kokoro on a CPU or modest GPU for embedded applications.
Choose Piper separately when Raspberry Pi efficiency is the priority.

Watch out

Voice-cloning alternatives may have non-commercial licenses; check each model's terms first.

Official site

Kling

Models · Cloud

Commercial

Good for

Generate high-resolution video with controlled motion.
Prototype visual scenes through a proprietary cloud service.
Review identity, rights, captions, and exports before publication.

Watch out

Capabilities, access, limits, and terms can change for this proprietary cloud service.

Official site

HunyuanVideo

Models · Self-hosted

Free

Good for

Generate short text-to-video clips with an open model.
Run it through ComfyUI in an existing video workflow.
Choose it when multi-GPU or rented high-memory hardware is available.

Watch out

Running it requires at least 60GB of VRAM, usually across substantial hardware.

Official site

Grok 4.3

Models · Cloud

Commercial

Good for

Analyze text and video with a large context window.
Use live X data when current social posts are relevant evidence.
Choose it when cloud access and X integration fit the task.

Watch out

It is a closed cloud model whose live-data distinction depends on X integration.

Official site

gpt-oss-20b

Models · Self-hosted

Free

Good for

Run an open-weight model locally for private tool-use tasks.
Use its 128K context for substantial prompts and documents.
Choose it when your machine has about 16 GB of memory.

Watch out

It still needs about 16 GB of memory despite its efficient active parameters.

Official site

gpt-oss-120b

Models · Self-hosted

Free

Good for

Run an open-weight reasoning model on substantial local hardware.
Use its 128K context for larger local tasks.
Choose it when self-hosting matters more than laptop portability.

Watch out

It needs roughly 61 GB of memory, beyond a typical laptop.

Official site

GPT-5.4 nano

Models · Cloud

Commercial

Good for

Classify, extract, route, and draft short responses at high volume.
Connect simple calls to OpenAI tools and integrations.
Choose it when tasks do not require the hardest reasoning.

Watch out

It is a closed cloud budget model intended for simpler, high-volume work.

Official site

OpenAI image generation

Models · Cloud

Commercial

Good for

Generate and edit images through supported OpenAI product surfaces.
Draft diagrams, labels, infographics, and other visual assets.
Review text, facts, rights, accessibility, and exports before use.

Watch out

Access, limits, models, and terms vary by product surface and can change.

Official site

GPT-5.5

Models · Cloud

Commercial

Good for

Handle multi-step reasoning, tool use, and messy agentic tasks.
Adjust reasoning effort when depth and response time need balancing.
Use text and image inputs within a large context window.

Watch out

It is a closed, cloud-only model; higher reasoning effort can increase latency.

Official site

GLM-5.2

Models · Self-hosted

Commercial

Good for

Use a long-context open-weight model for coding and agent tasks.
Access the flagship through Z.ai API or OpenRouter.
Downloadable weights support deployment planning beyond hosted access.

Watch out

The flagship needs cluster-scale hardware; hosted access is the practical option for many users.

Official site

Gemma 4

Models · Self-hosted

Free

Good for

Run open models locally with text and image inputs.
Choose a model size that fits a phone, laptop, or workstation.
Use GGUF or MLX formats for supported local runtimes.

Watch out

The family spans many sizes, so hardware requirements depend on the selected model.

Official site

Gemini 3.1 Pro

Models · Cloud

Commercial

Good for

Analyze large documents, media files, and whole code repositories.
Use multimodal prompts with a context window suited to long inputs.
Choose Deep Think when a difficult problem needs extra reasoning.

Watch out

It is a closed cloud model, and prompts over 200K tokens cost more.

Official site

Google Gemini Live

Models · Cloud

Commercial

Good for

Stream audio in and receive spoken responses for voice applications.
Combine conversation with tools, grounding, and multimodal inputs.
Use unified token billing when a Gemini API workflow fits.

Watch out

Long sessions still consume tokens across audio, text, and tool calls.

Official site

Gemini 3.1 Flash-Lite

Models · Cloud

Commercial

Good for

Process text, images, video, audio, and PDFs in bulk.
Use its large context for document or transcript extraction.
Choose Vertex AI regions when an available EU location fits.

Watch out

It is closed and cloud-only, with higher charges for audio and long prompts.

Official site

FLUX.2

Models · Cloud · Self-hosted

Free tier

Good for

Generate images or edit several references with one instruction.
Choose among local open-weight variants and proprietary API tiers.
Use ComfyUI when a node-based image workflow suits your work.

Watch out

Licensing and deployment options differ sharply between klein, dev, and API tiers.

Official site

F5-TTS

Models · Self-hosted

FreeOpen source

Good for

Generate expressive speech from text using an open-source TTS system.
Install it with pip or Docker on a PyTorch-capable machine.
Use pretrained models for local speech experiments and drafts.

Watch out

Released models carry a CC-BY-NC license, so generated audio is non-commercial.

Official site

DeepSeek V4

Models · Self-hosted

Commercial

Good for

Process long text prompts with a 1M-token context and thinking mode.
Choose it when hosted access or published weights fit the project constraints.
It is text-only, with full self-hosting requiring cluster-scale resources.

Watch out

It is text-only, and the open-weight releases require cluster-scale self-hosting.

Official site

Claude Sonnet 5

Models · Cloud

Commercial

Good for

Handle everyday coding, writing, and agentic tasks with a 1M-token context.
Choose it when image input, caching, and batch processing fit production work.
Its closed cloud deployment supports routing through selected cloud platforms.

Watch out

It is a closed cloud model and sits below Opus in the model range.

Official site

Claude Opus 4.8

Models · Cloud

Commercial

Good for

Use a large-context model for demanding coding, analysis, writing, and agentic tasks.
Choose it when long, multi-step work needs adjustable reasoning effort.
It accepts images and provides a 1M-token context window.