Heidelberg AICurriculum

I Built The Same App with Every LLM

Tech With Tim ·2026-06-26 ·2 min read

Summary written by us from the video's transcript. The video, and everything in it, is Tech With Tim's work.

The article compares how several major LLMs generate a medium‑complex Mario Kart‑style game from a single prompt, evaluating code quality, speed, and playability.

Takeaways

  • Using the same prompt across models reveals large gaps: only GPT 5.5 and Claude produced genuinely playable games.
  • Maximum‑thinking settings improve output but do not guarantee usability; many models still failed after two prompts.
  • Speed alone isn’t predictive of quality—Claude took 23 minutes yet delivered the best code, while Kimi was fast but produced poor results.
  • Providing identical assets and directory structures is essential for a fair comparison, as some models ignored the supplied sprites.
  • A single complex prompt can be too demanding for many LLMs; breaking tasks into smaller steps may yield better outcomes.

Test Setup and Models

Tim runs all experiments inside the Cursor coding harness to keep conditions identical. He tests Opus 4.8, Composer 2.5, Kimi 2.5, Gemini 3.1 Pro, Grok build 0.1, and OpenAI’s GPT 5.5, using each model’s maximum‑thinking setting and the same asset directory.

He limits interaction to two prompts per model: an initial one-shot description of the game and a single follow‑up if the first output is unusable.

Evaluation Criteria

Each generated app is judged on overall code quality (via an independent AI evaluator), completion time, number of files, lines of code, and whether the resulting game runs with zero setup (click‑to‑play).

The goal is to see which model can produce a functional, visually appealing Mario Kart clone under these constraints.

Initial Results – First Prompt

Opus 4.8, Composer 2.5, Kimi 2.5 and Grok 0.1 all produced broken builds (green screens, missing movement, or outright errors).

Gemini 3.1 Pro ran but was “pretty broken”; GPT 5.5 delivered the only playable version, though it omitted the supplied asset pack.

Second Prompt – Fix Attempts

After a brief feedback prompt (“the game isn’t playable”), Opus 4.8 produced the most functional result: sound effects, drifting, item usage and proper collision handling.

Composer 2.5 became runnable but got stuck in sand; Gemini 3.1 Pro gained basic movement and items; Grok 2.1 improved visual output but remained confusing; Kimi 2.5 still couldn’t move; GPT 5.5’s second pass was largely unchanged, still functional but without the asset pack.

Performance Metrics

Timing: Gemini 3.1 Pro finished in 3 min 56 s, GPT 5.5 in 6 min 15 s, Kimi 2.5 in 6 min 24 s, Cursor (Claude) in 8 min 45 s, Grok 21 min, Claude 23 min 18 s.

File/Line counts: Claude generated the most files and lines; Kimi produced a large codebase quickly; Opus created a modular engine with many small files; GPT’s output was concise.

Qualitative Findings

Claude (the Cursor model) scored highest in every quality category, despite its long runtime.

GPT 5.5 delivered the only consistently playable game with decent physics and basic items, though it missed custom assets.

Opus showed strong engineering effort—building a full engine—but its JavaScript code was messy.

Composer, Kimi and Grok produced minimal or broken games; Gemini generated only a handful of JS files and lacked core gameplay.