I Battle Tested Sakana Fugu's Fable Killer
Nate Herk | AI Automation ·2026-06-24 ·1 min read
Summary written by us from the video's transcript. The video, and everything in it, is Nate Herk | AI Automation's work.
An experienced AI practitioner tests Sakana Fugu Ultra’s multi‑model orchestration against Claude Opus 4.8, revealing comparable accuracy but much higher latency and cost.
Takeaways
- Fugu Ultra orchestrates multiple specialist models via a single API, acting as a manager rather than a new LLM.
- In benchmark tests it matched Opus 4.8’s accuracy but was 4‑5× slower and five times more expensive.
- Integration requires extra setup (markdown file) compared to simple endpoint swaps.
- Orchestration may benefit multi‑team or heavy development workflows, but isn’t yet cost‑effective for solo knowledge work.
What is Fugu Ultra?
Sakana Fugu’s **Ultra** model claims frontier performance by routing a single API call to several specialist models—GPT, Gemini, Opus, among others—through a manager that breaks down tasks and delegates them. It presents itself as one endpoint that internally coordinates multiple agents.
How the Test Was Set Up
The author ran a **slash‑goal prompt** in Claude Code that asked Fugu Ultra to build a YouTube analytics dashboard, then compared it with Claude Opus 4.8 across 38 benchmark tasks generated by Codex to avoid bias. All tests used API billing; the same prompts and inputs were fed to both systems.
Results: Accuracy vs. Speed & Cost
Both models tied on 36 of 38 tasks, showing essentially identical output quality. However, Fugu Ultra took **357 minutes** total—about **4.5× slower** than Opus’s **80 minutes**—and cost roughly **$50**, five times the **$10** spent on Opus.
Practical Experience Using Fugu Ultra
In a real‑world Claude Code workflow, Fugu Ultra successfully generated live dashboards and AI analyses but felt “really slow.” The orchestration also kept the context window empty, requiring extra handling via a provided markdown file. Integration was more complex than swapping a simple endpoint like with GLM 5.2.
When Might Orchestration Pay Off?
If you work across multiple codebases or teams and need automatic delegation—e.g., GPT for coding, Claude for writing, Gemini for research—the all‑in‑one API could save manual routing effort. For solo knowledge work, the author found existing subscriptions (Codex, Claude) more efficient.
Takeaways and Future Outlook
The author concludes that while Fugu Ultra’s orchestration is technically impressive, its current cost and latency outweigh benefits for most use cases. Mastering model selection and unit‑economics will become a crucial skill as multi‑model APIs proliferate, and the concept may mature into a valuable tool once efficiency improves.