A team at TryAI built a small agentic harness with one job: hand a frontier model a song, a hard dollar budget, and a set of video generation tools, then get out of the way and let it produce a full music video on its own. The results — four complete music videos, two from Claude Fable 5 and two from GPT-5.6 Sol, each at $25 and $100 budgets — are not great as videos. But as a window into how frontier models approach open-ended, long-horizon creative tasks, the experiment is revealing.
🔍 THE BOTTOM LINE
Both models produced valid, full-length music videos autonomously, with no human intervention beyond the initial prompt. The videos themselves were mediocre — character consistency broke down, tempo matching was weak, and neither model iterated on its own edit. But the process data is the real finding: Claude Fable 5 was more expensive but produced higher-quality clips, while GPT-5.6 Sol was more inventive with tool selection and editing techniques. The gap between “can complete the task” and “can complete the task well” is where agentic AI still lives.
The Setup
Each model ran an autonomous tool-calling loop with six tools: a planning tool (free), web search (free), a budget checker (free), image generation (costs money), video generation (costs money), and a local shell with ffmpeg for editing. The model had to research which video generation models exist, pick its own tools, generate clips, watch its own footage, edit with ffmpeg, and assemble a final cut.
Every run got the same inputs: Bruno Mars and Mark Ronson’s “Uptown Funk,” a short text description, and a time-stamped lyric transcript. Every model message, tool call, charge, and error was logged. The full harness is open source.
The experiment was designed to test long-horizon autonomy — not single-turn generation, but the kind of multi-step, self-directed work that defines agentic AI. This is the same category of capability Moonshot showcased with Kimi K3’s 48-hour chip design demo, but at a fraction of the scale and cost.
What the Numbers Reveal
The cost and token data tells a story the videos don’t.
| Run | Wall Time | Tool Calls | Clips Generated | Generation Spend | LLM Token Cost | Total |
|---|---|---|---|---|---|---|
| Fable 5 · $25 | 39m10s | 25 | 54 | $24.30 | $16.99 | $41.29 |
| Sol · $25 | 42m52s | 38 | 46 | $23.18 | $4.27 | $27.45 |
| Sol · $100 | 49m39s | 34 | 70 | $36.57 | $3.25 | $39.82 |
| Fable 5 · $100 | 38m56s | 28 | 80 | $48.60 | $25.05 | $73.65 |
Claude Fable 5’s token costs alone ran $16.99 to $25.05 per run — 30-40% of total cost. GPT-5.6 Sol’s token cost stayed near $3-4 despite similar token volume, thanks to aggressive caching (2.5M+ cached input tokens per run). The Claude Code cost inflation problem shows up here in miniature: Fable 5’s uncached input tokens are the budget killer.
At the $100 budget, neither model spent near the cap. GPT-5.6 Sol spent $36.57 on generation; Fable 5 spent $48.60. Both kept their step counts modest. The extra headroom went unused — a finding that suggests models don’t yet know how to allocate surplus budget strategically.
Where Both Models Failed
The TryAI team’s assessment was blunt: “None of the music videos were great.” The failures cluster in three areas.
Character and story consistency. Recurring characters drift between shots. None of the videos hold a coherent storyline from start to finish. A character that looks one way in clip one looks different in clip two. This is the fundamental limitation of current text-to-video models — no persistent identity across generations.
Literal lyric interpretation. “Make a dragon wanna retire, man” produces an actual dragon on screen. It’s amusing for a few shots, then becomes a creative straitjacket. The models can’t distinguish between literal and figurative language in a creative context.
No self-review. Once clips existed, the models concatenated and muxed, but rarely went back to re-cut or add effects. None seriously probed their own clips to confirm they were any good. GPT-5.6 Sol’s $100 run shipped some genuinely low-quality AI clips, while Claude Fable 5 happened to pick a model with more coherent output. The lack of self-critique is the real agentic gap — and it’s the same gap enterprise AI agent evaluations are surfacing in production environments.
The Divergence in Strategy
Left to choose their own tools, the models diverged in telling ways. Three of four runs went pure text-to-video. Only GPT-5.6 Sol at $25 used an image-to-video pipeline — generating stills first, then animating them. GPT-5.6 Sol at $100 mixed three different video models in a single run (Wan 2.5, Veo 3.1 Lite, and Hailuo 2.3 Standard), the only run to attempt multi-model mixing.
GPT-5.6 Sol at $25 was the most inventive editor. It overlaid text and animated still images with video effects — techniques none of the other runs tried. The rest mostly just stitched generated clips together. This is the same pattern agentic AI benchmarks show: Sol optimises for tool variety and creative exploration, Fable optimises for output quality within a single approach.
Neither model touched Replicate, despite having API keys for both FAL and Replicate. Both went exclusively with FAL. Platform lock-in happens even when the lock is self-imposed.
What This Means for Agentic AI
The experiment is a useful stress test of where frontier models actually are on long-horizon creative tasks. They can complete the task. They can’t complete it well. The gap between “produced a valid output.mp4” and “produced a music video someone would watch” is where the next year of agentic AI development will be spent.
For NZ’s creative sector — film, music, advertising — the signal is mixed. A $100 autonomous music video is a novelty today. A $10 autonomous ad spot that actually converts is the commercial frontier, and it’s closer than the music video experiment suggests. The AI microdrama opportunity for China is the same pattern at a different scale: the creative industries that move first on AI-assisted production will capture the cost advantage, and the ones that wait will be competing against $100 music videos that took 40 minutes.
❓ FAQ
Were the music videos actually good? No. The TryAI team explicitly said “none of the music videos were great.” Character consistency broke down, tempo matching was weak, and the models took lyrics too literally. The value of the experiment is in the process data, not the output.
Which model was better? Subjectively, the TryAI team slightly preferred Claude Fable 5’s $100 video. But GPT-5.6 Sol was more cost-efficient ($27.45 vs $41.29 at the $25 budget) and more inventive with tool selection. Fable 5 produced higher-quality individual clips; Sol explored more creative approaches.
Can I run this experiment myself? Yes. The harness is open source on GitHub. You need API keys for FAL (for video generation) and for Claude Fable 5 and GPT-5.6 Sol. The total cost to reproduce all four runs would be roughly $180.
What does this tell us about agentic AI more broadly? The key finding is the lack of self-review. Models can execute multi-step tasks autonomously, but they don’t yet evaluate their own output critically. This is the same gap showing up in enterprise AI agent deployments — agents that pass their tests but fail in production because they can’t tell when their own work is wrong.
🔍 THE BOTTOM LINE
A $100 autonomous music video is a parlour trick today. But the process data — tool calls, spending patterns, creative decisions, failure modes — is the real output. What it shows is that frontier models can complete long-horizon creative tasks end-to-end, but they can’t self-critique, can’t maintain visual consistency, and don’t know how to spend a surplus budget. The agentic AI gap isn’t capability. It’s judgment. And judgment is harder to train than capability.
