prague-bench

prague-bench

A single-prompt benchmark for AI coding agents. Every model gets the same spec: build a procedural Three.js drone flight over Prague Castle and Charles Bridge, render it headless and deliver a 58-second vertical video. No human help along the way. Here is what each run cost, how long it took, how fast the model wrote, and what a blind panel of three AI judges thought of the result.

measured runs
19
videos passing H1–H6
12
spent on all runs (list prices + billed)
$126
latest blind panel, 11 videos
14 Nov

Leaderboard

The blind panel of 14 Nov 2026 judged 11 videos. Three judges from three vendors scored 16 frames of each video without knowing which model made it. The score is the median of the three.

Latest blind panel: judge scores, median and change since earlier panels
#ModelHarnessAnthropicOpenAIGoogleMedianSpreadvs 13 Novvs 11 NovH1–H6
1Claude Fable 5.2Anthropic · highClaude Code868885863−3±0Pass
2Claude Opus 5.5Anthropic · highClaude Code8581758110−2+1Pass
3Claude Sonnet 5.5Anthropic · highClaude Code7377867713+1−1Pass
4GPT-6 AstraOpenAI · highCodex5863686310−1+5Pass
4Qwen3.8 MaxAlibaba · mediumOpenCode6353786325––Pass
6GPT-6.1 SolOpenAI · highCodex585860582−2±0Pass
7Claude Haiku 5.5Anthropic · highClaude Code5343915348±0+6Pass
8Gemini 4 ArgonGoogle · mediumAntigravity5259485211−3−3Pass
9Gemini 4 HydrogenGoogle · xhighAntigravity4148524811+3–Pass
10Grok 4.7xAI · mediumCursor372930308−14−8Pass
11Beam FreeOpenCode · mediumOpenCode33431−2±0Pass

Spread is the gap between the highest and the lowest judge. The last two columns show how the median moved since the earlier panels; “–” means the video was not in that panel yet. Differences of up to about 6 points are within the panel’s noise.

All runs

Every measured run, failures included. Model time counts only the time the model was generating; the rest of the wall time went to tools, renders and waiting. Select a model for its full record.

Select a column header to sort.

Key statistics of every run
ModelEffortStatusScoreCostWall timeModel timetok/sTokensOutputCache hitH1–H6
Claude Fable 5.2Claude Code · AnthropichighFinished86$43.541 h 40 min1 h 8 min97.425.2M397k91%Pass
Claude Opus 5.5Claude Code · AnthropichighFinished81$15.411 h 7 min56 min105.323.2M352k97%Pass
Claude Sonnet 5.5Claude Code · AnthropichighFinished77$7.2853 min39 min133.019.5M307k95%Pass
GPT-6 AstraCodex · OpenAIhighFinished63$4.4614 min11 min36.22.6M25k97%Pass
Qwen3.8 MaxOpenCode · OpenRoutermediumFinished63$18.74 †4 h 6 min3 h 59 min30.969.2M443k99%Pass
GPT-6.1 SolCodex · OpenAIhighFinished58$0.8314 min13 min36.14.0M27k98%Pass
Claude Haiku 5.5Claude Code · AnthropichighFinished53$1.3229 min22 min214.712.3M289k96%Pass
Gemini 4 ArgonAntigravity · GooglemediumFinished52$3.5737 min27 min67.57.6M110k88%Pass
Gemini 4 HydrogenAntigravity · GooglexhighFinished · harness error, no report48$3.0120 min14 min190.66.9M162k87%Pass
Grok 4.7Cursor · CursormediumFinished30n/a31 min–––––Pass
Beam FreeOpenCode · OpenCode Zen (free)mediumFailed · placeholder video3$08.0 min8.0 min12.3224k1.4k86%Pass
Claude Opus 5.5 (run 2, older spec)Claude Code · Anthropic · older specdefaultFinished–$24.212 h47 min98.026.5M278k89%Pass
GPT-6.1 TerraCodex · OpenAIhighFailed · gave up–$0.101.8 min1.7 min47.1518k4.9k90%No video
GPT-6 LunaCodex · OpenAIhighFailed · gave up–$0.0101.7 min1.7 min49.4555k5.0k89%No video
Big PickleOpenCode · OpenCode Zen (free)–Failed · free quota ran out–$01 h 19 min54 min153.538.1M509k99%No video
DeepSeek V4.1 Flash FreeOpenCode · OpenCode Zen (free)highDid not finish · free quota ran out–$052 min45 min90.619.0M251k97%No video
Kimi K3.1 ProOpenCode · OpenCode GohighDid not finish · 5-hour plan limit–$2.8423 min22 min58.88.8M77k98%No video
Claude Opus 5.5 (run 1, aborted)Claude Code · Anthropic · older specdefaultAborted · wrong route in the spec–$0.452.0 min––323k1.5k77%No video
DeepSeek V4.1 Flash Free (first attempt)OpenCode · OpenCode Zen (free)highNot run · free quota ran out–$00.0 min––00–No video

† billed amount, not list price

Score versus cost

Price and quality pull in opposite directions. A run is on the Pareto frontier when no other run is both cheaper and better.

Score versus cost (log scale)

  • Pareto frontier
Score versus costScatter plot of the latest panel score against the cost of each scored run, cost on a logarithmic axis, with the Pareto frontier drawn as a line.free020406080100$0.5$1$2$5$10$20$50Cost (USD)Panel scoreClaude Fable 5.2: 86 · $43.54Fable 5.2Claude Opus 5.5: 81 · $15.41Opus 5.5Claude Sonnet 5.5: 77 · $7.28Sonnet 5.5GPT-6 Astra: 63 · $4.46AstraQwen3.8 Max: 63 · $18.74 †Qwen3.8 Max†GPT-6.1 Sol: 58 · $0.83SolClaude Haiku 5.5: 53 · $1.32Haiku 5.5Gemini 4 Argon: 52 · $3.57ArgonGemini 4 Hydrogen: 48 · $3.01HydrogenBeam Free: 3 · $0Beam
Each dot is a scored run; the red line joins the Pareto frontier. Costs are API list prices unless marked †. Free models sit in the band on the left. Runs without cost data are left out.

Cost per score point

  • Pareto frontier
  1. Beam Free$0.0003 points
  2. GPT-6.1 Sol$0.01458 points
  3. Claude Haiku 5.5$0.02553 points
  4. Gemini 4 Hydrogen$0.06348 points
  5. Gemini 4 Argon$0.06952 points
  6. GPT-6 Astra$0.07163 points
  7. Claude Sonnet 5.5$0.09577 points
  8. Claude Opus 5.5$0.19081 points
  9. Qwen3.8 Max$0.298†63 points
  10. Claude Fable 5.2$0.50686 points
Cost divided by the panel score. Lower is better. Free models cost nothing per point.
Value metrics of the scored runs
ModelScoreCostCost per pointPoints per $Output tokens per pointPareto frontier
Claude Fable 5.286$43.54$0.5062.04,614Cost · Wall time
Claude Opus 5.581$15.41$0.1905.34,344Cost · Wall time
Claude Sonnet 5.577$7.28$0.09510.63,989Cost · Wall time
GPT-6 Astra63$4.46$0.07114.1389Cost · Wall time
Qwen3.8 Max63$18.74 †$0.2983.47,036–
GPT-6.1 Sol58$0.83$0.01469.9473Cost
Claude Haiku 5.553$1.32$0.02540.25,446–
Gemini 4 Argon52$3.57$0.06914.62,121–
Gemini 4 Hydrogen48$3.01$0.06316.03,378–
Grok 4.730n/a––––
Beam Free3$0$0.000∞464Cost · Wall time

Time

Wall time runs from the agent’s first to its last action and includes rendering and encoding, which are part of the task.

Score versus wall time (log scale)

  • Pareto frontier
Score versus wall timeScatter plot of the latest panel score against the wall time of each scored run, time on a logarithmic axis, with the Pareto frontier drawn as a line.0204060801005102050100200500Wall time (minutes)Panel scoreClaude Fable 5.2: 86 · 1 h 40 minFable 5.2Claude Opus 5.5: 81 · 1 h 7 minOpus 5.5Claude Sonnet 5.5: 77 · 53 minSonnet 5.5GPT-6 Astra: 63 · 14 minAstraQwen3.8 Max: 63 · 4 h 6 minQwen3.8 MaxGPT-6.1 Sol: 58 · 14 minSolClaude Haiku 5.5: 53 · 29 minHaiku 5.5Gemini 4 Argon: 52 · 37 minArgonGemini 4 Hydrogen: 48 · 20 minHydrogenGrok 4.7: 30 · 31 minGrok 4.7Beam Free: 3 · 8.0 minBeam
Each dot is a scored run; the red line joins the runs no other run beats on both time and score.

Where the time went

  • Model generating
  • Tools & waiting
  1. Qwen3.8 Max4 h 6 min97% model
  2. Claude Opus 5.5 (run 2, older spec)2 h40% model
  3. Claude Fable 5.21 h 40 min68% model
  4. Big Pickle1 h 19 min68% model
  5. Claude Opus 5.51 h 7 min83% model
  6. Claude Sonnet 5.553 min73% model
  7. DeepSeek V4.1 Flash Free52 min87% model
  8. Gemini 4 Argon37 min74% model
  9. Claude Haiku 5.529 min78% model
  10. Kimi K3.1 Pro23 min96% model
  11. Gemini 4 Hydrogen20 min71% model
  12. GPT-6.1 Sol14 min91% model
  13. GPT-6 Astra14 min81% model
  14. Beam Free8.0 min100% model
  15. GPT-6.1 Terra1.8 min94% model
  16. GPT-6 Luna1.7 min100% model
Model time is the sum of all model calls, from the request to the last token. Everything else is tool execution, renders, encodes and harness overhead. Runs without per-call timing data are left out.

Generation speed

Tokens per second while the model was actually generating. Tool calls and waits are excluded; time to the first token is included, so this is end-to-end model speed, not the pure decode rate.

Generated tokens per second

  1. Claude Haiku 5.5214.7 tok/sClaude Code
  2. Gemini 4 Hydrogen190.6 tok/sAntigravity
  3. Big Pickledid not finish153.5 tok/sOpenCode
  4. Claude Sonnet 5.5133.0 tok/sClaude Code
  5. Claude Opus 5.5105.3 tok/sClaude Code
  6. Claude Opus 5.5 (run 2, older spec)98.0 tok/sClaude Code
  7. Claude Fable 5.297.4 tok/sClaude Code
  8. DeepSeek V4.1 Flash Freedid not finish90.6 tok/sOpenCode
  9. Gemini 4 Argon67.5 tok/sAntigravity
  10. Kimi K3.1 Prodid not finish58.8 tok/sOpenCode
  11. GPT-6 Lunadid not finish49.4 tok/sCodex
  12. GPT-6.1 Terradid not finish47.1 tok/sCodex
  13. GPT-6 Astra36.2 tok/sCodex
  14. GPT-6.1 Sol36.1 tok/sCodex
  15. Qwen3.8 Max30.9 tok/sOpenCode
  16. Beam Freedid not finish12.3 tok/sOpenCode
Output and reasoning tokens divided by the summed generation time of all calls. Runs that did not finish are marked.

Tokens

Agents re-send their growing context on every call, so most tokens are cache reads. The mix differs a lot between harnesses: some write the cache explicitly, others only read it.

Token mix and cache-hit ratio

  • Uncached input
  • Cache read
  • Cache write
  • Output
  1. Qwen3.8 Max99% cache hit69.2M total
  2. Big Pickle99% cache hit38.1M total
  3. Claude Opus 5.5 (run 2, older spec)89% cache hit26.5M total
  4. Claude Fable 5.291% cache hit25.2M total
  5. Claude Opus 5.597% cache hit23.2M total
  6. Claude Sonnet 5.595% cache hit19.5M total
  7. DeepSeek V4.1 Flash Free97% cache hit19.0M total
  8. Claude Haiku 5.596% cache hit12.3M total
  9. Kimi K3.1 Pro98% cache hit8.8M total
  10. Gemini 4 Argon88% cache hit7.6M total
  11. Gemini 4 Hydrogen87% cache hit6.9M total
  12. GPT-6.1 Sol98% cache hit4.0M total
  13. GPT-6 Astra97% cache hit2.6M total
  14. GPT-6 Luna89% cache hit555k total
  15. GPT-6.1 Terra90% cache hit518k total
  16. Claude Opus 5.5 (run 1, aborted)77% cache hit323k total
  17. Beam Free86% cache hit224k total
Share of each token type in the run’s total. Cache hit = cache reads divided by all input tokens (uncached + cache read + cache write). Output includes reasoning.

Judges

The panel score hides how the judges arrived at it. These charts show what they rewarded, how much they agreed, and how stable the scores were across three separate panels.

What earned the points

  • Landmarks
  • City look
  • Camera & drone
  • Life
  • Overall
  1. Claude Fable 5.286Median
  2. Claude Opus 5.581Median
  3. Claude Sonnet 5.577Median
  4. GPT-6 Astra63Median
  5. Qwen3.8 Max63Median
  6. GPT-6.1 Sol58Median
  7. Claude Haiku 5.553Median
  8. Gemini 4 Argon52Median
  9. Gemini 4 Hydrogen48Median
  10. Grok 4.730Median
  11. Beam Free3Median
Mean of the three judges per rubric category in the latest panel (landmarks 25, city look 25, camera & drone 30, life 10, overall 10). The mean of the parts can differ slightly from the median score.

Judge agreement per video

  • Anthropic
  • OpenAI
  • Google
  • Median
Judge agreement per videoDot plot with one row per video showing each judge’s total score, the range between them, and the median.0255075100Claude Fable 5.2: Anthropic 86, OpenAI 88, Google 85; Median 86Fable 5.2Δ 3Claude Opus 5.5: Anthropic 85, OpenAI 81, Google 75; Median 81Opus 5.5Δ 10Claude Sonnet 5.5: Anthropic 73, OpenAI 77, Google 86; Median 77Sonnet 5.5Δ 13GPT-6 Astra: Anthropic 58, OpenAI 63, Google 68; Median 63AstraΔ 10Qwen3.8 Max: Anthropic 63, OpenAI 53, Google 78; Median 63Qwen3.8 MaxΔ 25GPT-6.1 Sol: Anthropic 58, OpenAI 58, Google 60; Median 58SolΔ 2Claude Haiku 5.5: Anthropic 53, OpenAI 43, Google 91; Median 53Haiku 5.5Δ 48Gemini 4 Argon: Anthropic 52, OpenAI 59, Google 48; Median 52ArgonΔ 11Gemini 4 Hydrogen: Anthropic 41, OpenAI 48, Google 52; Median 48HydrogenΔ 11Grok 4.7: Anthropic 37, OpenAI 29, Google 30; Median 30Grok 4.7Δ 8Beam Free: Anthropic 3, OpenAI 3, Google 4; Median 3BeamΔ 1
One row per video in the latest panel. The bar spans the lowest to the highest judge; the vertical tick is the median.

Score drift across panels

  • moved more than 6 points
  • stable within 6 points
Score drift across panelsSlope chart of each model’s median score in the three blind panels.02040608010011 Nov 202613 Nov 202614 Nov 2026Claude Fable 5.2: 11 Nov 86, 13 Nov 89, 14 Nov 86Fable 5.2 86Claude Opus 5.5: 11 Nov 80, 13 Nov 83, 14 Nov 81Opus 5.5 81Claude Sonnet 5.5: 11 Nov 78, 13 Nov 76, 14 Nov 77Sonnet 5.5 77GPT-6 Astra: 11 Nov 58, 13 Nov 64, 14 Nov 63Astra 63Qwen3.8 Max: 11 Nov –, 13 Nov –, 14 Nov 63Qwen3.8 Max 63GPT-6.1 Sol: 11 Nov 58, 13 Nov 60, 14 Nov 58Sol 58Claude Haiku 5.5: 11 Nov 47, 13 Nov 53, 14 Nov 53Haiku 5.5 53Gemini 4 Argon: 11 Nov 55, 13 Nov 55, 14 Nov 52Argon 52Gemini 4 Hydrogen: 11 Nov –, 13 Nov 45, 14 Nov 48Hydrogen 48Beam Free: 11 Nov 3, 13 Nov 5, 14 Nov 3Beam 3Grok 4.7: 11 Nov 38, 13 Nov 44, 14 Nov 30Grok 4.7 30
The same videos, re-judged blind with fresh letters each time a new model joined. Red lines moved by more than 6 points between two panels.
Rank stability between consecutive panels
PanelsVideos in bothSpearman ρLargest rank changeMean |Δ score|Largest |Δ score|
11 Nov → 13 Nov90.99613.36
13 Nov → 14 Nov100.98823.114

By harness and provider

The model is never measured alone: it runs inside a harness, and the provider decides prices and limits. Free tiers and plan limits ended most of the failed runs.

Runs grouped by harness and provider
HarnessProviderRunsVideosScoredBest scoreTotal costTotal time
Claude CodeAnthropic654Claude Fable 5.2 86$92.216 h 10 min
CodexOpenAI422GPT-6 Astra 63$5.4032 min
OpenCodeOpenCode Zen (free)411Beam Free 3$02 h 19 min
AntigravityGoogle222Gemini 4 Argon 52$6.5857 min
CursorCursor111Grok 4.7 30n/a31 min
OpenCodeOpenCode Go100–$2.8423 min
OpenCodeOpenRouter111Qwen3.8 Max 63$18.744 h 6 min