Claude Fable 5.1 Agent Test: Sonnet Tied It at 1/8 the Cost

Описание и применение

Все три модели корректно завершили проверку, но Sonnet 5.5 оказался самым быстрым и дешевым, в то время как Fable 5.1 был самым дорогим и медленным.

Задача

Determine which Claude model is most cost‑effective for a month‑end close that requires reading multiple CSV exports and matching payments to invoices.

Результат

All three models answered all ten questions correctly. Sonnet 5.5 was fastest (30 s) and cheapest ($0.087) in round 1, while Fable 5.1 was slowest (85 s) and most expensive ($0.70). In round 2, Sonnet remained fastest (157 s) and cheapest ($0.56), Fable cost $3.74 for one run, and Opus cost $1.02.

Рабочий процесс

• 1. Prepare three CSV exports: invoice list, bank statement, card‑processor report.
• 2. Define ten reconciliation questions and ten hidden traps.
• 3. Provide three tools: list_files, read_file (max 25 lines), calculate.
• 4. Run each model (Fable 5.1, Opus 5.5, Sonnet 5.5) in a Python script using Anthropic API with prompt caching.
• 5. Record tool calls, turns, cost, and score per run.
• 6. Score each run with code that awards one point per correct answer and requires a single JSON object.
• 7. Compare median time, cost, and turns across models.

Доказательства

All three models found all ten traps.

Источник

Cost per month‑end close, round 1 (median): Sonnet 5.5 $0.087, Opus 5.5 $0.20, Fable 5.1 $0.70.

Источник

Sonnet was also the fastest in both rounds: 30 s and 2.6 min, against 85 s and 6.2 min for Fable.

Источник

Fable worked one page at a time. In round 2 it made 80 tool calls over 81 model turns, one file page per turn.

Источник

Prompt caching is what made Fable affordable here.

Источник