Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

Anthropic Batch API: 112 of My 1,842 Jobs Never Came Back

说明与用途

使用 Anthropic Batch API 进行夜间任务时,由于结果映射错误导致 6.1% 的隐性失败。通过添加按请求状态检查、使用 custom_id 映射以及处理截断响应,失败率降至 0%,成本下降 50%。

问题

Assuming batch processing status "ended" means all requests succeeded caused silent failures and mis‑aligned results in a production pipeline.

结果

Reduced silent failures from 6.1% to 0%, ensured correct report mapping, and cut batch cost by 50% by using batch for nightly scoring jobs.

工作流

  • 1. Create batch with requests, each having a unique custom_id (e.g., session UUID).
  • 2. Poll batch until processing_status == "ended".
  • 3. Retrieve batch.request_counts to verify counts of succeeded, errored, expired, canceled.
  • 4. Stream results JSONL and build a dict mapping custom_id to result entry.
  • 5. For each pending session, fetch entry by custom_id; if missing, mark for retry.
  • 6. If entry.result.type == "succeeded" and entry.result.message exists, parse JSON; handle stop_reason "max_tokens" by retrying or truncating safely.
  • 7. Save report or score; otherwise log error and notify.
  • 8. Monitor metrics and alert on high error or expiration rates.

证据

batch.processing_status == "ended" does not mean success. Read request_counts and switch on each result line's result.type

来源

Results come back in arbitrary order. Zipping them against your input list silently attaches the wrong output to the wrong record.

来源

  • 记录日期: 2026-09-22T03:30:20.101Z