Anthropic Batch API: 112 of My 1,842 Jobs Never Came Back
说明与用途
使用 Anthropic Batch API 进行夜间任务时,由于结果映射错误导致 6.1% 的隐性失败。通过添加按请求状态检查、使用 custom_id 映射以及处理截断响应,失败率降至 0%,成本下降 50%。
问题
Assuming batch processing status "ended" means all requests succeeded caused silent failures and mis‑aligned results in a production pipeline.
结果
Reduced silent failures from 6.1% to 0%, ensured correct report mapping, and cut batch cost by 50% by using batch for nightly scoring jobs.
工作流
- 1. Create batch with requests, each having a unique custom_id (e.g., session UUID).
- 2. Poll batch until processing_status == "ended".
- 3. Retrieve batch.request_counts to verify counts of succeeded, errored, expired, canceled.
- 4. Stream results JSONL and build a dict mapping custom_id to result entry.
- 5. For each pending session, fetch entry by custom_id; if missing, mark for retry.
- 6. If entry.result.type == "succeeded" and entry.result.message exists, parse JSON; handle stop_reason "max_tokens" by retrying or truncating safely.
- 7. Save report or score; otherwise log error and notify.
- 8. Monitor metrics and alert on high error or expiration rates.
证据
batch.processing_status == "ended" does not mean success. Read request_counts and switch on each result line's result.type
Results come back in arbitrary order. Zipping them against your input list silently attaches the wrong output to the wrong record.
- 记录日期: 2026-09-22T03:30:20.101Z