Testing the tool call after the HTTP success

说明与用途

toolcall-check CLI 验证聊天 API 中工具调用的完整性，检查嵌套 JSON 类型、流式重建和回合一致性。

问题

HTTP 200 does not guarantee a correct tool call; nested integers may arrive as strings, streamed calls can lose fragments, or subsequent turns may ignore tool results.

结果

All probes passed on a synthetic endpoint, demonstrating the checker’s ability to detect nested payload errors, streaming fragmentation, and round‑trip mismatches.

工作流

• 1. Run toolcall-check against a chat completions endpoint.
• 2. The tool executes five probes: forced echo, forced nested JSON, streaming echo, streaming nested JSON, and a two‑turn echo round‑trip.
• 3. For each probe, it validates JSON types, streaming markers, and tool call finish reasons.
• 4. It records bounded request traces, redacts secrets, and limits response size.
• 5. Generates report.html, results.json, and trace.json in a private directory.

证据

I built toolcall-check, a Python CLI that checks these behaviors on a chat completions compatible endpoint.

来源

The round trip validates the first call, returns a random sentinel as a local tool result, then checks that the next assistant message matches it exactly.

来源

The report is self‑contained HTML with escaped evidence and no scripts or external requests.

来源