Testing the tool call after the HTTP success

Описание и применение

CLI‑утилита toolcall-check проверяет корректность вызовов инструментов в чат‑API, включая типы вложенных данных, восстановление потоковых фрагментов и согласованность результатов в двух‑тактном взаимодействии.

Задача

HTTP 200 does not guarantee a correct tool call; nested integers may arrive as strings, streamed calls can lose fragments, or subsequent turns may ignore tool results.

Результат

All probes passed on a synthetic endpoint, demonstrating the checker’s ability to detect nested payload errors, streaming fragmentation, and round‑trip mismatches.

Рабочий процесс

• 1. Run toolcall-check against a chat completions endpoint.
• 2. The tool executes five probes: forced echo, forced nested JSON, streaming echo, streaming nested JSON, and a two‑turn echo round‑trip.
• 3. For each probe, it validates JSON types, streaming markers, and tool call finish reasons.
• 4. It records bounded request traces, redacts secrets, and limits response size.
• 5. Generates report.html, results.json, and trace.json in a private directory.

Доказательства

I built toolcall-check, a Python CLI that checks these behaviors on a chat completions compatible endpoint.

Источник

The round trip validates the first call, returns a random sentinel as a local tool result, then checks that the next assistant message matches it exactly.

Источник

The report is self‑contained HTML with escaped evidence and no scripts or external requests.

Источник