The Second Turn Forgot the First: A Payload Debugging Retrospective

Описание и применение

Проблема возникла из‑за некорректного формата истории диалога, отправляемого клиентом после прерванного потока. Решение – аудит и нормализация payload‑а на стороне клиента, удаление пустых и дублированных ролей, добавление регрессионного теста.

Задача

An AI coding assistant produced incorrect output after an interrupted streaming turn, causing the assistant to “forget” the first user request.

Результат

By auditing and normalizing the conversation payload at the client boundary, the assistant no longer misinterprets truncated or empty turns, eliminating the false “memory loss” and improving reliability on free tiers.

Рабочий процесс

• 1. Log every outgoing request body (redacted) to a JSONL file.
• 2. After each turn, print the role sequence: system, user, assistant, user…
• 3. Flag empty content, duplicated roles, and assistant fragments shorter than a word.
• 4. Reproduce the issue with an exact transcript, not memory.
• 5. Normalize history by filtering out empty turns and enforcing strict role alternation.
• 6. Add a regression test that aborts a stream, builds the next payload, and asserts the shape of the history.
• 7. If payload is clean, escalate to the provider with evidence.

Доказательства

The variant I actually caught in my logs was even nastier: a completely empty assistant turn, pushed into history by an aborted stream, which broke the role alternation entirely.

Источник

Add a regression test that aborts a stream, builds the next payload, and asserts the shape of the history.

Источник