The Agent's First Command Is the Real Onboarding Test

Описание и применение

Обязательное подтверждение команды тестирования через JSON‑receipt устраняет ошибки агента при запуске тестов, ускоряя и упрощая процесс онбординга.

Задача

Automated coding agents often claim they understand a repository but then run incorrect test commands, leading to onboarding failures and wasted effort.

Результат

By enforcing a toolchain receipt, the agent’s first command aligns with the repository’s actual test setup, reducing onboarding friction and preventing incorrect test runs.

Рабочий процесс

• 1. Provide repository context (package.json, CI snippets, directory tree).
• 2. Request a small failing-test fix from the agent.
• 3. Agent outputs a toolchain receipt JSON file via a helper script.
• 4. Verify receipt includes correct package manager and inferred test command.
• 5. If receipt is missing or incorrect, reject the patch.
• 6. Once receipt is valid, accept and apply the patch.

Доказательства

I made a tiny script the agent must execute, not narrate, and I started calling the JSON it prints a toolchain receipt.

Источник

If that receipt cannot name the real test runner, the session is still guessing, no matter how polished the explanation sounds.

Источник