The Agent's First Command Is the Real Onboarding Test

Description and use

Requiring a toolchain receipt JSON ensures the agent’s first command matches the repository’s actual test setup, reducing onboarding friction.

Problem

Automated coding agents often claim they understand a repository but then run incorrect test commands, leading to onboarding failures and wasted effort.

Outcome

By enforcing a toolchain receipt, the agent’s first command aligns with the repository’s actual test setup, reducing onboarding friction and preventing incorrect test runs.

Workflow

• 1. Provide repository context (package.json, CI snippets, directory tree).
• 2. Request a small failing-test fix from the agent.
• 3. Agent outputs a toolchain receipt JSON file via a helper script.
• 4. Verify receipt includes correct package manager and inferred test command.
• 5. If receipt is missing or incorrect, reject the patch.
• 6. Once receipt is valid, accept and apply the patch.

Evidence

I made a tiny script the agent must execute, not narrate, and I started calling the JSON it prints a toolchain receipt.

Source

If that receipt cannot name the real test runner, the session is still guessing, no matter how polished the explanation sounds.

Source