A Senior Pairing Stopped Invented API Load With One Kept Decision

Description and use

Pairing with a senior engineer freezes a real traffic mix before code generation, eliminating invented load patterns and ensuring accurate performance metrics.

Problem

Generated API load tests often use synthetic traffic that does not reflect real production patterns, leading to misleading performance metrics and wasted review effort.

Outcome

By freezing the traffic mix before code generation, the team eliminated invented load patterns, reduced reviewer conflicts, and ensured that performance metrics reflected real production traffic.

Workflow

• 1. Senior engineer and implementer agree on a 10‑minute capture window and its traffic class.
• 2. They list the routes covering ≥90% of observed calls in that window.
• 3. They set a realistic maximum concurrency and separate latency budgets from observations.
• 4. They define failure conditions for any generated client that adds routes or weights not present in the mix.
• 5. They record who can refresh the mix file and require a pull‑request template quoting the new capture window.
• 6. The implementer writes the mix JSON file and a loader that validates routes, weights, and think times.
• 7. A gate script rejects any test that references missing or empty mix files or adds unknown routes.
• 8. Generated test harnesses are allowed only to read the validated mix and iterate over it.
• 9. Reviewers enforce the rule that hard‑coded mixes in code are a pairing violation.
• 10. The process is documented in a shared scratch buffer for future reference.

Evidence

Generated API tests stay honest only after pairing freezes a captured request mix, before any model writes load code.

Source

The kept decision was narrow on purpose, so later models could not widen it through helpful extras.

Source