A Senior Pairing Stopped Invented API Load With One Kept Decision

说明与用途

与高级工程师配对冻结真实流量混合，随后仅生成使用该混合的测试，消除虚构负载并确保性能指标的准确性。

问题

Generated API load tests often use synthetic traffic that does not reflect real production patterns, leading to misleading performance metrics and wasted review effort.

结果

By freezing the traffic mix before code generation, the team eliminated invented load patterns, reduced reviewer conflicts, and ensured that performance metrics reflected real production traffic.

工作流

• 1. Senior engineer and implementer agree on a 10‑minute capture window and its traffic class.
• 2. They list the routes covering ≥90% of observed calls in that window.
• 3. They set a realistic maximum concurrency and separate latency budgets from observations.
• 4. They define failure conditions for any generated client that adds routes or weights not present in the mix.
• 5. They record who can refresh the mix file and require a pull‑request template quoting the new capture window.
• 6. The implementer writes the mix JSON file and a loader that validates routes, weights, and think times.
• 7. A gate script rejects any test that references missing or empty mix files or adds unknown routes.
• 8. Generated test harnesses are allowed only to read the validated mix and iterate over it.
• 9. Reviewers enforce the rule that hard‑coded mixes in code are a pairing violation.
• 10. The process is documented in a shared scratch buffer for future reference.

证据

Generated API tests stay honest only after pairing freezes a captured request mix, before any model writes load code.

来源

The kept decision was narrow on purpose, so later models could not widen it through helpful extras.

来源