4,300 tests, written by an AI: how they were planned, run and enforced

Описание и применение

В проекте ParkEase, созданном с помощью Claude Code, применена стратегия «тесты сначала» с реальной инфраструктурой и обязательными интеграционными наборами, что позволило снизить количество ошибок и обеспечить надёжность кода.

Задача

Ensuring AI‑generated code is correct requires a robust testing strategy that treats tests as contracts rather than a post‑hoc phase.

Результат

Adopting a test‑first, test‑driven workflow with real infrastructure and mandatory suites reduced bugs, ensured critical invariants, and made the testing process a contract that AI‑generated code had to satisfy.

Рабочий процесс

• 1. Write a failing test before code (test‑first).
• 2. Create a test plan entry for each task in the registry.
• 3. Use real PostgreSQL 18 with PostGIS in a throwaway container for integration tests.
• 4. Run four mandatory integration suites (booking concurrency, ledger balance, idempotency, search performance) on every change.
• 5. Perform stress tests (100 drivers racing for 10 slots).
• 6. Run API tests through real HTTP to exercise guards and interceptors.
• 7. Add component tests for new screens and table‑driven tests for data screens.
• 8. Execute Playwright end‑to‑end runs on phone and desktop sizes with accessibility scans.
• 9. Run lint, typecheck, unit suites, mandatory integration suites, dependency audit, and secret scan locally before PR.
• 10. Log gaps such as fixture mismatches and lack of real Android devices.

Доказательства

Tests weren’t a phase at the end. They were the contract the generated code had to meet.

Источник

Integration tests run against a real PostgreSQL 18 with PostGIS in a throwaway container. No mocked database.

Источник

Four integration suites are mandatory on every change: booking concurrency, ledger balance, idempotency, search performance.

Источник

API tests go through HTTP. Tests that call a command directly skip the guards, interceptors, status codes and error envelopes.

Источник

The last gate before any pull request: lint, typecheck, every unit suite, the four mandatory integration suites, a dependency audit and a secret scan, run locally, with the output pasted into the PR.

Источник