Replayable API Smoke Tests in GitHub Actions

说明与用途

在 GitHub Actions 中的 API 烟雾测试通过添加明确的运行 ID、受限等待和证据工件变得可重现且信息丰富。

问题

API smoke tests often pass without revealing which environment, data, or timing caused the success, making failures hard to reproduce and investigate.

结果

Smoke tests become reproducible and informative; failures include run ID, environment, and evidence artifacts, enabling quick triage and retry decisions.

工作流

• Define run contract variables (API_BASE_URL, SMOKE_RUN_ID, POLL_DEADLINE_SECONDS) via env or config.
• Create unique test data using SMOKE_RUN_ID and clean up or tag for janitor.
• Set up GitHub Actions workflow to run on push and workflow_dispatch with steps: checkout, setup-node, install deps, run smoke test with --run-id, always upload artifacts.
• Implement bounded polling for async email checks with deadline and distinguish missing vs rejected messages.
• Generate machine‑readable receipt JSON per run and upload as artifact.
• Review checklist: dispatchable, unique fixtures, bounded wait, sanitized evidence, distinct error types, environment identification, documented dependencies.

证据

Give each run an explicit input, a bounded wait, and an evidence bundle. That turns GitHub Actions from a button that says “probably okay” into a developer tool you can inspect and re-run.

来源

The test should create unique data with SMOKE_RUN_ID, then clean it up when the API allows that.

来源