I Broke My Own GitHub Trending Scraper 5 Ways on Purpose. All 5 Builds Stayed Green.

说明与用途

一份自动化配方，向 GitHub Actions 采集 Trending 数据的 CI 流程添加完整性检查，确保在发布前验证行数、描述覆盖率、仓库名格式和快照年龄。

问题

A CI pipeline that aggregates GitHub Trending data was only checking for build success, not data integrity, allowing corrupted or incomplete data to be published without detection.

结果

All 201 runs succeeded in CI, but five intentional HTML corruptions caused silent failures; the new canary prevented publication of corrupted data and detected stale snapshots.

工作流

• 1. Schedule GitHub Actions cron (0 */6 * * *) to run scraper.
• 2. Use urllib.request to GET six trending pages (weekly, python, javascript, typescript, rust, go).
• 3. Apply regex parser to extract repo name, description, language, stars, forks, weekly gain.
• 4. Write trending.json, summary.json, index.html and upload as Pages artifact.
• 5. Before publishing, run completeness canary: check row count ≥15, description coverage ≥80%, repo name contains exactly one '/', and snapshot age ≤8h.
• 6. If any assertion fails, block publish and flag the build as failed.

证据

Every single test returned normally. No exception, no non-zero exit, no failed build.

来源

Four assertions, none of which is "did it crash": Row floor, Description coverage ratio, Name shape, Snapshot age.

来源