Automated QA for 56,000 browser games: how we catch broken embeds before players do

说明与用途

自动化 QA 流程每晚检查 56,000 款浏览器游戏，检测嵌入错误，将误报降至每周仅数例。

问题

Maintaining a large catalog of embedded HTML5 games is impossible with manual QA; broken embeds cause player loss.

结果

The pipeline runs on a single modest server, covering the full catalog in ~6 hours nightly with concurrency 8, costing far less than a part‑time QA person. False positives dropped to a handful per week after rule refinements.

工作流

• 1. Load game page and wait for embed iframe.
• 2. Verify iframe source returns HTTP 200; flag 404/403 immediately.
• 3. Wait for first paint inside iframe to confirm canvas rendering.
• 4. Listen for console errors; flag if many uncaught exceptions.
• 5. Capture screenshot of game area and diff against last known‑good image; flag if significant change.
• 6. If flagged, retain screenshots, logs, and last good date in review queue.
• 7. Human reviews queue each morning, deciding to fix embed URL, replace archived file, or delist game.

证据

Manual QA at this scale is impossible. Nobody is clicking through 56,000 games to check they still load.

来源

The screenshot diff was the breakthrough. HTTP checks alone missed about 30 percent of real failures, games that loaded fine technically but rendered nothing.

来源

This brought false positives down to a handful per week, each one worth a human look.

来源