Automating Data Quality Checks in GitHub Actions with DataGuard AI

说明与用途

使用 DataGuard AI 在 GitHub Actions 中自动化数据质量检查，可在数据进入报告前发现错误，生成 JSON 和 HTML 报告，供开发者在工作流工件中查看。

问题

Data quality issues can slip through ETL pipelines and affect downstream reports before detection.

结果

Automated scans run on every push or pull request to main, producing reports that expose missing emails, null values, and out‑of‑range ages. The workflow does not fail automatically on findings; a separate policy step is needed to enforce quality gates.

工作流

• Install DataGuard AI (python -m pip install dataguard-ai==0.2.1)
• Prepare sample data (customers.csv) and configuration (dataguard.yml)
• Run local scan (dataguard scan customers.csv --config dataguard.yml --json-out report.json --html-out report.html)
• Create GitHub Actions workflow (.github/workflows/data-quality.yml) that checks out code, sets up Python, installs DataGuard AI, runs the scan, and uploads reports as artifacts

证据

DataGuard AI will inspect the dataset and write two reports: report.json — structured findings suitable for automation; report.html — a human-readable report for reviewing findings

来源

Important limitation: DataGuard AI v0.2.1 generates findings, but the workflow above does not automatically fail merely because a quality issue is detected.

来源

Create a file called customers.csv: ... This intentionally includes a missing email address and an invalid negative age.

来源