I Built an Open-Source QA Agent Because AI Shouldn’t Grade Its Own Homework

Описание и применение

MaruCheck — независимый слой проверки, который сравнивает код, генерируемый ИИ, с постоянным контрактом качества, обнаруживает семантические отклонения и учитывает исторические регрессии, обеспечивая, чтобы продукт соответствовал исходным требованиям.

Задача

AI coding agents can generate both code and tests, risking semantic drift where the same misunderstanding propagates into passing tests that mask incorrect implementations.

Результат

MaruCheck prevents semantic drift by enforcing a Quality Contract, performs risk‑based test selection, and recalls historical regressions, leading to more reliable AI‑generated code that truly meets product requirements.

Рабочий процесс

• Define a Quality Contract for the feature
• Coding agent implements the feature
• MaruCheck scans the repo and analyzes the git diff
• Risk-based verification selects relevant tests
• MaruCheck runs tests and compares results to the contract
• If a semantic drift is detected, MaruCheck blocks the PR and reports the issue
• Developer fixes the implementation and re‑runs MaruCheck
• Repeat until all checks pass

Доказательства

If the model misunderstands the requirement once, it can easily carry that same misunderstanding into the tests.

Источник

MaruCheck verifies. If it finds something: BLOCKING ISSUE SUB-004 violated.

Источник