Free AI Compute for SQL Review: A Structured Debate and a Decision Rule

说明与用途

使用MonkeyCode免费层和PostgreSQL沙箱进行AI生成SQL审查者的每周盲校准的自动化流程。

问题

Teams lack a systematic way to measure how accurately humans review AI‑generated SQL, leading to costly mistakes that go unnoticed.

结果

Teams can run weekly blind calibration tests for ~40k tokens, covering ~250 rounds on the free tier, and obtain actionable accuracy metrics that inform review practices and gate decisions.

工作流

• Create sandbox schema with python calibrate_reviewer.py init
• Define or extend test cases with ground‑truth verdicts
• Run each case through EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON) and wrap writes in a transaction that rolls back
• Emit review_pack.json containing plans but no verdicts
• Have reviewers classify each query as correct, slow, or dangerous in a separate file
• Score the verdicts with python calibrate_reviewer.py score review_verdicts.json and review the accuracy report

证据

A blind calibration harness costs almost nothing when tokens and a sandbox server are free, so the measurement can run weekly instead of annually.

来源

The harness below is a condensed version of the one I run for review calibration. It targets Postgres 14 or newer, works on the free server option or any local sandbox, and never executes a write outside a rolled‑back transaction.

来源