An AI hardware advisor that walks every path — and gets its math checked by the rules in Sanity

说明与用途

AI 硬件顾问利用 Sanity 的结构化数据，基于规则提供运行 AI 模型的硬件建议，并通过验证所有计算结果来确保准确性。

问题

Provide accurate, rule‑based hardware recommendations for running AI models, including options to buy, rent, or use existing hardware, while ensuring calculations are mathematically correct.

结果

The advisor achieved 59/78 checklist compliance in the latest version, with all 11 solution paths evaluated and validated. Validation reduced first‑answer rejections from 14/27 to 8/27, and recomputed laws improved accuracy in key metrics such as KV cache size and electricity cost.

工作流

• 1. Agent queries Sanity Context MCP for relevant documents (laws, rules, solution paths).
• 2. Agent retrieves structured data and applies rules as a checklist for missing fields.
• 3. Agent performs calculations for each solution path and records results in a hidden check block.
• 4. The check_answer tool recomputes each law’s formula using the agent’s inputs and compares against the agent’s output.
• 5. If discrepancies are found, the agent is prompted to revise; otherwise the recommendation is accepted.
• 6. The final recommendation, including hardware specs and cost estimates, is returned to the user.

证据

An agent that answers "what computer do I need to run this AI model?" — including "don't buy, rent" and "you already have enough".

来源

The check covers numbers, completeness, sources and identity. It does not cover whether a verdict makes sense.

来源

Measured, honestly: Blind grader (a different model, sees only the dialogue and a frozen checklist), 9 scenarios × 3 runs, 78 checklist items, same base state for all arms.

来源

The same code checks the base itself: every law variable and rule path must be a real schema field (coverage check), and laws get property tests.

来源