An AI hardware advisor that walks every path — and gets its math checked by the rules in Sanity
Description and use
An AI hardware advisor uses structured Sanity data to provide rule‑based recommendations for running AI models, validating all calculations against stored laws to ensure accuracy across multiple solution paths.
Problem
Provide accurate, rule‑based hardware recommendations for running AI models, including options to buy, rent, or use existing hardware, while ensuring calculations are mathematically correct.
Outcome
The advisor achieved 59/78 checklist compliance in the latest version, with all 11 solution paths evaluated and validated. Validation reduced first‑answer rejections from 14/27 to 8/27, and recomputed laws improved accuracy in key metrics such as KV cache size and electricity cost.
Workflow
- 1. Agent queries Sanity Context MCP for relevant documents (laws, rules, solution paths).
- 2. Agent retrieves structured data and applies rules as a checklist for missing fields.
- 3. Agent performs calculations for each solution path and records results in a hidden check block.
- 4. The check_answer tool recomputes each law’s formula using the agent’s inputs and compares against the agent’s output.
- 5. If discrepancies are found, the agent is prompted to revise; otherwise the recommendation is accepted.
- 6. The final recommendation, including hardware specs and cost estimates, is returned to the user.
Evidence
An agent that answers "what computer do I need to run this AI model?" — including "don't buy, rent" and "you already have enough".
The check covers numbers, completeness, sources and identity. It does not cover whether a verdict makes sense.
Measured, honestly: Blind grader (a different model, sees only the dialogue and a frozen checklist), 9 scenarios × 3 runs, 78 checklist items, same base state for all arms.
The same code checks the base itself: every law variable and rule path must be a real schema field (coverage check), and laws get property tests.
- Record date: 2026-10-03T06:55:43.436Z