I Realized I Couldn't Trust AI With My Research

Description and use

Research Council provides evidence verification from a structured Sanity Knowledge Base, preventing unreliable comparisons of AI memory architectures.

Problem

Researchers need a trustworthy method to compare AI memory architectures, but existing papers use incomparable benchmarks, models, and metrics, leading to unreliable conclusions.

Outcome

Research Council delivers evidence-backed verdicts, preventing fabricated conclusions and highlighting when evidence is lacking.

Workflow

• 1. Populate Sanity Knowledge Base with 13 research papers and custom instructions for recording results.
• 2. Use Sanity Context tools: initial_context, knowledge_base_search, knowledge_base_read to fetch relevant evidence.
• 3. Run four agents in parallel: Researcher A, Researcher B, Contradiction Hunter, Evidence Auditor.
• 4. Verifier retrieves cited entries via knowledge_base_read and checks quotes word‑for‑word.
• 5. Judge aggregates verified evidence and outputs a verdict.
• 6. Result is a clear indication of whether evidence supports, contradicts, or is insufficient for the research question.

Evidence

The problem was that almost every approach had results showing that it worked, but those results weren't always directly comparable.

Source

One full investigation takes just 5 model calls: four agents in parallel, then the Judge.

Source

Research Council doesn't treat a research question as a prompt that needs an answer. It treats it as an investigation.

Source