I Realized I Couldn't Trust AI With My Research

Описание и применение

Система Research Council обеспечивает проверку доказательств из структурированной базы знаний Sanity, предотвращая недостоверные сравнения архитектур памяти AI.

Задача

Researchers need a trustworthy method to compare AI memory architectures, but existing papers use incomparable benchmarks, models, and metrics, leading to unreliable conclusions.

Результат

Research Council delivers evidence-backed verdicts, preventing fabricated conclusions and highlighting when evidence is lacking.

Рабочий процесс

• 1. Populate Sanity Knowledge Base with 13 research papers and custom instructions for recording results.
• 2. Use Sanity Context tools: initial_context, knowledge_base_search, knowledge_base_read to fetch relevant evidence.
• 3. Run four agents in parallel: Researcher A, Researcher B, Contradiction Hunter, Evidence Auditor.
• 4. Verifier retrieves cited entries via knowledge_base_read and checks quotes word‑for‑word.
• 5. Judge aggregates verified evidence and outputs a verdict.
• 6. Result is a clear indication of whether evidence supports, contradicts, or is insufficient for the research question.

Доказательства

The problem was that almost every approach had results showing that it worked, but those results weren't always directly comparable.

Источник

One full investigation takes just 5 model calls: four agents in parallel, then the Judge.

Источник

Research Council doesn't treat a research question as a prompt that needs an answer. It treats it as an investigation.

Источник