Autonomous Deep Research for AI Agents: Eliminating Hallucinated Citations and 85% Token Burn in Python

Описание и применение

Автономные пайплайны глубоких исследований теперь используют agent-deep-research, сокращая потребление токенов более чем на 85 % и обеспечивая криптографически проверяемые цитаты.

Задача

Deep research pipelines consume 200k–500k tokens per question and produce hallucinated citations.

Результат

Token usage reduced by >85% and all citations are cryptographically verified, eliminating hallucinations.

Рабочий процесс

• 1. Fetch candidate sources via free endpoints (DuckDuckGo, Wikipedia, ArXiv).
• 2. Partition content into 250‑word passages and index with an in‑memory BM25 lexical matrix.
• 3. Extract top‑k highest‑density factual passages matching query sub‑claims.
• 4. Stamp each fact with a SHA‑256 hash of the source text.
• 5. Feed distilled facts to an LLM synthesizer to generate a report with 100% verifiable citations.
• 6. Optionally run the engine as an MCP server for agent integration.

Доказательства

Building autonomous deep research pipelines (like Stanford STORM or OpenAI Deep Research) typically faces two massive hurdles: Insane Token Burn and Hallucinated Citations.

Источник

To solve this, we engineered agent-deep-research — a local-first Python engine that slashes token consumption by over 85% while guaranteeing zero hallucinated citations through cryptographic SHA-256 provenance ledgers.

Источник

Raw Sources (250,000 tokens) | v [BM25 Fact Distiller] --> Cuts 87% Token Waste | v Distilled Facts (<18,000 tokens) + SHA-256 Hash Ledger | v [LLM Synthesizer] --> 100% Verifiable Citations

Источник

pip install agent-deep-research

Источник

python -m agent_deep_research.server

Источник