Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

My Friend Was Tired of AI Answers That Sounded Right. So I Built One That Checks Itself.

Description and use

ARIA is a system that lets a cybersecurity student get answers grounded in their own study materials, using hybrid retrieval and context evaluation.

Problem

A cybersecurity student needed precise answers from his own study materials; generic AI responses often did not match the specific documents he was using.

Outcome

ARIA provides a transparent, modular pipeline that retrieves and evaluates information from a user’s own cybersecurity resources, ensuring answers are grounded in the specific study materials and allowing for easy inspection and replacement of components.

Workflow

  • 1. User uploads or links cybersecurity resources to ARIA’s knowledge base.
  • 2. User asks a question about a topic.
  • 3. ARIA performs hybrid retrieval using ChromaDB and BM25 to find relevant documents.
  • 4. A Judge LLM evaluates the retrieved context and the generated answer.
  • 5. If the answer or retrieval is unsatisfactory, ARIA’s self‑healing engine refines the retrieval and repeats the process.
  • 6. Final answer is presented to the user.
  • 7. User can inspect, replace, or update models and retrieval components as needed.

Evidence

I built ARIA to make that process easier.

Source

ARIA lets him add his own cybersecurity resources and ask questions based on those materials.

Source

I use Llama 3.1 8B through Ollama for generation, with ChromaDB and BM25 for hybrid retrieval.

Source

The system retrieves relevant cybersecurity knowledge, then uses a Judge LLM to evaluate the retrieved context and the generated answer.

Source

If the retrieval or answer isn't good enough, ARIA's self‑healing engine can refine the retrieval and try again.

Source

  • Record date: 2026-10-04T23:25:47.407Z