What I learned building a RAG assistant for scanned Hindi and English university circulars

Description and use

A RAG assistant was built for scanned Hindi and English university circulars, employing OCR, table reconstruction, hybrid retrieval, and answer verification to deliver accurate, citation‑rich responses.

Problem

University staff needed to find answers in a large pile of scanned Hindi and English circulars that were photocopied at an angle, full of ruled tables, and difficult to search.

Outcome

The assistant now provides accurate, citation‑rich answers to staff queries about university circulars, with robust OCR, table reconstruction, hybrid retrieval, and verification steps, all deployed in a simple CI/CD pipeline.

Workflow

• Detect script per page and run Tesseract with the appropriate language model
• Rebuild tables cell by cell using ruled line detection
• Flag low‑confidence words for reviewer visibility
• Embed each paragraph and table row with location metadata into PostgreSQL with pgvector
• Rewrite user query into English and Hindi queries
• Run vector search for similar passages
• Run exact matching for codes, dates, and rare tokens
• Merge and re‑rank results with a cross‑encoder model
• Generate answer from retrieved passages with numbered citations
• Run a second model to verify answer against sources
• Stream answers to the user
• Implement draft‑to‑active approval workflow, role‑based visibility, bulk import, automatic expiry, two‑step lockouts, and an automatic answer‑quality test in CI
• Deploy via GitHub Actions to an AWS server using Docker and Caddy
• Expose a single API used by a Next.js web app and a Flutter Android app

Evidence

If the text coming out of OCR is broken or wrong nothing that follows can be good.

Source

Each paragraph. Every table row gets embedded and stored in PostgreSQL with pgvector.

Source

The re‑ranker was the biggest improvement in the whole project.

Source