Designing a WhatsApp-First OCR Pipeline with Human Review (Lessons from Fleet Bills)

Description and use

An OCR pipeline for WhatsApp with human review, used in fleet management, reduces manual entry and accounting errors.

Problem

In Indian trucking, drivers send photos of fuel, toll, weighbridge, and repair slips via WhatsApp, leading offices to retype hundreds of images monthly, risking costly ledger errors if OCR output is taken as fact.

Outcome

The pipeline reduces manual retyping, flags inconsistencies, and ensures only validated entries enter accounting, while allowing supervisors to pause automation and reviewers to see explicit reasons for review.

Workflow

• 1. Intake: store original attachment with sender, timestamp, and channel; verify sender is known.
• 2. OCR & field extraction: run OCR supporting Devanagari and Latin, extract fields with per‑field confidence.
• 3. Validation & matching: perform arithmetic consistency, duplicate detection, vehicle/trip matching, and date sanity checks.
• 4. Gate: if all checks pass, auto‑post; otherwise enqueue for review with explicit reasons.
• 5. Review: human corrects or retries extraction; upon approval, post to operational/accounting record linked to original.

Evidence

Real‑world slips are hard: Mixed Hindi and English text, sometimes handwritten

Source

If your pipeline writes the extracted values straight into an accounting ledger, you’ll create confident‑looking wrong numbers.

Source

Only post automatically when every check passes. Anything else goes to a queue with explicit reasons.

Source