Your Extraction Is 98% Accurate. One Document in Three Still Needs a Human.

Description and use

High per‑field accuracy does not ensure error‑free documents. Implementing deterministic validation rules and provenance tracking improves throughput and reduces manual review.

Problem

Document extraction demos often report high per‑field accuracy (≈98 %) but still leave a significant fraction of documents with errors, eroding real‑world productivity gains.

Outcome

Implementing deterministic validation rules and provenance tracking reduces the proportion of documents requiring human intervention from ~33 % to a lower, measurable rate, improving straight‑through throughput and overall efficiency.

Workflow

• 1. Classify the incoming document (invoice, credit note, etc.).
• 2. Extract structured fields (supplier, date, invoice number, line items, tax, total).
• 3. Validate extracted data: check arithmetic consistency, master data matches, duplicate detection, and date/range rules.
• 4. Flag any inconsistencies for human review.
• 5. Post validated records into the business system, ensuring correct supplier or case linkage and no duplicates.
• 6. Maintain provenance metadata (page, bounding box) for each field to aid review and debugging.

Evidence

Accuracy figures are almost always quoted per field. Your users experience the system per document.

Source

If each of twenty fields is right 98 times out of 100, the chance that every field on a document is right is 0.98 to the power of 20, which is about 0.67.

Source

The strong signal is checking extracted values against things you already know.

Source

Keep provenance on every field.

Source