Three different numbers get called "accuracy" and vendors quote the most flattering one. Character accuracy compares letters. Field accuracy requires every letter in a field to be right. Document accuracy requires every field to be right — and that is the only one that predicts how many people you still need. They compound downward, which is how a system advertised at 99% still sends a meaningful share of documents to a human. Before accepting any figure, ask which of the three it is, what sample it came from, and what share of fields the system declined to answer.
Every document-automation proposal arrives with a number on it. Ninety-nine per cent. Ninety-nine point five. Occasionally 99.9. The numbers are rarely dishonest and almost always useless, because the word they attach to is ambiguous and the ambiguity runs in the vendor's favour.
This matters most in onboarding and KYC, where the documents are identity documents, the fields are things like document number and date of birth, and a wrong value is not a typo but a compliance event. It is worth being precise about what is being measured.
The same read, three different scores
A commonly cited example makes it concrete. An invoice number should read INV-20260412. The system returns INV-2O260412 — a capital letter O where a zero belongs. One character in twelve is wrong.
The same effect applies to names, which are the fields KYC actually turns on. Ilan read as Ivan is three characters out of four correct and a completely wrong identity. There is no partial credit in a sanctions screen.
The compounding is the real problem
Field accuracy is the honest measure of a single value. But onboarding does not consume single values — it consumes documents, and a document is only straight-through if every field on it is right.
Consider the arithmetic. If a form carries twelve extracted fields and each is independently 99% accurate, the chance that all twelve are right is 0.9912, or roughly 89%. More than one document in ten goes to a human, from a system that could truthfully advertise 99% field accuracy.
Errors are not actually independent — a poor scan degrades many fields at once, so failures cluster and the real document accuracy is somewhat better than that product suggests. But the direction is right and it is the direction that matters: document accuracy is always worse than field accuracy, which is always worse than character accuracy. Quote the top of that stack and the operational picture looks nothing like the number.
What thresholds does published guidance suggest?
Varied, and mostly vendor-authored, so treat all of it as indicative rather than authoritative. One source suggests critical identity fields — document number, date of birth — should reach 98 to 99%, with less critical fields such as place of issue or subdivision code acceptable at 95 to 97%. Another puts the straight-through processing threshold at 99.9%, and specifies that this is at field level, not aggregate document accuracy. For messy mixed batches of invoices, purchase orders and utility bills, one vendor reports typical averages of 91 to 96% before human review.
We quote these to show the spread, not to endorse them. None is a standardised benchmark, several conflict, and crucially none was measured on your documents, your capture process or your population. A figure produced on someone else's sample is marketing; the same figure produced on yours is an acceptance criterion. This is the same distinction we drew about vision systems in why vision accuracy drops between the demo and the floor — an accuracy number describes a dataset, never a model.
The deferral trick
One more thing to ask about, because it is rarely volunteered. Most extraction systems can attach a confidence score to a read and route low-confidence values to a human rather than answering.
That is good design. It is also a way to make accuracy look spectacular. A system that declines to answer 40% of fields and is right about the 60% it does answer can report excellent accuracy while quietly handing nearly half the work back to your operations team. The accuracy figure and the deferral rate are only meaningful together, and a vendor who cannot give you both has not measured the thing you care about.
How to measure it properly
On your own labelled sample, with scanned and photographed documents in the proportions you really receive — not a clean set someone assembled to demonstrate with. Report field accuracy per field, since your critical fields and your cosmetic ones warrant different thresholds. Report document accuracy separately, because that is the number that predicts headcount. Track the deferral rate alongside both. And measure first-pass accuracy before any human correction, because after correction every system scores perfectly and the measurement is worthless.
One cheap thing to check first: image quality. One source notes that resolution below 300 DPI causes a measurable drop in character recognition. Phone photographs taken at an angle, compressed scans, screenshots of documents and anything behind lamination are the usual culprits. Improving capture at the source often moves the numbers further than changing the engine behind them, and it costs a fraction as much.
The question to open with
When a vendor hands you an accuracy figure, three questions settle it. Which of the three measures is this? What sample was it measured on? What was the deferral rate?
If all three come back clearly, you are talking to someone who has actually run the system in production. If any of them produces hesitation, the number was never a measurement in the first place.
Questions we get asked
Character accuracy versus field accuracy?
Character accuracy compares letters independently. Field accuracy requires every character in the field to be right. INV-2O260412 for INV-20260412 is about 92% character-accurate and 0% field-accurate. Same for names — Ilan read as Ivan is a wrong identity with no partial credit.
Why does document accuracy matter more?
Straight-through processing needs the whole document right. Twelve fields at 99% each gives roughly 89% of documents fully correct — more than one in ten to a human, from a "99% accurate" system. Errors cluster so reality is a little kinder, but the ordering never changes.
What should KYC extraction reach?
Published guidance (mostly vendor-authored) suggests 98–99% for critical fields like document number and date of birth, 95–97% for less critical ones, and one source puts straight-through processing at 99.9% at field level. Mixed batches reportedly average 91–96% before review. None are standardised, and none were measured on your documents.
Why are vendor claims hard to compare?
Because "accuracy" does three jobs and marketing rarely says which. Character, field or document level; clean samples or realistic scans; with or without a confidence threshold quietly deferring uncertain reads. Without the measure, the sample and the deferral rate, two numbers aren't comparable.
How should we measure it?
Your own labelled sample, in the proportions you actually receive. Field accuracy per field, not blended. Document accuracy reported separately. Deferral rate tracked alongside. And first-pass, before human correction — after correction everything scores 100%.
Does image quality matter much?
Substantially, and it's the cheapest fix. One source notes below 300 DPI causes a measurable drop. The usual culprits are angled phone photos, compressed scans, screenshots of documents and lamination. Improving capture often beats changing the extraction engine, at a fraction of the cost.
We build document processing systems that run on client infrastructure and write into core banking, which means we care about the document-level number rather than the headline one. If you are evaluating a vendor, the deferral rate is the question that separates the serious answers from the brochures.