You cannot validate a field you cannot name, so classification is a precondition for extraction rather than a routing convenience. Three things break in production and all three are classification problems: one upload containing several documents, a document type the model has never seen (which a forced-choice classifier will label confidently and wrongly), and sub-types — because "passport" is not a layout, it is forty of them. And if you measure only end to end, every one of those failures arrives disguised as an extraction problem, so you tune the layer that was already working.
Document automation projects tend to be scoped around extraction, because extraction is what the demo shows and what the pricing is based on. Classification gets treated as plumbing — decide what kind of thing this is, send it to the right place, move on.
That framing is the source of a lot of disappointing pipelines. Commentary from the field puts it bluntly: many implementations automate routing but leave the extraction layer untouched, and the real bottleneck is extraction accuracy together with document classification. The second half of that sentence is the one that gets ignored.
Classification is not routing
Routing decides where a document goes — which queue, which team. Classification decides what the document is, and that determines which extraction model runs, which fields are expected, which validation rules apply, and what a complete result even looks like.
The distinction matters because the two have different tolerances. Routing can be approximate and the system still works, because a human notices a misrouted document and moves it. Classification cannot be approximate, because a misclassified document is extracted against the wrong schema and produces output that is structured, confident and wrong. Nobody notices that by looking at it.
What actually arrives
Three things in that picture break real deployments.
One upload is not one document
Customers send a single PDF containing a passport page, two pages of a bank statement and a utility bill, in arbitrary order, some rotated. The pipeline has to find the boundaries before it can label anything, which makes splitting a classification problem in its own right — usually page-level classification plus a rule about where a document starts.
Teams that assume one upload equals one document discover otherwise within days of launch, and the symptom is extraction output that makes no sense at all.
The open-set problem
This is the failure worth designing against explicitly. Most classifiers are forced-choice: presented with a document, they must select from the classes they know, and they will pick the nearest one with a high score even when the right answer is none of the above.
So a tenancy agreement arriving at a system trained on passports, utility bills and bank statements does not come back as unrecognised. It comes back as a bank statement, confidently. Extraction then runs the bank-statement schema against it and returns a tidy object full of nonsense, with no signal anywhere that something went wrong.
The remedy is cheap and structural: an explicit unknown bucket with a confidence floor beneath which nothing is classified at all, routed to a person. It costs a little throughput and it removes an entire category of silent failure.
Sub-types are where the quality actually lives
"Passport" is not a layout. It is roughly as many layouts as there are issuing countries, each with its own field positions and conventions. Utility bills are worse, because every provider designs its own and redesigns it periodically.
A classifier that answers "passport" has told the extraction layer far less than it appears to. This is why classification accuracy measured at type level can look excellent while extraction quality stays stubbornly poor — the type was right and the sub-type, which is what determines where the fields are, was never asked about.
For a bank onboarding customers across several nationalities, this is usually the single biggest driver of extraction quality, and it is invisible in any metric reported at type level.
Measure the three numbers separately
Here is the measurement trap that keeps teams optimising the wrong layer.
If you only measure end to end, a classification failure looks identical to an extraction failure. The document was labelled wrongly, extracted against the wrong schema, and what reaches your dashboard is missing and incorrect fields. The natural engineering response is to improve extraction — which cannot possibly help, because extraction executed correctly on wrong instructions.
Report splitting accuracy, classification accuracy (at sub-type level) and extraction accuracy as three separate numbers. The moment they are separate, the real bottleneck becomes obvious, and in our experience it is more often the first two than the third. That sits alongside the distinction we drew in field-level accuracy — in both cases, a single blended number hides the thing you need to act on.
Where this fits
Classification is the first of the five steps between an extraction product's output and a usable customer record, which we set out in getting extracted document data into a core banking system. It is also the cheapest of the five to get right and the most expensive to skip, because every downstream step inherits its mistakes and none of them can detect one.
Questions we get asked
Why must classification come first?
Because you can't validate a field you can't name. Which values you expect depends entirely on what the document is — a passport has a document number and expiry, a utility bill has an account number and billing period. Without classification a pipeline is pattern matching and hoping, and can't distinguish a missing field from one that was never going to be there.
Classification versus routing?
Routing picks a queue; classification determines which model runs, which fields are expected and which rules apply. Routing can be approximate because a human fixes a misroute. Classification can't, because a misclassified document gets extracted against the wrong schema and produces confident, structured, wrong output.
What about document types never seen before?
The open-set problem, and the commonest production failure. Forced-choice classifiers must pick a known class and will do so confidently — a tenancy agreement comes back as a bank statement, not as unrecognised. Build an explicit unknown bucket with a confidence floor, routed to a person.
Why do sub-types matter more?
Because a type isn't a layout. "Passport" is roughly as many layouts as there are issuing countries; utility bills are worse. A classifier answering "passport" has told extraction much less than it seems. Type-level accuracy can look excellent while extraction stays poor.
One PDF, several documents?
Splitting comes before classification and is a classification problem itself — usually page-level classification plus a boundary rule. Customers routinely upload a passport page, two statement pages and a utility bill in one file, unordered and sometimes rotated. Assume one upload equals one document and you'll find out within days.
Why measure classification separately?
Because end to end, a classification failure looks exactly like an extraction failure — and you'll improve extraction, which can't help, since it executed correctly on wrong instructions. Report splitting, classification (at sub-type level) and extraction as three numbers and the real bottleneck becomes visible.
We build document processing systems end to end — splitting and classification included, because they decide what the extraction layer can achieve. If your extraction quality has plateaued, measuring the three numbers separately is the fastest way to find out why.