A grayscale TIFF once crashed our text detection. Nothing was wrong with it. It just stored 16 bits per pixel, and most image code expects 8.
Every production parser collects files like that one. If you plan to build your own document parser on an LLM, this post is a preview of your support queue.
Key Takeaways
- A document parser prototype on top of an LLM takes a weekend. Making it survive real customer files takes years.
- Most failures happen before the LLM sees the document. Fonts map characters wrongly, page boxes start below zero, scans use 16-bit pixels and watermarks leak into the text.
- A better model does not fix these bugs. It only guesses better at input that is already broken.
- Parseur's code base holds an estimated 150 to 300 fixes for odd real-world input, built over 4,588 commits since 2015 and guarded by 294 test files.
- Each fix started with a customer file that failed. A team that builds its own parser finds the same files one support ticket at a time.
- Build when documents come from a few known sources whose format never changes. Buy when they come from customers and suppliers.
A production document parser, by the numbers
What 11 years of real customer files leave behind
- commits in Parseur's code base since 2015
- 4,588
- of them, by our estimate, fix odd real-world input
- 150 to 300
- test files keep those bugs from coming back
- 294
- documents processed, and they still surprise us
- 100M+
Why your weekend parser breaks on the first real customer file
A weekend document parser fails on real files because the hard part is not the AI, it is everything that turns a file into something the AI can read. The LLM gets text or pixels. If the font lies, the page box is odd or the image format is unusual, the model reads garbage and answers with confidence anyway.
Where a document parser breaks
All six files in this post broke the pipeline before the LLM saw them
Step 1
Customer file
PDF, scan or image, from any sender
Step 2: where the six files broke
Turning the file into text and pixels
-
Fonts and encodings
Files 1 to 3: (cid:72) codes, doubled letters, a font that lies about its encoding
-
Page geometry
File 4: a page box that starts below zero
-
Image formats
File 5: a 16-bit grayscale scan
-
Noise
File 6: a DRAFT watermark inside the total
Step 3
LLM
Reads whatever step 2 hands it, broken or not
It worked on your test files because your test files were simple and tidy.
Real customer files are complex and hairy. We have been fixing them since 2015. Our code base counts 4,588 commits, and we estimate that 150 to 300 of them fix odd real-world input. 294 test files make sure those bugs stay dead, and many of them are the exact customer document that broke something.
Here are six of our favorites.
The PDF shows text. The text layer disagrees.
A PDF that shows text on screen does not always store that text in a readable way. Three files proved it.
1. (cid:72) means "e", if you add 29
Some PDFs do not store letters. They store the number of each letter's drawing inside the font, a bit like a seat number in a theater. pdfminer, the library we use to read PDF text, cannot turn that number back into a letter, so it returns a code such as (cid:72).
The trick: many of these fonts list their drawings in the same order, the standard Mac order. In that list, the space is drawing number 3. In ASCII, the numbers computers use for letters, the space is 32. Every letter is off by the same 29. Add 29 to 72 and you get 101, the ASCII number for "e".
Without the fix, an invoice total reads as a row of (cid:...) codes. The LLM sees noise and has to guess.
2. TToottaall
Some PDF tools draw each character twice at the exact same place, probably to fake a bold font. On screen it looks fine. In the text layer, "Total" becomes "TToottaall".
The fix removes any character that sits at the exact same position as another one. Simple, once you know it happens.
3. The font that lies about its own encoding
The PDF standard says a TrueType font maps its characters in UTF-16BE. Some fonts in the wild claim UTF-8 in their character map. pdfminer believes them and reads garbage.
We patched pdfminer itself to ignore that claim and decode those fonts the way they were actually written. An open-source library gets you the spec. The patches on top of it are your job.
Where does a page start? Not always at zero
4. A valid A4 page that is 0 pt high
A PDF page box is two corners: lower left and upper right. Most PDFs start at the point (0, 0), so the last two numbers happen to be the width and the height. Code that reads only those two numbers works on almost every file.
Then a PDF arrived with a page box of [0, -841.89, 595.28, 0]. That is a normal A4 page that starts below zero. Read the lazy way, it is 0 pt high. Our resolution check rejected the document as invalid. Every PDF viewer opens it without a blink.
Images with more bits than you planned for
5. The 16-bit scan
Back to the TIFF from the introduction. Its pixels used 16 bits of gray, and the conversion before OCR text detection broke. The fix converts the image to 8 bits from its own range of values. That detail matters. Some scanners write 12-bit data in a 16-bit file, and a plain conversion would turn the whole scan black.
Noise that reads like text
6. The DRAFT watermark that joins the sentence
A diagonal "DRAFT" watermark is text too. Its letters land between the real words, and the extracted text can turn into "Total DR 1,250.00 AFT". Your LLM will try very hard to make sense of that.
The fix drops characters that are slightly rotated. Real text sits flat on the line. A watermark leans.
The real cost of building: your customers become your test suite
Every fix above started with a customer file that failed, and none of them can be found by reading the PDF specification alone. A team that builds its own document parser meets the same files, in its own order, through its own support tickets.
Each one runs the same loop: a confused user, a sample file, hours to find the cause, a fix, and a test so it never comes back. Now multiply by a few hundred.
An LLM does not skip that loop. It reads what the extraction gives it. When the extraction is wrong, the model is wrong with confidence, and nobody notices until a total lands in the wrong column. Our article on LLMs for document processing covers the model side of that story, and the build vs buy cost comparison puts numbers on the engineering side for invoices.
Won't the next model fix this?
Models improve every few months. None of the six fixes above would go away. A smarter model that receives (cid:72) in place of "e", or "DR 1,250.00 AFT" in place of a total, can only make a better guess. Your model vendor ships better reasoning. It does not ship your PDF pipeline.
When building your own parser is the right call
Build it when your documents come from a few known sources and their format never changes. A fixed export from your own system is a good example. The weekend prototype may be all you ever need.
Buy when files come from customers, suppliers and their scanners. Then every edge case they can send becomes yours, and it arrives as a support ticket.
Bring us your worst file
Yes, we sell a document parser, so read this post with that in mind. The six files above are still real, and so are the support tickets behind them.
We have processed over 100 million documents, and they still surprise us. The next odd file in your inbox has a good chance of being one we have already met.
So send us the file that broke your last tool. If it breaks Parseur too, fixing it is our problem.
Last updated on
