OCR with a Cache and Distrust
Why an OCR pipeline for receipt data only feels reliable once you secure every recognized value with a hash cache, a confidence gate and plausibility rules instead of believing it.
OCR with a Cache and Distrust
In its TSV output format, Tesseract returns a confidence value for recognized words, usually between zero and one hundred; structural lines can carry -1. This internal score is a useful signal, but not a calibrated probability that the word is correct. image_to_string doesn’t return it. If you only take the string, you throw away an important piece of information about the recognition.
I noticed this in a small inventory project in which ledger entries from an account statement were supposed to be matched with the corresponding invoice items. Sounds like an afternoon of work: read in PDFs, pull out amounts, dates and receipt numbers, match them against the ledger entries, done. The catch wasn’t in the matching. It was in the fact that some of the invoices existed only as scanned images and I had to extract the fields via OCR. OCR is not a function that returns text. OCR is a measuring instrument: it gets it right with a certain probability and is an expensive source of errors in the remaining cases.
This view shaped the architecture of the little tool more than any matching rule. As soon as you understand OCR as an unreliable sensor, three duties follow almost by themselves: you store the measurement, because it is expensive and can change with preprocessing, configuration or engine version. You keep the confidence score, because otherwise a misreading looks exactly like a correct hit. And you check the result against domain rules, because an amount that looks plausible is far from being the right amount.
The error that convinced me#
All this caution sounds exaggerated at first. A single record convinced me. A scanned invoice showed a gross amount of 138.00 euros. The OCR read 133.00 euros, because the 8 was set narrow in the font used and the scan had swallowed the upper loop. An 8 that turns into a 3 is not far off visually; for a model that reconstructs glyphs from pixels, it is an obvious slip.
The insidious part wasn’t the misreading. Misreadings happen. The insidious part was what happened next. The account statement actually contained an entry of 133.00 euros from a different transaction. The fuzzy matching, which weights receipt number, date and amount, found a seemingly clean partner for the misread invoice and assigned it to the wrong entry. The actual 138-euro item was left behind as an orphaned account entry. Two errors that made each other look plausible. No stack trace, no exception, no red log. Just a report that looked tidy and was wrong.
That is exactly where the danger lies: an OCR error without confidence and without validation does more damage than a crash. A crash forces you to look. A silent misreading travels through the entire pipeline and disguises itself as a correct result. With monetary amounts, this is not an academic concern.
Why OCR is a sensor and not a function#
It’s worth briefly separating what you are dealing with. Plain text extraction from a digitally generated PDF reads an existing text layer. OCR, by contrast, works on pixels, and the result depends on resolution, contrast, font, page orientation, preprocessing, configuration and engine version. A local Tesseract run with identical bytes and an identical environment is usually reproducible; if one of these factors changes, the result can change too.
That’s why I treat both paths separately. First I try to read the embedded text layer. Only if it is missing or empty does the pipeline fall back to OCR. This fallback is the point where uncertainty enters the system, and exactly this point has to be marked, measured and contained.
It helps that serious OCR engines report their own uncertainty. With Tesseract, the word confidence comes via the TSV output format, together with the bounding box and position on the page. Cloud services return comparable values, often as a float between 0 and 1 per field. This value is the most important piece of information in the whole recognition, and yet it usually ends up nowhere, because image_to_string returns only a string. If you use image_to_data with tsv instead, you get the confidence per word and can build a decision on it.
In my receipt dataset, word values below about 95 were often critical and values above 99 were much more reliable. These are not universally valid limits. The threshold has to be calibrated on labeled examples for engine, document type and field, and it also depends on the damage an error causes. For a free-text field, the bar can be lower than for an amount that gets posted to the ledger.
Cache by content hash, not by file name#
Before I talk about thresholds, first the unspectacular measure that brings the most calm into the pipeline: a cache that computes a document’s OCR exactly once.
OCR is slow and, with cloud services, expensive per page. An inventory run that you repeat twenty times during development would recognize the same pages twenty times. That costs time and money. A cache also pins the result observed once, so that a change of engine, configuration or external service doesn’t silently alter historical evaluations.
The crucial point is the cache key. The file name won’t do. Files get renamed, moved, re-exported, and two scans with identical content can have different names. Conversely, the same name can hide changed content. The only stable key is the content itself, that is, a cryptographic hash of the bytes, usually SHA-256. Same content, same hash, same cache entry, regardless of what the file happens to be called.
But the key needs more than just the file hash. If I change the OCR configuration, such as the page segmentation mode or the engine version, old results are no longer comparable. That’s why a version identifier of the pipeline also goes into the key. This way, a changed configuration invalidates the cache automatically, without me having to clean up by hand.
import hashlib
import json
from dataclasses import dataclass, asdict
from pathlib import Path
import pytesseract
from PIL import Image
# Changes to this configuration must invalidate the cache,
# otherwise results from different setups get compared.
OCR_VERSION = "tesseract-5.3.4+psm6+v3"
CONF_MIN = 95.0 # word values below this count as uncertain
CACHE_DIR = Path(".ocr-cache")
@dataclass
class Word:
text: str
conf: float # 0..100, straight from the TSV output format
def file_key(path: Path) -> str:
"""Content hash plus pipeline version. The file name deliberately plays no role."""
digest = hashlib.sha256(path.read_bytes()).hexdigest()
return hashlib.sha256(f"{digest}:{OCR_VERSION}".encode()).hexdigest()
def run_ocr(path: Path) -> list[Word]:
"""Expensive; cache the observed result together with its pipeline version."""
data = pytesseract.image_to_data(
Image.open(path),
config="--psm 6",
output_type=pytesseract.Output.DICT,
)
words: list[Word] = []
for text, conf in zip(data["text"], data["conf"]):
text = text.strip()
if not text:
continue
words.append(Word(text=text, conf=float(conf)))
return words
def ocr_cached(path: Path) -> list[Word]:
"""Idempotent access: same content always yields the same cache entry."""
CACHE_DIR.mkdir(exist_ok=True)
cache_file = CACHE_DIR / f"{file_key(path)}.json"
if cache_file.exists():
raw = json.loads(cache_file.read_text())
return [Word(**w) for w in raw]
words = run_ocr(path)
cache_file.write_text(json.dumps([asdict(w) for w in words], ensure_ascii=False))
return words
The cache deliberately sits on disk here as one JSON file per document. For a local tool that is perfectly sufficient and has a pleasant side effect: the cache files are readable and can be diffed locally. With invoices and other documents containing personal data, though, such OCR output doesn’t belong unencrypted in the Git repository. Traceability needs protected storage with appropriate retention and access control.
The confidence gate#
With the cache and word confidence in hand, the silent error from above becomes a loud one. The trick is not to ignore the confidence but to make it the decision: fields above the threshold count as trustworthy and may flow on automatically, fields below it are flagged and end up in manual review.
import re
AMOUNT_RE = re.compile(r"^\d{1,3}(?:[.\s]\d{3})*,\d{2}$") # e.g. 1.234,56
@dataclass
class Field:
raw: str
conf: float
needs_review: bool
reason: str = ""
def extract_amount(words: list[Word]) -> Field:
"""Finds the gross amount and decides how to proceed based on the confidence."""
candidates = [w for w in words if AMOUNT_RE.match(w.text)]
if not candidates:
return Field(raw="", conf=0.0, needs_review=True, reason="no amount detected")
# The last plausible amount on the receipt is usually the total.
best = candidates[-1]
if best.conf < CONF_MIN:
return Field(
raw=best.text,
conf=best.conf,
needs_review=True,
reason=f"Confidence {best.conf:.0f} below threshold {CONF_MIN:.0f}",
)
return Field(raw=best.text, conf=best.conf, needs_review=False)
If the misread 133.00 had had a low confidence, it would have been caught here. But that doesn’t finish the gate. OCR engines are sometimes confidently wrong. A cleanly printed 3 that should actually be an 8 can come back with confidence 98. The model is sure the glyph is a 3, because it looks like a 3. The information that it should really be an 8 isn’t in the pixels but in the domain context. Confidence alone is therefore not enough. It is necessary, but it only catches the cases in which the model itself hesitates.
Validation against the world, not against itself#
The second line of defense doesn’t check how sure the recognition was, but whether the result fits reality. Such rules cost little and catch exactly the confident errors where the confidence gate fails.
With receipt data, there are a number of obvious checks. A gross amount must equal the sum of net amount and tax; if it differs, at least one of the three values is wrong. An invoice date must come before the posting date and must not lie in the future. A receipt number often has a fixed format or even a check digit; if it doesn’t match, the recognition was wrong. And an amount that is supposed to be matched against a ledger entry must lie within a narrow tolerance, not within five euros.
In the concrete error case, a single one of these rules would have been enough. If the receipt shows net and tax separately, then 138.00 as gross at 19 percent gives a net value of 115.97, while 133.00 leads to 111.76. If the OCR reads net and gross independently and the sum doesn’t add up, the wrong 3 stands out immediately, no matter how sure the engine was. The redundancy on the receipt is the control measure that the pixels can’t provide.
from decimal import Decimal
def parse_eur(s: str) -> Decimal:
return Decimal(s.replace(".", "").replace(",", "."))
def cross_check_total(netto: Field, steuer: Field, brutto: Field) -> Field:
"""A high confidence is unconvincing when the sum does not add up."""
if any(f.needs_review or not f.raw for f in (netto, steuer, brutto)):
return brutto # missing parts are already handled by the review anyway
expected = parse_eur(netto.raw) + parse_eur(steuer.raw)
actual = parse_eur(brutto.raw)
if abs(expected - actual) > Decimal("0.02"): # tolerance only for rounding
return Field(
raw=brutto.raw,
conf=brutto.conf,
needs_review=True,
reason=f"Sum {expected} differs from read gross {actual}",
)
return brutto
What matters is the order in your head: confidence and validation are not either-or. A field with high confidence that violates a domain rule still belongs in review. And a field with low confidence that satisfies all rules is still suspicious enough to show it to a human. Both gates have to be open for a value to pass through without supervision.
The decision flow at a glance#
flowchart TD
A[Document] --> B{Text layer present?}
B -->|yes| C[Read text directly<br/>deterministic]
B -->|no| D{In the hash cache?}
D -->|yes| E[Load cache entry]
D -->|no| F[Run OCR<br/>expensive, uncertain]
F --> G[Cache result under content hash]
G --> E
C --> H[Extract fields]
E --> H
H --> I{Confidence above threshold?}
I -->|no| R[Flag: manual review]
I -->|yes| J{Domain rules met?}
J -->|no| R
J -->|yes| K[Automatically into matching]
R --> L[Store correction as data]
L --> KTwo gates stand between a recognized value and its automatic use. If you take them seriously, you accept that some of the receipts pass through a human’s hands. That is not a defeat. Mature systems process most documents without intervention and, depending on the receipts, route a smaller share into review. That this share exists is the admission that OCR remains a sensor and not an oracle.
Corrections are data, not one-off actions#
What happens to the fields that end up in review? A human looks at the receipt and the flagged value and corrects the 133 back to 138. If this correction only flows into the current run and then disappears, you haven’t solved the problem, only steered around it once. On the next run with the same file, the OCR reads 133 again, and someone corrects it again.
That’s why I treat corrections as first-class data. A correction is bound to the content hash of the document and overrides the OCR result for exactly this field. It sits, protected, next to the cache, versioned and traceable. On the next run, the correction applies before the uncertain measurement even comes into play. This turns manual review from a recurring burden into a one-time investment per document. Along the way, you get a sensitive dataset from which you can later read where the recognition is systematically weak – with the same protection and deletion obligations as for the source documents.
This separation has another advantage for traceability. In the report, you can say for every value where it came from: from the deterministic text layer, from OCR above the threshold, or from a human correction. Three origins, three levels of trust, and all three are visible. A value that comes from a correction carries its history with it instead of disguising itself as an anonymous string.
What I would build this way right away next time#
Looking back, only a few decisions made the difference, and none of them was laborious. The content hash as the cache key took half an hour and made every further development run cheap and reproducible. The confidence gate was a handful of lines once I switched from image_to_string to image_to_data and stopped throwing the confidence away. In the end, the domain rules were the most valuable layer, because they catch exactly the class of errors in which the model is sure and wrong at the same time.
The common denominator is an attitude, not a framework. Where a model returns a confidence or uncertainty value, the system should keep it without hastily reading it as a probability. Where such a value is missing or poorly calibrated, domain rules, spot checks and human review have to make the uncertainty visible. With monetary amounts, I prefer a flagged line in the report to a clean number I can’t trust.
Further reading#
- Tesseract, TSV output format with word confidence: Tesseract OCR and explanation of the TSV format
- Confidence thresholds in practice: discussion on how Tesseract computes confidence
- Validation and human review in receipt OCR: Invoice OCR Error Handling
Revised and translated from German with the help of AI.