Skip to content

Deterministic CLIs

Determinism is an architecture decision: same inputs, same report, reviewable via diff. It pays off wherever an artifact is checked repeatedly.

≈ 9 min readSoftwareDataTools

Deterministic CLIs

Two runs, the same input files, two different reports. The difference consists of a timestamp in the header and a table whose rows are in a different order this time. Nothing has changed in substance, but the diff shows a hundred lines, and none of them means anything. So the reviewer reads the whole report again. The next time, they don’t read it at all.

A deterministic CLI turns this situation around: whatever shows up in the diff has also changed in substance. Byte for byte. That sounds like an implementation detail, but in reality it is a decision about system boundaries.

Determinism is usually treated as a Side Effect that you pick up “on top.” In truth, it is the prerequisite for a result to become reviewable. And reviewability is the currency in which digital product development actually pays: what I can’t compare, I can’t approve, can’t refute, and can’t automate with a clear conscience.

Why determinism is an architecture lever#

The reflex is to see determinism as a property of the output. I see it as a property of the system boundary. A CLI that works reproducibly forces me to make every source of nondeterminism explicit: timestamps, random numbers, unsorted maps, network responses, OCR results, model outputs. I have to freeze, sort, cache, or keep each of these sources out of the result.

Exactly this exercise is valuable, regardless of the tool. If you name the nondeterminism, you have already half documented the state and dependency boundaries of a system. The question “why isn’t this run reproducible?” is almost always the same as “which part of my system is hidden, untamed I-don’t-know?”

The business outcome is concrete. A deterministic report can be reviewed via diff. A reviewer sees in seconds what changed between two runs instead of reading an entire output again. That shortens the feedback cycle from “I’ll read the whole thing again” to “I’ll look at the three changed lines.” In teams that produce reports like these every day, that is the difference between friction and flow.

The cost model – determinism isn’t free#

Determinism has a price. If you keep quiet about it, you are delivering an advertising message instead of a trade-off.

You pay in explicit state control. Time has to be injected instead of read. Randomness needs a seed. External sensors – OCR, HTTP, a model – need a cache so that a second run sees the same thing as the first. Sorting becomes mandatory, not optional, because otherwise the order of a hash map changes the report. That is extra work, and for small throwaway scripts, it often doesn’t pay off.

What you get in return is reviewability, auditability, and testability in one. A deterministic report is automatically a golden master test: I check in an expected report and let CI diff against it. I don’t need a separate desert of assertions; the artifact itself is the test. The costs are incurred once; the effect repeats with every run. That is exactly why I use determinism where something runs often and is checked often – and leave it out where a script tips something over once and then disappears.

This trade-off is the real engineering achievement, not the implementation. I have seen teams that wanted to force every script to be deterministic and stumbled over seed injection in throwaway tools. And I have seen the opposite: a daily report that nobody read anymore because it looked different on every run and a review would have cost hours. The line doesn’t run between “always” and “never.” It runs along the question of whether the artifact is checked repeatedly. If it is, the price pays off by orders of magnitude. If it isn’t, determinism is gold-plating.

The boundary before the tool#

Before choosing a tool, I clarify which data is authoritative, what has to be traceable later, and which parts should be changeable independently. That way, I can swap the PDF parser without touching the matching rules or the report format. Determinism helps with the last point but doesn’t replace a clear domain boundary: a reproducible report can also reliably repeat ambiguity.

A concrete example – matching receipts against bookings#

I show this with an inventory tool that I use again and again as a pattern. The task: link an account ledger to PDF receipts via invoice numbers, match inventory items against invoice line items, and make the gaps visible. It sounds like bookkeeping, but in truth it is an architecture exercise about uncertain data.

Three design decisions carry the whole thing, and none of them is a tool.

The first is the three-stage structure: link the account ledger to receipts, match line items, and handle non-PDF vendors separately via rules on the booking text. These stages are more important than any file format. They are the domain boundary at which I can later change something independently.

The second is the clean separation of hard match and fuzzy fallback. First, defined tokens must appear in the product title; fuzzy matching only comes into play when the hard path finds nothing. That keeps the automation explainable. For every row, I can say why it was assigned – hard via a rule or soft via a similarity above a threshold. A fuzzy match that nobody can justify anymore isn’t a feature – it is the debugging session of the day after tomorrow.

The third is treating OCR as an uncertain sensor. Vision provides the base, Tesseract corrects amounts specifically in the debit/credit region, and a JSON cache makes later runs reproducible. The decisive sentence: the recognized text is a measurement, not the truth. Amounts, names, and numbers can be misread. That is why this layer gets a cache, thresholds, manual alias rules, and – this is the core – diffability.

The report itself is then an architecture artifact, not a by-product. Inventory items, invoice numbers, purchase date, match type, orphaned account entries, and unused invoice line items appear side by side. That is exactly what turns a script into a verifiable data system: you see not only what was matched but also what doesn’t fit – and that is usually the more interesting information.

How it is called#

inventory-correlate \
  --account account.pdf \
  --inventory inventory.pdf \
  --out report.md

# review is just a diff against the last known-good run
diff last-report.md report.md

The example is deliberately small. It isn’t meant to simulate a complete application but to show the boundary. A good example doesn’t have the most features. It has the one decision you can argue about after five minutes. Here, it is in the second call: because the report is deterministic, the review is a diff. No dashboard, no state, no “read the whole thing again.”

flowchart LR
  PDFs --> Parser
  Parser --> Normalize
  Normalize --> Match["Hard match<br/>then fuzzy fallback"]
  Match --> Report["Deterministic<br/>Markdown report"]
  Report --> Diff["Review via diff"]

Markdown as an audit artifact#

Why Markdown and not a UI right away? Because a Markdown report is reviewable, diffable, and archivable without me having to build a dashboard. For data tools, diffability is a strong quality attribute. If a report exists as Markdown, a change can be seen in the review – line by line, with full history in Git. That is often more valuable than a UI that only shows the latest state and can’t answer the question “what changed?” at all.

The same principle carries far beyond bookkeeping. Publishing tools can work according to the same pattern: input in, deterministic artifact out, changes visible as a diff. Whether I am matching receipts or generating presentations – the architecture is identical, because the quality problem is identical: I want to see changes, not guess states.

How I check the benefit#

A diff limits the review to the lines that actually changed. Whether that saves time can be read from the review duration of a recurring report. An expected report can also serve as a golden master; an unexpected diff is then a concrete signal to check. For that, I invest in caches, seeds, and explicit sorting – not because determinism would be an end in itself for every script.

What I check before calling it production-ready#

An idea only holds up once its failure path is described as well. What happens on repetition? What is idempotent? Where is the transaction boundary? Which logs or reports help with analysis later? These questions seem dry, but they are often the difference between a demo and operations.

With a deterministic CLI, there is one specific question on top: is the nondeterminism really captured, or is it just quiet by chance right now? A report that is stable today because the map happens to come out sorted tips over tomorrow on a different runtime. That is why I enforce sorting explicitly, version caches, and treat manual corrections as data, not as manual steps. Fuzzy matching needs explicit thresholds and a way to prefer hard rules – otherwise the result can no longer be explained. A tool may be uncertain, but it has to show its uncertainty.

Before using it in production, I check three things:

  • The same inputs produce a byte-for-byte identical result.
  • Time, randomness, order, and external responses are explicitly constrained.
  • Errors and uncertain assignments show up in the verifiable artifact.

Typical mistakes#

The first mistake is confusing the tool with the solution. Docker isn’t an architecture. GraphQL isn’t a domain model. React isn’t a state concept. An LLM isn’t a process. These tools can express good boundaries, but they don’t create them automatically. Determinism is the same: it is a decision about system boundaries, not a flag you set.

The second mistake is overrating the happy path. Many systems look clean in demo mode. It gets interesting with repetition, partial failures, old data, network problems, wrong permissions, or human correction. That is exactly where it shows whether the model holds – and whether the report will produce the same diff tomorrow as it does today.

Closing#

I invest in determinism when a result is checked repeatedly. Then the diff shows the change, and caches, seeds, or explicit sorting have a concrete purpose. For a one-off throwaway script, the same effort would be gold-plating.

Further reading#

Revised and translated from German with the help of AI.