Skip to content

Good Enough: The Model Is Rarely the Bottleneck

For the everyday tasks I work on with AI myself, the choice of model is rarely the first bottleneck. How the task is cut, the context and the verification usually matter more.

≈ 8 min readSoftwareAIMethodology

Part 2 of the series “From Session to System” – start with the opening post.

End of November, an architecture review with a team I support, day two, first coffee break. A developer walks over to me, cup in hand, and asks the question that has come up in every team conversation and every hallway chat at conferences for two years: “Which model do you use?” This time with particular urgency, because November 2025 was wild: GPT-5.1 on the 12th, Grok 4.1 on the 17th, Gemini 3 on the 18th, Claude Opus 4.5 on the 24th. Four flagship announcements in twelve days, plus the usual benchmark screenshots making the rounds on the timelines. His team, he tells me, has been discussing for weeks whether they need to switch.

I used to answer this question seriously, with model names and reasons. Every one of those answers was outdated after six weeks. These days I answer differently: “That is the second question. Let’s talk about the first one.”

The convergence nobody likes to talk about#

Looked at soberly and without vendor loyalty: for the large block of everyday tasks I use AI for, the big models were close enough to each other in my own trials at the end of 2025. Reading and explaining code, proposing a refactoring, designing tests, reviewing a concept paper, summarizing API documentation, sketching a migration path – I got that done in usable quality with GPT-5, Claude Sonnet 4.5 and Gemini. Not equally well in every single run, but well enough that the choice of model was rarely my first construction site.

Of course they are not identical. Every model has its own tone, its own preferences, its own weaknesses, and for individual tasks the differences can be large. Public benchmarks only help me to a limited extent, because they reflect neither my context nor my review criteria and shift quickly with new versions. For an architecture decision, I therefore need measurements on my own tasks rather than a general ranking.

I do not like to be the first to say this in such meetings, because it cools down the discussion everyone was looking forward to. The model comparison is entertaining, it feels like engineering work, and it has the charm that you do not have to change anything about your own way of working. But it answers the wrong question.

The noise in daily work is larger than the difference between models#

When a session with Claude or ChatGPT goes wrong for me, it is almost never because of the model. It is because I stated the task unclearly. Because the model did not know the context that was decisive for the solution – the architecture decision from two years ago, the unwritten convention in the team, the reason why this one service is built differently from all the others. Or because I did not check the result and the mistake only surfaced in review.

An example from a team conversation at a large enterprise, anonymized as always: two colleagues, the same model, the same task – understand a piece of legacy code and propose a restructuring. One got a usable, concrete plan. The other got generic textbook text. The difference was not the model, not even the prompt in the narrow sense. One had given the model the relevant files, the test cases and the constraints. The other had typed a question into an empty window.

In my own sessions, the result often varies more with how the task is cut, the context and the verification than with switching between the top models under consideration. That is not a general measurement of the market, but an observation from this work. For me it follows: before I switch models, I first check whether task, context and acceptance criterion are clean enough.

If you bet on the best model, you bet on a commodity#

That brings me to the thesis I consider the most important in this post: if you build your way of working, let alone your product, on “the best model,” you are betting on a commodity you do not own.

The models belong to a handful of vendors. I rent access through an API key or a subscription, just like my competitor, just like every other team. I cannot control when a version is deprecated, when behavior quietly shifts after an update, when the price drops or the next model upends the ranking. November demonstrated this: if you based your choice in October on a benchmark lead, you saw that lead change hands three times within two weeks.

What I can own, on the other hand, is the layer around the model. That is not a platform and not a product, but simply everything I have in my own hands before and after the model computes:

  • how the tasks are cut, meaning the ability to break work down so that a piece of it becomes delegable
  • the context a session gets at the start, instead of the model having to guess my reality
  • the team’s knowledge in retrievable form, from conventions to the reasons behind old decisions
  • the process that governs when the AI works, when a person reviews and what may go into the main branch
  • the verification itself, meaning reliable ways to determine whether a result is correct before it has any effect

This layer is the opposite of a commodity. It is specific to my team, my codebase, my domain. Nobody can take it away from me with a release, and the competitor cannot simply rent it too. Above all: it survives model changes. An hour I invest in better task descriptions, in a well-maintained CLAUDE.md or AGENTS.md, in review criteria still pays off with the model after next. An hour of studying benchmarks is devalued with the next release.

There is a pleasant side effect that I know from architecture work: if you own the layer, your switching costs drop. When task cutting, context and verification are solid, the model behind them is as replaceable as a dependency behind a clear interface. Then I can even switch calmly when there is a real reason – because my actual capital does not hang on the model name.

Where the choice of model does matter#

Honesty is part of this, otherwise the thesis turns into dogma. There are situations in which the choice of model plays a real, measurable role. I run into three of them regularly.

First, the special cases. If you work with exotic languages or niche frameworks, have strongly multimodal tasks or have to stay on-premises for data protection reasons, you quickly end up with smaller open models – and there the quality differences are real and the trade-offs are different. “Good enough” applies to the large frontier models for everyday tasks, not to every class of model for every task.

Second, cost and latency. As soon as models no longer run interactively but in volume – classification in a pipeline, autocompletion in the editor, processing thousands of documents – price per token and response time become hard criteria. Between models there are factors there, not percentage points, and measuring pays off.

Third, long, agentic tasks. This is where the models still really differ noticeably at the end of 2025: how long they stay focused on a multi-step task, how reliably they operate tools, how they deal with their own mistakes. It is no coincidence that Claude Sonnet 4.5 in September and Opus 4.5 in November were promoted on exactly this capability. If you delegate entire work packages instead of individual questions, you may be picky about the choice of model.

But – and this matters more to me than the three exceptions themselves – all of this is the second question. It can only be answered seriously once the first one has been answered. Without a clear cut of the task and without reliable verification, I cannot compare models at all; I am then only comparing gut feelings on different days. Only when the layer is in place does the choice of model become what it should be: an optimization with measurements, roughly as exciting as choosing a database index. And as there, the rule is: reassess occasionally, pin versions deliberately, take behavior changes after updates seriously. “Good enough” means neither “all the same” nor “forever” – it just means: not the bottleneck.

The first question#

Back to the coffee break. Stick with the model you have, I told him, and put your team’s discussion energy into the questions before and after. How do you recognize that a result is correct? What would a session need to know about your system so it does not have to guess? Which of your tasks are even cut in a way that lets you hand them off? If you answer these three questions well, you get more out of any of the current models than your neighbor with the benchmark winner and an empty context window.

He was visibly unhappy – the answer is less spectacular than a model comparison, and it means work. But it has a consequence that has occupied me ever since and that carries the rest of this series: if the model is rarely the bottleneck, then something else makes the difference between the session that delivers and the one that burns time. For me, it comes down every time to the environment the model works in – how much it shows the model, what it allows it to do and what it checks its results against. That is exactly where the next post picks up.

Further reading#

Revised and translated from German with the help of AI.