The Socratic Engineer
What’s your edge when the machines know everything?
I’ve always felt extremely self-conscious about not knowing everything. My inner perfectionist would fret that I was only one failed question away from having my cover blown, and being exposed as an imposter for all to see. I over-prepared but under-performed in interviews, and shied away from risks because I didn’t believe I was quite good enough.
Only recently have I noticed my self-critical mindset changing, and it’s coincided with having AI omniscience at my fingertips. It’s enabled me to make peace with the fact my value is not dependent on knowing everything, or proportional to my expertise. There’s no getting away from the fact that the machines have read everything ever published by every expert, and can deliver answers at peerless speeds.
Experts get very defensive when told their precious expertise, which they’ve spent a lifetime painstakingly accumulating, is now commoditised. Ah! they retort, what about the deep intuitive truths we’ve learned? Are those the ones you invented from first principles, I might counter, or the ones you read somewhere? Either way, the machines have digested all that knowledge, and vastly more besides.
Every AI model has an encyclopedic recall of every subject you’ve studied, in both theory and in practice. They’ve digested every textbook, whitepaper, and public source code repo in existence. You didn’t invent your field of expertise, you slowly learned it, and you’ll never out-memorise a trillion parameter matrix that contains the sum of human knowledge.
So given all that, what is your edge now?
Technological capabilities are moving at a dizzy speed. Twelve months ago AI models could write code with hand-holding, but the results were brittle. Now, the latest models can implement complex working features autonomously at the first attempt.
So rather than compete with our new artificial assistants, the real question now is how we can elevate our combined performance. It’s irrelevant whether you know answers without looking them up, what matters is whether you can discern effective results from plausible sounding progress that turns out to be a tarpit.
AI models are graphs of possibility space, modern day Libraries of Babel that contain everything already written, and every subtly wrong variation of it. Consequently, the treasure of truth lies alongside the fools gold of the almost right. It turns out reciting rapidly is considerably easier than proving it’s actually correct.
Since inventions require experimentation and evaluation, in a strict Popperian sense AI is incapable of inventing new things. AI can predict and find novel correlations within existing data at a staggering scale, but it lacks the worldview to formulate a risky conjecture and then actively try to falsify it. Science advances slowly through refutations and repeatable validation, that’s why human expertise still matters. Someone needs to mark the homework. Talent scouts exist because the ability to spot excellence is a talent in itself.
Instant automated answers are now plentiful and cheap to produce. Remember “Prompt Engineering”? The arcane art of precisely formulating just the right questions to get what you need isn’t necessary any more. The real scarcity, and where the real value lies, is critical thinking: being able to assess whether you’re on-course for what you want to achieve, or discreetly veering away from it.
Once you give an agent a direction, it’ll keep going, with a cheery momentum that hides the intrinsic drift from your original intention. Leave it running, and it will flawlessly and rapidly build you a system that’s wrong in all sorts of subtle ways. It lacks the meta-awareness to step back and say, “Why are we designing it this way in the first place?” It just executes. It’s trained to spit out the most plausible stream of tokens, and it’s vital to be aware of that limitation.
Critical thinking is your edge. It encompasses not only the ability to set a threshold of acceptability and distinguish good enough from great, but also the capacity for taking the initiative. Despite their name, agents have low agency. No matter how complex their scaffolding, they’re still fundamentally reactive. Agents can assess the current state of a problem, perform iterative actions, and route-find their way towards a target, but they do not possess teleology, the internal generation of purpose.
Agents do not wake up with a desire to solve a specific problem, disrupt a market, or express an original idea. The initial instruction, the constraint boundaries, and the motivation of why we should bother to do this in the first place must originate from the real world. The machine has a compass and a jet engine, but relies on its pilot to set its destination.
So your role is to be the executive who provides the initiative and the acceptance standards. This is not because it’s the only job remaining, like some kind of consolation prize, but because it’s actually the highest-leverage position in the software development process, and your messy emotional brain is perfect for it.
Why Agents Need You
I’ve always thought a sure sign that you’re reading an AI-generated response is it tries to botsplain the topic to you. That’s because they’re trained by reinforcement learning to be helpful, rather than trying to mimic the kind of responses a human being would return.
AI models are constructed to simulate the appearance of thinking, but as we’ve still no idea what goes on inside our heads, we haven’t even attempted to replicate it. Instead we employ some clever hacks to simulate creativity. One is to embrace randomness, a parameter known as temperature, which sets how much the model is allowed to risk responses chosen from the fringes of the token probability distribution (think of it as a bell curve of plausible answers).
This is how hallucinations arise, and the more reasoning steps a task involves, the greater the chance the model will err, and the answer will be imperfect, or just plain wrong. This is known as autoregressive drift, and the error is cumulative. Effectively there’s a stumble threshold, and the more steps you take, the greater the chance the LLM will trip up.
This is exacerbated by the fact AI reasoning is typically simulated through chains of thought, and once an error is made, it propagates. That’s why every single chatbot has a warning at the bottom of the screen that says: “This is AI and can make mistakes. Please double-check responses.”
The most common cause of AI mistakes is very rarely missing data, but the probabilistic way a model composes its answer by choosing which path to follow.
When the field of Artificial Intelligence was christened in 1956, thinking was exclusively biological. Seventy years later, non-biological thinking is no longer a novelty, and there’s a case for regarding AI as Augmented Intelligence, a symbiosis between two thinking systems. A hybrid combining a gigahertz-speed digital pattern recogniser that’s read everything, and an organic brain with unquenchable curiosity and scepticism that have been honed by surviving in the face of adversity over thousands of generations.
The Human + Machine partnership covers each other’s blind-spots. Human beings excel at spotting things that look out of place. The ability to develop and exploit contrarian opinions, demurring from the accepted wisdom of the majority, is an elite thinking strategy. Your ability to disagree is actually your superpower.
The Socratic Method is a form of argumentative discourse during which the questioner aims to ask the most challenging questions possible. Like, how do you actually know that? Assumptions are revealed, as we attempt to dig down to first principles. The aim is not to keep your companion talking, but to uncover the reasons behind their opinions.
If you disagree with a strong model and it’s actually right, it’ll present more evidence to justify its position. If you disagree and it’s wrong, it’ll merrily acknowledge its error and change to the correct course. In other words, pushing back against an AI model has no downside. It’s not going to adopt a wheezy robotic voice and say “I find your lack of faith disturbing”.
Unlike human beings, who often get frustrated if you persistently call them into question, LLMs don’t see a challenge as a criticism of their ability, to them it’s just another possibility space to explore. Modern models are more resistant to gaslighting, able to hold the line on objective logic and facts, yet be flexible when the topic is more subjective.
In other words, your role in the AI age is to become a Socratic Engineer. This job is so much more than being a ringmaster, cracking the whip to get the agents to perform. You are the agent’s missing sense of scepticism.
We use different terms to distinguish between intelligence and wisdom. One is knowing things, the other is knowing what actually matters. The art of getting the best possible answers is asking the wisest questions, and being able to distinguish what’s actually true from plausible-sounding fantasies.
Knowing what to ask requires a mental map built from years of watching things fail. It’s the ability to look at a project and ask the awkward questions that an agent can’t introspectively ask itself — like what exactly are we trying to optimise, and what are we prepared to compromise on? Or, are we solving a real problem here, or just adding another thousand lines of complexity for no real benefit?
You are shifting from being the person who writes the script to the one who defines the lines of inquiry. Once identified, wrong turns can be refactored in seconds; but if we continue down the wrong path we can waste weeks of compute, capital, and opportunity. The human in the loop ensures there’s always someone with skin in the game to keep asking: Wait, what problem are we really trying to solve here?
Engineers who write software aren’t relics of a bygone era, they are the enforcers of quality standards, and they are vital to prevent generative systems wandering confidently off the proverbial cliff. Human beings are just as fallible, of course, but we’ve personal experience of how the world works, with all its frustrations. AI models have only read about it.
So you are no longer valued for your ability to grind out a solution. You’re valued for your ability to collaborate on delivering an even better solution than either you or your agent partner could produce alone.
Agent Debt
This anecdote highlights one of the biggest risks in agentic software development: that you outsource hard things, and stop thinking for yourself. Instead, you’re content to act as a relay, pasting what you receive into a chat session, and then waving through the answers it returns.
People wave things through not because they’re inherently indolent, but because we intuitively prioritise the impression of progress. Crafting something beautiful is impressive, but we feel an even greater pressure to move as fast as possible. Our company may be trying to enter a market and secure an early advantage. If you have a limited runway, it’s suicidal to wait until you’ve perfected your product. What matters is shortening the time taken to launch, get feedback, and then iterate.
So the antithesis of the Socratic Engineer is the TokenMaxxer — one who has a compute budget to spend and sees unspent tokens akin to idleness. Diffs are prioritised over thoughtfulness. Issues are closed with lightning speed, new features arrive in a seeminglessly endless cascade of shipping. Project velocity soars.
But is this really progress, or merely productivity theatre? How would you know if you were running fast in the wrong direction? Are you really solving the right problem if you’re anxious your window of opportunity will only be open for a few weeks?
There’s already a term for the consequences of moving at the Speed of Compute: Agent Debt. Agents love writing code, but they’re much less capable when it comes to removing it. All those “nice to haves” become permanent parts of your codebase, an overdraft of unverified features accumulates, ostensibly working but fundamentally unproven.
Back in the day, when large teams of human developers raced to turn capital into code it resulted in something we called technical debt. The problems being stored up through agentic coding are no different merely because the process is automated, we just accumulate the debt faster, with far less awareness of the liabilities we’ve incurred.
TokenMaxxing is the AI-era equivalent of “move fast and break things” — a vanity metric that prioritises the appearance of novelty over sustainable value. Personally I’ve always loathed this mindset, since it never stops to answer: why are we actually doing this in the first place? Craftsmanship doesn’t mean spending an excruciatingly long time on something, but it does mean making changes intentionally.
Personally I’m not a TokenMaxxer because I’d never dream of treating pair-programming with a human team like a speed-dating session. A few minutes to tell them what to do, the bell rings, and move onto the next desk. That would be an awful way to work.
It’s not bad practice to give one challenge your full and undivided attention, and to act as the quality control so all new features are added purposefully. If I’m not steering what’s built, why would I even be involved at all? I made technology my vocation to implement ideas, not to spend my working life winding up clocks.
Ah, you might say, pitying my archaic working practices, don’t you know we can use LLMs now to judge the quality of the work agents produce? But if you use an agent to evaluate the creations of another, you’re just introducing a new layer of non-determinism: hallucinations checking hallucinations.
You need to be exceptionally good at writing evals to get “LLM as Judge” to work, because great work meets qualitative levels, not quantitative ones. How are you going to check whether a piece of software feels elegant to use, whether a narrative voice hits the right emotional resonance, or whether a technical architecture is unnecessarily brittle?
I know the creator of Claude Code has gone on the record saying he runs a lot of agents concurrently, but few have accumulated his intuitions on the strengths and limitations of agentic coding. If you want to operate using multi-agent development you need to scale up your working practices gradually, identifying where your personal intervention has the greatest leverage.
Simply spinning up more and more agents is a recipe for chaos. Human attention doesn’t divide well, so the only way to keep the engine of perceived productivity whirring is by not supervising them at all, and trusting them not to break things.
So we should own our work, but be conscious of what we’re prioritising and why. What’s important at any given moment shifts depending on whether we’re launching something new, or consolidating what’s already been proven.
You are Ground Truth
If the role of the modern technologist is to be the ringmaster who provides the initiative and enforces the evaluation metrics, that’s not a minor consolation prize. It’s the highest-leverage position in the entire pipeline, as you get to set ground truth.
It’s also a psychologically healthier way to work. You don’t have to be infallible, you just have to be a filter. You’re no longer valued for your ability to grind out a solution, but your ability to deliver an even better solution than either you or your agent partner could produce alone.
The technical expert is no longer a bricklayer — in fact, they’re not even a structural engineer. They’re more akin to a town planner, authorising what needs to be built. They are responsible for setting the real world constraints the design must abide by: like the build’s practical, financial, and aesthetic requirements.
So if you’re an expert, the good news is you aren’t obsolete; the industry has just finally automated the most tedious parts of the job. You’ve been promoted, with far more important responsibilities than deciding where the next brick goes.
Jevons’ Paradox says technology creates more work than it replaces. This is because it extends the horizon of what’s possible. The notion that we’ll ever run out of problems to solve is absurd. As long as we remain ambitious as a species, technology will act as a force multiplier for human ingenuity.
AI itself is still one such unsolved problem. But something seems to have shifted: nothing seems intractable any more. The more compute we can devote to a challenge the deeper we can explore it, and the more experts will be needed to explore and validate true discoveries.
Even if your motivation is more prosaic than pushing back the frontiers of human knowledge, expertise is still invaluable. Once services are commoditised the race is on to deliver them even better — that’s hard if everyone has the same capabilities. Only deep intuition and hard won expertise set you apart from the crowd.
A frontier AI model is a savant, not a genius. Like any brilliant but erratic employee, it needs to be managed closely. The value of knowing arcane technical knowledge is trending to zero. What matters now is a proximity to customer problems, and the subtle expertise accumulated over a lifetime.
The guiding light of anyone developing products in the AI age must be to authentically care about the experience of their users. AI models care about the intrinsic rewards their training has baked into them, only engineers emotively care about their users’ joy and success.
That’s your moat.
— Jaron, June 2026
