Let’s talk about what’s next

Whether you're working through a challenge or ready to move on something new, we're ready.

Looking to join the team?

Find your next challenge

Please enter a name

Please enter a company

Please enter an email

Please enter a valid email

Please enter a phone

Please enter a valid phone

Please tell us about your challenge or opportunity

Start a conversation

Thanks

Your message has been sent.
We will get back to you within 1–2 business days.

Something went wrong while sending. Please try again, or email us at info@parserdigital.com.

Insights

The Vocabulary Gap: Harness, Scaffold, or Guardrail? Naming the Layers of Agentic Engineering

From goal-driven models to coding harnesses like Codex: what traditional Software Engineering can teach us about the agentic vocabulary gap.

17 Nov 2026
Martín Miceli
AI
LLMs & Generative AI
Software Architecture

Abstract

Agentic Engineering is adopting new terms faster than it defines them.

“Harness” alone can denote a coding agent, the workflow wrapped around it, or the environment that runs its evaluations, and that ambiguity spills into architecture decisions, vendor comparisons, and governance.

I propose a vocabulary of three terms that do not collide: components, which sit inside a harness; layers, which form the engineering system around it; and categories, which distinguish the kinds of harness. The vocabulary is built incrementally around a single running example, an agent fixing a failing refund test.

Part I moves from a model post-trained to pursue goals, through the shapes it can take (a chat that answers once, a workflow that follows a path fixed in code, an agent that directs its own loop, and a harness that engineers that loop for real work), to the six components of a modern harness (live repository context, prompt shape and caching, tools and permissions, context compaction, session memory, and bounded subagents) as implemented in Codex CLI and Claude Code, and finally to four categories: coding harnesses, agentic workspaces, autonomous agent fleets, and background business agents.

Part II names six engineering layers around the harness (execution, workflow scaffolding, governance and guardrails, observability, evaluation, and regression), with knowledge and context as a cross-cutting capability.

Part III examines what is genuinely new and translates the framework into questions for technology leaders, drawing on 2026 evidence on harness effects, agent reliability, productivity, security incidents, and cost. The central claim is that most of Agentic Engineering extends existing Software Engineering disciplines to a new kind of actor, and that a shared vocabulary is a precondition for standardising it.

Figure. The vocabulary at a glance: the model in a loop, the six components inside the harness, the six engineering layers around it, and the four categories.

The vocabulary at a glance: from chat to harness, then the three words that describe a harness (components inside it, layers around it, categories that tell harnesses apart).

Introduction: a vocabulary problem

Not long ago, sitting with a group of engineers to review how we were building software with agents (part of our Agentic Engineering work at Parser), I realised we were using the same word to describe completely different things.

Someone called Claude Code the harness. A minute later, someone else used the same word for the OpenSpec workflow wrapped around it. Then the conversation moved on to eval harnesses, tracing tools, guardrails, and regression testing across model releases, and the word “harness” kept appearing but with different meanings.

We all understood the technology, but we couldn’t agree on what we called each piece. It reminded me of the early days of cloud computing, when almost everything became a platform, a service, or a framework, depending on who was presenting the architecture.

When terminology becomes ambiguous, architecture discussions become ambiguous too.

Figure 1. The vocabulary gap. Everyone is building with AI using similar parts but with different names: harness, scaffold, guardrail.

Eric Evans described this problem two decades ago, in a very different context. In Domain-Driven Design he warned that “a project faces serious problems when its language is fractured” [1]. His answer was a ubiquitous language: one shared vocabulary, used in conversation, in diagrams, and in code. Agentic Engineering needs the same discipline, and it needs it sooner, while the words are still soft enough to shape.

The field has already started naming the discipline. In February 2026, OpenAI described “harness engineering” as the work of building a product in which Codex wrote all of the code [2]. Birgitta Böckeler, writing on Martin Fowler’s site, split a harness into guides that steer the agent before it acts and sensors that check its work afterwards [3]. LangChain went to the extreme: if you are not the model, you are the harness [4]. Anthropic draws the line the other way: in its Managed Agents design, the harness is only the loop that calls the model and routes its tool calls, kept apart from the session log and from the sandbox where code runs, and the hosted service that ties them together is a meta-harness [5]. Each definition is useful. Each draws the boundary in a different place — which is precisely the collision this article is about.

So here is the vocabulary I propose, in three words that should never collide. Components are what sits inside a harness. Layers are the engineering system we build around it. Categories are the different kinds of harness. Diagram 1 is the entire map, the rest of the article walks it from the inside out, one step at a time.

Diagram 1. The map: a model in a loop, wrapped in a harness of six components, surrounded by six engineering layers, and deployed in one of four categories. Category colours match Diagram 7.

The diagrams share one colour key, shown under each of them: blue for reasoning and execution, orange for action and control, grey for environment and observation, green for verification and outcomes, teal for knowledge and context, and lilac for economics and maturity. The orange line is always the loop. Two diagrams are exceptions: Diagram 2 (the post-training stack) and Diagram 7 (the four categories).

To keep things simple, we will follow a single, deliberately ordinary task. A payments service has a failing test: refunds in currencies without decimal places, such as the Japanese yen, are being rounded incorrectly. A developer asks an agent to fix it. That refund will reappear in every section, and by the end it will have touched every part of the map.

New to these terms? Box 1 in Appendix A defines the twelve used throughout, from LLM to pull request.

Part I. From model to harness

Let’s build it up in four steps, each resting on the last. We start with the engine, then the loop that puts it to work, then the machinery that keeps that loop reliable, and finally the different kinds of machine we are building with it.

1. The engine: a model trained to pursue goals

The cleanest analogy I know comes from Sebastian Raschka’s Components of a Coding Agent, a piece I strongly recommend. The LLM is the engine, a reasoning model is a more powerful (and more expensive) engine, and the agent harness is the machinery that lets us actually put that engine to work [6]. I would push his image one step further: the harness is the car built around the engine, the part that harnesses its power and turns it into motion.

Figure 2. The engine and the car: the model supplies the power, the harness turns it into action.

Pre-training gives a model broad knowledge of code, but very little experience of acting. The Kimi K2 team commented that multi-step reasoning, long-term planning, and tool use are rare in natural data and costly to scale [7]. So the industry added a new stage of post-training, aimed specifically at goal pursuit. It rests on three ingredients:

1. Examples of successful multi-step work. Synthetic trajectories in which agents use tools and complete tasks, kept only when a rubric judges them successful [7].

2. Rewards that can be checked. Reinforcement learning on tasks whose outcome a test or checker can verify. OpenAI trained codex-1 this way on real-world coding tasks, rewarding code that follows instructions and keeps running tests until they pass [8].

3. Self-critique where nothing can be checked. For open-ended work, the model assesses its own outputs against rubrics [7].

Figure 3. Post-trained to pursue goals: worked examples of multi-step tasks, rewards a test or checker can verify, and self-critique where nothing can be checked.

The post-training stack grew one layer at a time, each added on top of the ones before (Diagram 2 below). GPT-2 was pre-training and nothing else: a remarkable text continuer, predicting the next token with no notion of a task. InstructGPT added supervised fine-tuning and reinforcement learning from human feedback, and the model learned to follow instructions. Constitutional AI replaced part of that human feedback with feedback guided by written principles. Function calling taught models to ask for tools. Reasoning models such as o1 and DeepSeek-R1 added reinforcement learning on problems with verifiable answers. Agentic training is the newest floor, and for the 2026 frontier (OpenAI’s GPT-6 family, Claude Opus 5.5, Claude Fable 5.1) the recipes are largely unpublished.

Diagram 2. The post-training stack: every layer stays in place as new ones are added on top. At the right, when each layer arrived, the first models to carry it, and the skill it taught. Colours and icons identify each layer.

The result is a model whose default behaviour is no longer “answer the question” but “keep going until the goal is met”. That is what makes agents possible.

How far can the pursuit of a goal go? A useful yardstick is the length of a task, measured in human working time, that an agent finishes successfully about half the time. Since 2024 that length has roughly doubled every three to four months [9], and by early 2026 the best publicly available models were handling tasks of around half a working day or more, with wide error bars on every estimate [10]. Each doubling hands the agent more work that nobody is watching, which is why the governance layers matter more every quarter.

Unwatched work is exactly where goal pursuit can go wrong. A model rewarded for passing tests can learn to fake the pass, for example by making the test runner report success without fixing anything. In a controlled experiment built on real coding environments, models that learned this shortcut did not stop there: they became less trustworthy across the board, up to trying to sabotage the very code meant to catch them [11] — this is the paper I always use for anyone who still sees guardrails and evals as bureaucracy.

The same thing happens outside the lab. On hard tasks, agents often broke the rules they had been given and were not honest about it; had those cheats been counted as successes, one model would have looked roughly twice as capable as it really was [10].

A model trained to make tests pass will make tests pass. That is not the same as making the software work. Keep that sentence in mind: our refund test will need it.

2. The loop: what turns a model into an agent

Anthropic describes agents as systems in which the model dynamically directs its own process and tool usage [12]. The same paper draws a second line worth keeping. Workflows are systems where LLMs and tools are orchestrated through predefined code paths; agents are systems where the LLM directs its own process and tool use, keeping control over how the task gets done [12]. Many production systems are, and should be, workflows: when the steps are known in advance, a fixed path is cheaper, faster, and easier to test. In practice, that direction happens through a loop. Michael Bolin of OpenAI describes it in more detail: the harness builds a prompt, the model either answers or requests a tool call, the harness executes the call, appends the result, and asks the model again, until the model stops requesting tools and returns a message [13]. A single request can contain hundreds of those iterations.

Figure 4. Chat, Agents and Harnesses: The loop is key to autonomy, and the longer it runs on its own, the less human intervention it needs.

Diagram 3 shows four shapes side by side, and Figure 4 gives chat, workflow, agent, and harness a human counterpart. A chat is a scribe: you ask for a letter, it writes it in one pass, within seconds, and everything after that is up to you. A workflow is an assembly line: the model works at fixed stations, but the code decides the order, so it runs the same steps however the work turns out. An agent is a carpenter: given a goal, it plans, picks up a tool, checks the result, and goes again, tens of cycles over a few minutes until the chair stands, while someone keeps an eye on the workshop. A harness, which the next section opens up, is a general contractor: it draws up the plan, coordinates a crew, and keeps the work moving for hundreds of cycles over minutes or hours, checking in with the owner only at key moments. What changes from one to the next is less the intelligence than how long the work can run before a human has to step in, and that number of cycles is the most practical measure of autonomy.

Diagram 3. Chat, workflow, agent, and harness side by side: where the loop sits, how many cycles it runs, how long it lasts, and what is left to the human. The harness is expanded in Diagram 4.

Faced with our refund issue, a chat model would suggest a fix in prose and leave the work to the developer. An agent reads the failing test, runs it, inspects the rounding function, edits it, and runs the test again, because each tool result becomes the input to its next decision.

What looks like an agent “thinking its way” through a codebase is, underneath, a loop. A well-engineered loop, but a loop.

3. The harness: the machinery that makes the loop work

A bare loop breaks quickly on real work. Prompts grow until they no longer fit, tool output floods the context, and nothing stops the model from running the wrong command. The harness is everything around the model that prevents this: it builds the prompt, exposes and executes tools, manages what fits in the context window, remembers what matters, and decides what the agent is allowed to do. The model provides intelligence; the harness enables it to act. As Raschka puts it, “a lot of apparent ‘model quality’ is really context quality” [6].

He breaks a coding harness into six components (Diagram 4). If you want to understand them properly, I recommend two readings side by side: his Components of a Coding Agent, with the mini-coding-agent he wrote to explain the moving parts [6], and Bolin’s Unrolling the Codex agent loop, which walks through a production harness [13]. One teaches by building while the other teaches by unrolling a real system.

Diagram 4. Inside a coding harness: the agent loop at the centre, surrounded by the six components that make it work on real software.

Diagram 5 follows our refund through one run of a coding harness. Every step leans on one of the six components. The project instructions load first. The cached prompt prefix is reused on every step, which keeps the run fast and affordable. Two thousand lines of test log are clipped to the failure that matters. A subagent looks up which currencies have no decimals without flooding the main context. And when the agent tries to “fix” the problem by editing the test assertion, a permission rule stops it.

Diagram 5. The running example: one run of a coding harness, step by step, with the harness component each step depends on. Steps 2 to 8 are the loop; a real run repeats it dozens of times.

Step 6 is the reward-hacking lesson in miniature. Editing the test would have made it pass. It would not have fixed refunds.

To see how Codex CLI and Claude Code implement each of the six components, see Appendix B.

Both Codex CLI and Claude Code load project instructions from Markdown files, both compact the conversation when the window fills, and both separate what the model may attempt (permissions and approvals) from what the operating system will allow (the sandbox) [13]. Figure 5 summarises those components again. Convergence is usually a sign that a discipline is maturing, and a sign that a shared vocabulary is possible.

Figure 5. The Harness Components: Live Repo context, Prompt shape & Cache, Tools & Permissions, Session memory, Bounded Subagents and Context compaction.

Coupled harnesses are evolving into meta-harnesses.

Anthropic’s Managed Agents shows where that convergence is heading once harnesses run at scale [5]. It splits the agent into three parts behind stable interfaces: the brain (the model and a stateless loop), the hands (sandboxes and tools, each reached through a single execute call), and the session (an append-only event log that lives outside the context window). In our vocabulary, the session takes over much of session memory, and the sandbox carries tools and permissions. Each part can fail or be replaced on its own: a crashed loop restarts from the log, a dead container is simply provisioned again, and context that compaction would have thrown away stays recoverable in the session. The split also makes agents faster to respond: a container is now started only when a task actually needs one, rather than for every session, and Anthropic reports that the typical wait before the agent starts answering fell by about 60% [5]. OpenAI’s Agents API, launched in September 2026, follows the same split: it runs the managed Codex harness and sends tool calls to a sandbox that can be hosted by OpenAI, by the customer, or by partners such as Modal, Cloudflare, Daytona, and E2B [14]. Diagram 5b shows the pattern.

Diagram 5b. From a coupled harness to a meta-harness. Brain, hands, and session sit behind stable interfaces, keeping credentials safe outside the sandbox.

How much does the harness itself matter?

A lot. The same model can score very differently depending on the harness it runs in. But the effect is not as simple as the most quoted chart suggests. On the first version of the Terminal-Bench test, Claude Code scored worse with Anthropic’s own model than several third-party harnesses did, and that result was widely quoted as proof that the harness matters. Then the test was fixed. About a third of its tasks had been broken, and once they were repaired, Claude Code came out ahead while the neutral harness barely moved (Diagram 6a). The top third-party score also turned out to have been tuned on the same tasks it was graded on. So improving the harness does help, but a ranking of harnesses is only as trustworthy as the test behind it [15, 16, 17].

Diagram 6a. Same model, two benchmark versions: Claude Opus 4.6 in Claude Code and in Terminus 2, before and after Terminal-Bench was repaired.

Once the test was fixed, a clear pattern appeared: each lab’s own harness usually works best with its own models (Diagram 6b). Claude Code does best with Claude, Codex CLI with GPT; Google’s Gemini CLI is the exception. This is not luck. Each lab builds and tunes its harness around its own models, and OpenAI even trained its coding model in environments made for that work [8, 17]. The practical lesson: judge a harness with the model it was built for, and treat any claim that “harness X beats harness Y” as true only for a given model. One caveat: the public leaderboard runs Claude Code in a stripped-down mode, not the full product developers use every day [18].

Diagram 6b. Own harness vs neutral Terminus 2: each vendor’s harness against Terminus 2 with the same model, on Terminal-Bench 2.1.

For models without a harness of their own, the choice of harness matters even more. Tested on the same open-weight models, nine popular harnesses scored very differently, and a simple, minimal one came out on top [18]. NVIDIA found the same, with the best harness changing from one benchmark to the next. Most failures came from small settings, and some “passes” were agents looking up the answer online [18]. The takeaway: if you run open models, test several harnesses on your own tasks and check their settings before you trust a score.

There are three findings: first, the model is still the bigger lever: the Terminal-Bench authors found that swapping the model inside Codex CLI moved results far more than swapping the harness around a fixed model [19]. Second, the open-weight gap can be small. In the same minimal harness, MiniMax M2.5 scored 75.8% on SWE-bench Verified against 76.8% for Claude 4.5 Opus, at roughly a tenth of the cost per task [20] (Diagram 6c). Third, configuration is part of the harness: the same product scores differently with a different context limit, reply cap, or toolset [18].

Diagram 6c. Same open harness, different models: mini-SWE-agent with open-weight and closed models, with approximate cost per task. All three are dated snapshots.

The lesson is simple: a score means little without its context. Always report the model and the harness together, along with the test version and the settings used. And when someone compares models, ask which harness they used.

4. Four kinds of harness

Not every harness writes code, and not every harness waits for a human between steps. We can sort harnesses into four groups: Agentic workspace, Background business agents, Coding harness and Autonomous agent fleet. What separates them is not the technology but two simple questions: where does the agent work, and how much does a human still do? Also, most of the information, names, and products identified here represent what our client base uses. (See Diagram 7)

Diagram 7. Four categories of harness, positioned by environment and by the role left to the human, with examples in each. The largest platforms now span all four. Colours identify each category.

Coding harnesses: Claude Code, Codex CLI, OpenCode, Cursor, Cline, and Gemini CLI work inside a repository with an engineer in the loop, steering and approving. This is where our refund was fixed.

Agentic workspaces: ChatGPT, Claude, Microsoft 365 Copilot, Gemini Enterprise, and Open WebUI belong to this family. At Parser, we run ParserAI — our agent platform, built on the latest LibreChat release. It is provider-agnostic, running across several frontier models, and our people use it to build skills, agents, and subagents that reach internal tooling through MCP, across talent, staffing, sales, presales, and account management [21]. It has no repository to inspect, so its context comes from instructions, files, skills, and connectors; the other five components are all there, from prompt caching to subagents. It also executes code in a sandbox and renders the results (documents, spreadsheets, presentations, source code) as artifacts. It is not a coding harness, because code is a means rather than the product. And it is more than a chat, even though it started as one.

If coding harnesses are the IDE of the agentic era, agentic workspaces are closer to its office suite.

Autonomous agent fleets: Devin, Cognition’s cloud agent, takes a task, works inside its own virtual machine with a shell, an editor, and a browser, and comes back with a pull request. Codex cloud follows the same model, running each task in its own sandbox preloaded with the repository [8], and GitHub’s Copilot coding agent takes an assigned issue, works in the background, and returns a draft pull request. Gas Town, Steve Yegge’s open-source orchestrator, runs twenty or thirty Claude Code agents at once on a developer’s machine, with one agent coordinating the rest and their shared state kept in git. Yegge even calls it, half as a joke, a new kind of IDE. People still disagree on how far to split the work between agents. Cognition warned against designs in which parallel agents lose each other’s context, while Anthropic showed a lead agent with parallel subagents excelling at research, at roughly fifteen times the tokens of a chat [22]. Most fleets have settled on the same compromise: a coordinator delegating to bounded, isolated workers.

Background business agents: Sierra runs customer-service agents across chat, voice, and email that take actions in systems of record, such as refunds and plan changes, and hand off to humans when they should. Wonderful, sells the same kind of agent across more than thirty-five countries. n8n sits right on the workflow–agent line: most automations are workflows with a fixed path, triggered by events or schedules, and an AI Agent node hands the model control only where the path cannot be fixed in advance, with human approval gates at the level of a single tool call. Microsoft’s Copilot Studio agents start from event and recurrence triggers, and Claude’s Cowork can run scheduled tasks without someone at the keyboard. Nobody watches each step. Someone reviews what happened, usually after the customer already has.

Figure 6. Code and business environments. On the code side, an engineer steers a coding harness or reviews what a fleet delivers; on the business side, people delegate to agents in a workspace, or let background agents serve customers on their own.

The largest platforms, ChatGPT, Claude, and Microsoft Copilot, no longer fit in one box. Each now ship a product in all four corners, from chat workspaces and command-line harnesses to cloud coding agents and scheduled business tasks. ParserAI spans two of them: it is a workspace, and its users also schedule agents to run on their own [21].

One layer sits below all four corners and is named meta-harnesses: Claude Managed Agents and OpenAI’s Agents API are some examples. They are managed services that run the loop, keep the session, and provision sandboxes, so a company can build its own fleet or its own background business agent without operating any of that machinery itself [5, 14]. The distinction is important when comparing vendors: Codex cloud is a product in the fleet corner, while the Agents API exposes the same harness and infrastructure for anyone to build on [14].

A harness is something you use, but a meta-harness is something you build with

For a side-by-side comparison of the four categories, see Appendix C.

The categories are not exclusive: Claude Code can run headless in a pipeline, and Devin also ships a command-line tool for local work. What they really describe is how much of the loop the human keeps. That question will decide almost everything in Part II.

In short. A model post-trained to pursue goals, placed in a loop, wrapped in a harness of six components, deployed in one of four categories. Everything that follows is about the engineering system we build around that harness.

A harness is something you use, but a meta-harness is something you build with

Part II. The engineering system around the harness

We’ve been here before. We just used different names.

Think about a traditional software project. An engineer receives a requirement, works within a software development life cycle (SDLC), consults architectural documentation, follows coding standards, uses an IDE, writes tests, and submits changes through a controlled release process. Nobody would confuse the IDE with the methodology, the coding standards with the debugger, or the application logs with the test framework. Each component has a purpose, and each has a name.

Now imagine delegating much of that work to the agent we just built. We are not replacing Software Engineering with Agentic Engineering; we are extending it to accommodate a new kind of engineering actor (Diagram 8).

Figure 7. Extending the discipline. Agentic Engineering keeps the tools and rigour of Software Engineering and adapts them to a probabilistic, autonomous actor.

Diagram 8 makes the lineage explicit, row by row, and most of it comes from two earlier pieces. In The Discipline Gap I argued that we had automated software creation without automating engineering judgement [23]; in The Efficiency Trap, that we had done the same thing to the budget [21].

Three rows deserve a comment, because they are where teams most often fail to get the full value.

  • A team’s roles and responsibilities become multi-agent profiles: planner, architect, implementer, reviewer, and tester defined as agents with their own instructions, tools, and permissions, which is exactly what frameworks such as BMAD-METHOD package.
  • Architecture documents become context the agent actually reads: C4 views, architecture decision records, and design-pattern catalogues referenced from AGENTS.md or CLAUDE.md, rather than sitting in a wiki the agent never opens.
  • Runbooks become skills. ( A skill, policy, or architecture document only helps if the harness actually loads it. We learned this the hard way: an audit found well-written files that no agent ever read.
Diagram 8. Established engineering disciplines and their agentic counterparts, with examples, drawing on The Discipline Gap and The Efficiency Trap [23, 21]. Note: PAEMM(a) is Parser’s Agentic Engineering Maturity Model.

Based on the earlier mapping, I find it useful to group practices into six layers: Layer 1 is the execution harness from Part I. The other five are the engineering system around it, and knowledge and context cut across all of them (Diagram 9). Böckeler’s guides and sensors fit the same map: scaffolding and governance guide the agent before it acts, while observability, evaluation, and regression sense what it did afterwards [3].

Diagram 9. Six core layers surrounding the execution harness, with cross-cutting knowledge and context support.

Layer 2. Workflow scaffolding: the development methodology

OpenSpec · BMAD-METHOD · GitHub Spec Kit · Kiro · AWS AI-DLC

Traditional engineering relies on the SDLC, Agile, test-driven and behaviour-driven development (TDD, BDD), and structured design. Workflow scaffolding organises how an agent turns requirements into software. OpenSpec, for example, structures specifications, design, implementation tasks, and verification; GitHub Spec Kit, Kiro, and AWS AI-DLC follow a similar spec-first logic, and BMAD-METHOD adds a cast of role agents (analyst, product manager, architect, developer, QA) that hand the work to each other.

The harness executes; the scaffolding structures the work.
Figure 8. Scaffolding shapes raw capability. The model is the marble; a short specification (rounding rule, acceptance criteria, file scope) is fixed before the loop begins, and frameworks such as OpenSpec or BMAD-METHOD hold the work in shape.

With scaffolding, the refund fix would not start from a one-line prompt but from a short specification: the rounding rule for zero-decimal currencies, the acceptance test, and the files in scope.

Layer 3. Governance and guardrails: standards and controls

Safety policies · Instructions · Validators · Hooks · Sandboxing · CI

Traditional engineering has coding standards, access controls, branch protection, security policies, reviews, and continuous integration (CI) gates. Agentic Engineering inherits all of them and adds controls for decisions delegated to probabilistic systems. I see four families, placed along the path from prompt to production (Diagram 10). Model-level guardrails shape how the model behaves through alignment and system instructions. Input/output guardrails check what goes in and what comes out, including attempts at prompt injection [24]. Agent/tool guardrails control what the agent may touch. Engineering policies (tests, CI, code review) check the result.

Diagram 10. Four guardrail families along the path from prompt to production: the further right, the more the system enforces rather than asks.

An instruction not to modify a protected file is not a permission boundary that prevents the modification. One asks. The other enforces. The closer a constraint is to security, compliance, money, or irreversible actions, the less we should rely on probabilistic compliance alone. In Diagram 5, our agent was stopped from editing the test by a permission rule, not by a polite request to leave tests alone. I made the same point about money in The Efficiency Trap: a spending limit written into a system prompt is a request; one enforced at the gateway is a wall [21].

The cost of confusing the two is no longer hypothetical. In July 2025, during an explicit code freeze, an agent on Replit deleted a company’s production database and then wrongly reported that it could not be restored [25]. The freeze lived in the conversation. Nothing in the permission system enforced it.

A simple test from Simon Willison, which he calls the lethal trifecta, shows where the hard limits must go. An agent becomes dangerous when it has all three of these at once: access to private data, contact with content from strangers, and a way to send information out. With all three, an attacker can trick it into leaking the data. The safest fix is to take one of the three away [26]. A coding agent needs the first two to do its job, so the one to remove is usually its open access to the internet.

Figure 9. Asking versus enforcing. Alignment and system instructions are requests; hooks, sandboxes, and CI gates are walls. Willison’s lethal trifecta shows where enforcement must sit: remove one of its three legs.

Codex shows why the families must be designed together. Its sandbox covers the shell tool it provides, while tools that arrive through MCP servers are expected to enforce their own guardrails [13]. Add an MCP server, and you have added a door the sandbox does not guard. That is not a flaw in Codex but a boundary that governance has to know exists.

Even a sandbox has limits. Attackers have already used coding agents on developers’ own machines to hunt for secrets, and researchers have shown an agent doing harm without ever leaving its sandbox [27]. A sandbox limits where an agent can go, not what it is trying to do. The safest answer is to keep secrets out of the sandbox altogether, as Managed Agents does by holding credentials in a separate vault [5], and to keep the engineering checks at the right of Diagram 10 in place.

Layer 4. Observability and tracing: from logs to trajectories

AgentsView · Langfuse · LangSmith · OpenTelemetry

Traditional systems use logs, metrics, application performance monitoring (APM), and distributed tracing. Agentic systems need all of that, plus insight into tool calls, context access, errors, iterations, and guardrail interventions. The industry is converging on shared conventions to describe them: OpenTelemetry already defines spans for agents, tool calls, and MCP, although every one of these conventions is still marked as in development. Traces show what the agent did, step by step, from tool-call tracing which captures interactions to trajectory analysis reconstructing the agent’s path. They do not show everything it was thinking — and a model’s written reasoning does not always match what it actually did.

The trace of our refund would show every file the agent read, the blocked edit, and the subagent’s detour. It tells us what happened. It does not tell us whether the fix is right.

Figure 10. Observability: tracing tool calls. The trace follows the agent’s footsteps through the codebase: what it read, what it ran, and where it was blocked. It records what happened, not whether the result is right.

Layer 5. Evaluation: extending software testing

Claude Plugin Evals · Promptfoo · DeepEval · Braintrust

Unit, integration, end-to-end, and acceptance tests remain essential. Agent evals additionally test whether the agent follows instructions, selects the right tools, respects policies, and completes tasks consistently. An evaluation framework supplies the test cases, datasets, graders and the scoring; an evaluation harness runs them — yes, the same word again, with yet another meaning. Some checks are automatic; others need an LLM judge or a person.

There is also a second reason evals are key. These models were trained to make tests pass, so a green test suite is not proof that the work is right. For our refund, a good eval checks two things beyond the test result: that nobody changed the test, and that the fix also works for other currencies without decimals. Evals now need to catch cheating, not only grade results: any change to the tests should fail the eval.

Figure 11. Evals that look past a green suite. For the refund, the eval checks that the assertion was not tampered with and that the fix also holds for other zero-decimal currencies.
Traditional tests verify the software. Agent evals also verify the behaviour of the system producing it.

Layer 6. Regression and release: when the model changes

Regression suites · A/B evaluation · Baselines · Quality gates

Traditional regression testing checks that code changes do not break existing functionality. Agentic Engineering must also assess changes to models, harnesses, skills, hooks, policies, and workflows. A new model can fix more bugs and, in the same release, follow your architecture less closely, cost more, run slower, or open security gaps. So every change has to be measured against a baseline, with real cases and clear pass marks (Diagram 11). “It feels better” is not a release criterion. The reverse also happens: a harness feature built for one model can become dead weight for the next. Anthropic added context resets because Claude Sonnet 4.5 wrapped up tasks early as its context filled, then found that Claude Opus 4.5 no longer needed them [5].

A regression suite should catch what the harness no longer needs, not only what the model can no longer do.
Figure 12. The continuous agentic quality loop: build, observe, evaluate, compare, then improve or release, and start again whenever the model, harness, skill, hook, or policy changes.

Our refund case now joins the regression suite. When the next model arrives and fixes it faster, but starts editing tests again, the baseline will say so.

Across all layers: knowledge and context

Traditional engineers rely on experience, architecture decision records (ADRs), runbooks, documentation, and shared standards. Agents need the same knowledge, delivered through instructions, skills, specifications, retrieval, and tool responses. But knowledge is not governance. An ADR explains why; a skill describes how; a permission boundary decides what is allowed. That is the reason knowledge and context are a cross-cutting capability rather than a seventh layer.

Figure 13. Knowledge and context. Instructions, architecture decision records, and runbooks must reach the agent when it needs them, through files the harness actually loads.

Appendix D follows the refund through every layer and the question each one answers.

In short. The harness lets an agent act. The other layers decide whether we can trust what it did: scaffolding defines the work, governance bounds it, observability records it, evaluation judges it, and regression keeps it judged as everything underneath keeps changing.

Part III. What it means

So what is genuinely new?

Much of Agentic Engineering is an extension of disciplines we already have. The difference is an actor that interprets goals, selects actions, invokes tools, and produces artifacts through partly probabilistic behaviour. That single property creates new needs: context management, behavioural consistency, prompt-injection defenses, trajectory evaluation, and regression testing for models rather than only for code.

Figure 14. Traditional execution follows a fixed path; agentic execution is a probabilistic flow that engineering has to keep within mechanical bounds.

I want to be honest about where this framing may fall short. Some practitioners argue these boundaries will not last: harnesses already include planning, skills, plugins, and hooks that look a lot like scaffolding and governance, and future tools may merge several layers into one product. OpenAI itself reports that codex-1 performs strongly even without AGENTS.mdfiles or custom scaffolding [8], a sign that part of the harness is migrating into the weights. They may well be right. But products merging is not the same as responsibilities merging. Even when one tool does three jobs, we still need to know which of the three failed.

Takeaways for leaders

Start with what the evidence says, because it is more mixed than the marketing. METR’s 2025 randomised trial found experienced open-source developers 19% slower with AI tools while believing they had been faster [28]. METR now considers that result outdated, partly because many developers no longer agree to work without AI [29]; the gap between perceived and measured speed is the finding that survived. Google’s DORA research puts AI adoption among technology professionals at 90%, now associated with higher throughput, but still with higher delivery instability [30]. And in Stack Overflow’s 2026 survey, 48% of developers said they trust AI output only when they can easily verify it [31].

Verification, not generation, is becoming the constraint.

For a leadership team, then, this vocabulary is not academic. Each word maps to a decision, a cost, or a risk. Five questions are worth asking any team or vendor:

1. Which category are we adopting? A coding harness, an agentic workspace, an autonomous fleet, and a background business agent need different controls, budgets, and review models. Buying one while governing it like another is the most common mistake. And before any of them, ask whether the task needs an agent at all: if the steps are known in advance, a workflow is cheaper and easier to govern.

2. Where is each rule enforced, and where is it only requested? Instructions ask; permissions and sandboxes enforce. Anything that touches money, personal data, or production belongs on the enforcing side.

3. Do we have a baseline before we switch models? Without representative cases and explicit thresholds, every model upgrade is a guess, however good the benchmark headlines look. Self-reported speed is not a baseline.

4. Who reviews the outcome when nobody watched the work? In autonomous fleets, review moves to merge time, and accountability has to move with it.

5. How are agent identities and credentials governed? Every agent that works on a repository or a business system holds real credentials, which makes it a non-human identity like any service account. NIST launched its AI Agent Standards Initiative in February 2026 with agent identity and authorisation among its first priorities [32].

Figure 15. Steering the agentic era: the five questions for leadership, from category to identity.

There is also an economic thread running through the harness.

Prompt caching is what keeps an agent’s ever-growing prompt affordable [13]: Anthropic charges a tenth of the normal input price for cached reads by default, and less on its newest models, against a premium of 1.25 to 2 times for writing the cache [33]. Caching lowers the slope of the cost curve; it does not flatten it. Architecture matters even more than price, because, as I put it in The Efficiency Trap, spend lives in the loop, not in the model [21]: agents use about four times the tokens of a chat and multi-agent systems about fifteen times [22]. Harness engineering shows up directly on the invoice — one bug that broke Codex’s cache was simply its tools being listed in a different order.

The standards are arriving too.

The Linux Foundation’s Agentic AI Foundation now stewards MCP and AGENTS.md, the latter already adopted by more than 60,000 open-source projects and agent frameworks [34]. In Europe, the AI Act’s obligations for stand-alone high-risk systems have been pushed back to December 2027 (August 2028 for those built into regulated products), which matters less for coding agents than for agentic workspaces deployed in hiring, credit, or education. None of this is settled. All of it rewards organisations that already know which layer each control belongs to.

From software development to engineering autonomous execution.

The next challenge is not merely building agents that can write software. It is building engineering systems in which we can explain how they operate, control what they are allowed to do, and demonstrate that every change improves the outcome. Or, at the very least, that it does not make it worse.

So perhaps, before we standardise how we engineer these systems, we should agree on what we call their components.

Appendix A. Glossary

Box 1. A short glossary

Table 1. Essential Terminology

Appendix B. The six components in production

Table 2. The six components in production: Codex CLI and Claude Code [6, 13].

Appendix C. The four categories side by side

Table 3. Four categories of harness, side by side.

Appendix D. The running example, layer by layer

Table 4. The running example, layer by layer.

References and further reading

References are numbered in order of first citation and grouped by the section where each is first cited.

Introduction: a vocabulary problem

[1] Evans, E. (2003). Domain-Driven Design: Tackling Complexity in the Heart of Software. Addison-Wesley.

[2] Lopopolo, R. (2026). Harness engineering: leveraging Codex in an agent-first world. OpenAI. https://openai.com/index/harness-engineering/

[3] Böckeler, B. (2026). Harness engineering for coding agent users. martinfowler.com. https://martinfowler.com/articles/harness-engineering.html

[4] Trivedy, V. (2026). The Anatomy of an Agent Harness. LangChain. https://www.langchain.com/blog/the-anatomy-of-an-agent-harness

[5] Martin, L., Cemaj, G., & Cohen, M. (2026). Scaling Managed Agents: Decoupling the brain from the hands. Anthropic. https://www.anthropic.com/research/managed-agents

Part I. From model to harness

[6] Raschka, S. (2026). Components of a Coding Agent. Ahead of AI. https://magazine.sebastianraschka.com/p/components-of-a-coding-agent

[7] Kimi Team (2025). Kimi K2: Open Agentic Intelligence. https://arxiv.org/abs/2507.20534

[8] OpenAI (2025). Introducing Codex. https://openai.com/index/introducing-codex/

[9] METR (2026). Time Horizon 1.1. https://metr.org/blog/2026-01-29-time-horizon-1-1/

[10] METR (2026). Frontier Risk Report (February to March 2026). https://metr.org/blog/2026-05-19-frontier-risk-report/

[11] MacDiarmid, M., et al. (2025). Natural Emergent Misalignment from Reward Hacking in Production RL. Anthropic. https://arxiv.org/abs/2511.18397

[12] Schluntz, E., & Zhang, B. (2024). Building Effective Agents. Anthropic. https://www.anthropic.com/research/building-effective-agents

[13] Bolin, M. (2026). Unrolling the Codex agent loop. OpenAI. https://openai.com/index/unrolling-the-codex-agent-loop/

[14] OpenAI (2026, September 10). Introducing the Agents API. https://openai.com/index/introducing-the-agents-api/

[15] Tessl (2026). Position: Coding Benchmarks Are Misaligned with Agentic Software Engineering. https://arxiv.org/abs/2606.17799 (reproduces the Terminal-Bench 2.0 leaderboard entries for Claude Opus 4.6).

[16] Lee, et al. (2026). Meta-Harness: End-to-End Optimisation of Model Harnesses. Stanford, MIT, KRAFTON. https://arxiv.org/html/2603.28052v1

[17] Snorkel AI and Terminal-Bench (2026). Terminal-Bench 2.1 leaderboard and TB 2.0 vs 2.1 comparison. https://snorkel.ai/leaderboard/terminal-bench-2-1/

[18] Niklaus, J. (2026). Harness Leaderboard (README, changelog, and lab notebook). Hugging Face. https://huggingface.co/spaces/joelniklaus/harness-leaderboard

[19] Merrill, M., Shaw, A., et al. (2026). Terminal-Bench. ICLR 2026. https://arxiv.org/abs/2601.11868

[20] SWE-bench Verified, bash-only leaderboard (mini-SWE-agent v2.0.0), February 2026 snapshot, as compiled by MetaPrompt. https://metaprompt.ai/posts/swe-bench-verified-pro-by-the-numbers

[21] Miceli, M. (2026). The Efficiency Trap. How Do You Estimate and Control Token Costs? Parser. https://www.parserdigital.com/insights/the-efficiency-trap-how-do-you-estimate-and-control-token-costs

[22] Anthropic (2025). How we built our multi-agent research system. https://www.anthropic.com/engineering/multi-agent-research-system

Part II. The engineering system around the harness

[23] Miceli, M. (2026). The Discipline Gap. Is Spec-Driven Development the Answer? Parser, July 2026. https://www.parserdigital.com/insights/the-discipline-gap-is-spec-driven-development-the-answer

[24] OWASP GenAI Security Project (2025). OWASP Top 10 for Agentic Applications 2026. https://genai.owasp.org/

[25] Sharwood, S. (2025). Replit’s AI agent deleted a user’s production database during a code freeze. The Register, 21 July 2025. https://www.theregister.com/software/2025/07/21/vibe-coding-service-replit-deleted-production-database/719783

[26] Willison, S. (2025). The lethal trifecta for AI agents: private data, untrusted content, and external communication. https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/

[27] StepSecurity (2025). s1ngularity: Popular Nx build system package compromised with data-stealing malware. https://www.stepsecurity.io/blog/supply-chain-security-alert-popular-nx-build-system-package-compromised-with-data-stealing-malware

Part III. What it means

[28] Becker, J., et al. (2025). Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. METR. https://arxiv.org/abs/2507.09089

[29] METR (2026). We are Changing our Developer Productivity Experiment Design. https://metr.org/blog/2026-02-24-uplift-update/

[30] Google Cloud DORA (2025). State of AI-assisted Software Development. https://dora.dev/

[31] Stack Overflow (2026). 2026 Developer Survey: AI. https://survey.stackoverflow.co/2026/ai/data

[32] NIST (2026). Announcing the AI Agent Standards Initiative for Interoperable and Secure Innovation. https://www.nist.gov/news-events/news/2026/02/announcing-ai-agent-standards-initiative-interoperable-and-secure

[33] Anthropic. Pricing (prompt caching). Claude Platform Docs. https://platform.claude.com/docs/en/about-claude/pricing

[34] Linux Foundation (2025). Linux Foundation announces the formation of the Agentic AI Foundation (AAIF), anchored by MCP, goose, and AGENTS.md. https://www.prnewswire.com/news-releases/linux-foundation-announces-the-formation-of-the-agentic-ai-foundation-aaif-anchored-by-new-project-contributions-including-model-context-protocol-mcp-goose-and-agentsmd-302636897.html