What is an agent harness? The part of an AI agent that isn't the model

What an agent harness is#

An agent harness is all the code and configuration that surrounds a language model and turns it into something that can act. The model is a function: text in, text out. The harness is everything else: the prompt you assemble, the tools you expose, the program that runs those tools, the memory you carry between turns, the rules about what is allowed, and the logic that decides when the job is done.

A useful shorthand is agent = model + harness. You usually do not train the model. You do write, test and operate the harness. That is why two teams using the same model can ship agents with wildly different reliability.

The term is borrowed from test harnesses and wiring harnesses: the thing that holds the components together and connects them to the outside world. A model on its own cannot read a file, send an email or remember yesterday. The harness gives it hands, eyes and a notebook, and also puts guardrails around them.

Model versus harness: who does what#

The split is cleaner than it first looks. The model decides what to say or which tool to request next. The harness does everything that touches the real world.

• The model reads the context it is given and produces either text or a structured request such as "call search_issues with these arguments".
• The harness builds that context, validates the request, executes the tool, captures the result, trims it, appends it to the conversation and calls the model again.

This matters because tool calling, as exposed by model APIs, does not execute anything. The API returns a structured description of a call the model wants made. Your program has to run it. Every safety property you care about, such as "never delete without asking", has to live in code on the harness side, because the model cannot enforce it on itself.

The core loop in a dozen lines#

At the center of every harness is a loop. Strip away the frameworks and it looks like this:

messages = [system_prompt, user_task]
for step in range(MAX_STEPS):
    reply = model.call(messages, tools=tool_definitions)
    messages.append(reply)
    if not reply.tool_calls:
        return reply.text            # the model decided it is done
    for call in reply.tool_calls:
        check_permission(call)       # allow, ask a human, or refuse
        result = run_tool(call)      # the harness acts, not the model
        messages.append(tool_result(call.id, trim(result)))
raise StepLimitReached()

Every interesting design question hides in one of these lines. How do you build system_prompt? What goes into tool_definitions? What does check_permission do when the answer is "ask"? How does trim keep a 5 MB response out of the window? What happens when MAX_STEPS is hit? Those choices are the harness.

What is inside a harness#

Real harnesses grow past the loop. The parts that show up again and again are:

• Tools: the actions the agent can take, with schemas and descriptions the model can understand.
• Context management: deciding what the model sees on each call, and what gets dropped or summarized.
• Memory: facts and preferences that survive beyond one conversation.
• Permissions and approvals: which actions run freely, which need a human, which are forbidden.
• Control flow: step limits, retries, timeouts, planning, sub-agents.
• Evaluation: repeatable tasks that tell you whether a change helped or hurt.
• Observability: a record of every model call and tool call so you can see what happened.

The core vocabulary is covered in our concepts guide, and the next post in this series walks through the layers one by one. For now, the point is that each one is ordinary software with ordinary failure modes.

Why the harness decides whether an agent works#

Consider the common ways an agent goes wrong and where the fix actually lives.

• It calls the wrong tool. Usually two tool names or descriptions overlap, so the fix is in the tool definitions.
• It forgets what it did ten steps ago. The fix is in context handling: the earlier result was trimmed or compacted badly.
• It loops forever. There is no step limit, no repeated-call detection and no budget.
• It does something destructive. There was no permission check between the model's request and the execution.
• You cannot tell why it failed. Nothing was logged.

None of these are fixed by switching to a bigger model. A stronger model may hide some of them for a while, but a harness without limits and logs just fails more expensively. That is the argument for treating harness engineering as a discipline rather than glue code.

Common harness mistakes#

Giving the agent every tool you have

Each tool definition costs context on every call and adds another way to choose wrongly. Start with the few tools the task needs and add more only when an evaluation shows a gap.

Returning raw API responses

A tool that dumps a full JSON payload into the conversation wastes the window. Return the fields the model needs to decide the next step, and a way to ask for more.

Trusting the model to police itself

An instruction like "never delete anything without asking" in the prompt is a request, not a control. Enforce it in code at the point where the tool is executed.

No stop conditions

A loop needs a maximum number of steps, a timeout, and ideally a check for the same call repeating with the same arguments. Without them, one confused run can burn through a budget.

How Setu approaches this#

Setu is the part of the harness that sits between an agent and the apps people actually use. Instead of every team building OAuth flows, token storage and approval logic from scratch, Setu provides them as managed pieces. Connecting an app uses managed OAuth, so you do not register your own OAuth application. Tokens are encrypted at rest with AES-256-GCM envelope encryption. Each user gets a private MCP URL, so the agent calls tools through one endpoint.

The permission layer is built in. New accounts ask before writes and deletes, and you can set a per-app rule of go ahead, ask or never for read, write and delete actions. An activity log records what happened in plain language. See the MCP docs and security overview for details.

How to build your first harness#

If you are starting from zero, build in this order.

1. The bare loop with a hard step limit and a timeout. Make it run one task end to end.
2. Two or three tools, each with a precise name and description. Resist adding more.
3. Result trimming so no tool can flood the context.
4. A permission check before execution, defaulting to ask for anything that writes or deletes.
5. Logging of every model call and tool call with arguments and results.
6. A handful of fixed test tasks you re-run after every change.

Only after that should you add memory, sub-agents or planning. Each of those is useful, and each makes failures harder to diagnose if the basics are not already in place.

Frequently asked questions#