Context engineering for agents: what to keep in the window

Treat the window as a budget#

A context window has a hard limit, but the limit is not the only problem. Even well inside it, a long, cluttered context makes it harder for the model to find and use the one instruction or fact that matters. Cost and latency also grow with the amount of input you send on every call.

In an agent loop this compounds. Each step appends the model's reply and the tool result, and the whole thing is sent again next step. A twenty-step run resends early content twenty times.

So ask a single question about every item you are about to include: does the model need this to make its next decision? If not, leave it out, store it somewhere retrievable, or shrink it. That is context engineering in one sentence.

Selection: decide what earns a place#

Start by listing what is in your context today. It usually falls into these groups:

• Instructions: the system prompt and rules.
• Tool definitions: names, descriptions and schemas for every tool.
• The task: the user request and any constraints.
• History: earlier messages and tool results.
• Retrieved material: documents, memories, search hits.

Go through each and prune. Tool definitions are the easiest to overlook: if the task only needs email and calendar, do not load tools for ten other apps. Instructions tend to accumulate patches after every bug, so periodically rewrite them from scratch and delete rules that no longer matter. Keep the instruction text specific and positive about what to do, since a long list of prohibitions is easy for the model to lose.

Retrieval versus always-on context#

For each piece of knowledge you have a choice: put it in the prompt every time, or fetch it when needed.

Always-on suits small, stable and frequently needed facts: the agent's role, output format, the user's name and timezone, a few hard rules. It costs tokens on every call but never gets missed.

On demand suits large or rarely needed material: documentation, past tickets, project notes, file contents. Give the agent a search or read tool and let it pull in the relevant piece. The cost is an extra step and the risk that it does not look. You reduce that risk with good tool descriptions and, for critical facts, a one-line pointer in the always-on prompt such as "Team conventions are in the notes tool; check it before drafting."

A rule of thumb that you can defend: if you would not want to read it on every single step yourself, make it retrievable.

Trim tool results at the source#

Tool output is the biggest source of context bloat. A single call that returns a full web page, a 500-row spreadsheet or a raw API payload can outweigh everything else you have so far.

Fix it in the tool, not in the prompt:

• Return selected fields, not the whole object.
• Cap list lengths and add pagination with a note such as "showing 25 of 310".
• Truncate long text and offer a way to fetch a specific range.
• Strip markup, tracking parameters and metadata the model cannot use.
• For big payloads, store the full result outside the window and return a short summary plus a handle the agent can query.

// Instead of returning the full email thread:
return {
  subject: msg.subject,
  from: msg.from,
  received: msg.date,
  snippet: msg.body.slice(0, 500),
  hasMore: msg.body.length > 500,
  id: msg.id,
};

Once a result has served its purpose, consider replacing it in the history with a one-line note of what it showed.

Compaction: summarize history without losing the plot#

Long runs eventually need compaction: replacing older turns with a shorter summary. Done badly, it deletes exactly the detail you needed. Done well, it keeps the run coherent for far longer.

Practices that help:

1. Compact before you hit the limit, not after, so you are choosing what to keep rather than having it cut for you.
2. Keep the task and decisions verbatim. Summarize the exploration, not the goal. The original request, constraints and decisions already made should survive intact.
3. Preserve identifiers. File paths, issue numbers, record IDs and URLs are the things a summary loves to drop and an agent cannot guess.
4. Record open items. A short "done / in progress / next" list prevents repeated work.
5. Write important state to a file or note outside the window, so it survives a compaction.

Test compaction like any other part of the harness: run a long task and check whether the agent still knows what it was doing afterwards.

Sub-agents to isolate context#

A sub-agent is a separate agent run with its own fresh context. The main agent delegates a bounded job, such as "find every mention of the old API in these repositories", and receives only a short result. All the noisy intermediate work, the dozens of searches and file reads, stays in the sub-agent's window and is discarded.

This is useful when a subtask is read-heavy, when it needs a different set of tools, or when you want parallel work. The cost is real: more model calls, a handoff that can lose nuance, and harder debugging. So the contract matters. Give the sub-agent a precise task, tell it what format to return, and cap that return size.

Do not reach for sub-agents by default. If a single agent with trimmed tool results handles the job, keep it simple. Add delegation when you can point to a context problem it solves.

Order the prompt for caching#

Many model providers offer prompt caching: if the beginning of your request matches a recent request exactly, that prefix can be reused instead of processed again, which reduces cost and latency. The details vary by provider, so check yours, but the design principle is stable: put what never changes first, and what changes most last.

A cache-friendly order is:

1. System instructions and rules.
2. Tool definitions, in a fixed order.
3. Stable reference material.
4. Conversation history, append-only.
5. The newest user message and tool results.

Things that break the prefix: putting a timestamp or a request ID near the top, reordering or editing earlier messages, and changing the tool list mid-run. If you need the current time, add it near the end. If you must compact history, accept that you are paying to rebuild the cache once, and do it infrequently.

How Setu approaches this#

Setu sits on the tool side of your context. Instead of connecting a separate integration for each app, an agent reaches the apps a user has connected through one private MCP URL, so you choose which connections to expose rather than loading everything. Read-only connections let an agent look without being able to change anything, and rules per app and risk level keep risky actions behind an approval.

The plain-language activity log gives you a record of what the agent actually did, which makes it easier to see which tool calls were worth their context. Browse the available integrations and the concepts guide to plan which apps an agent really needs.

Frequently asked questions#