Mastering Long-Horizon AI Tasks: How Context Engineering Powers Deep Agents

The Challenge of Long-Running AI Agent Tasks

For readers tracking the shift, Large Language Models (LLMs) are incredibly powerful, yet their ability to maintain focus and context over extended, complex tasks presents a significant hurdle. While they excel at short, defined jobs, asking an AI agent to perform an hour-long task involving hundreds of tool calls often leads to two predictable failures: context overflow and goal loss. This isn’t primarily a limitation of the model itself, but rather an issue with how its “harness”—the surrounding management layer—handles information.

Meanwhile, This article looks at the ingenious mechanisms employed within this harness to transform shallow, easily distracted agents into robust, goal-oriented systems capable of tackling long-horizon tasks.

Why Bigger Context Windows Aren’t the Ultimate Solution

It might seem intuitive that a larger context window would solve the problem of context overflow. However, evidence suggests diminishing returns.

Research, such as Chroma’s Context Rot report, indicates that LLM performance becomes increasingly unreliable as input length grows, even on simple retrieval tasks. Anthropic‘s context engineering guide explains the underlying mechanism:

In practical terms, Attention creates n² pairwise relationships for n tokens, so every added token depletes a finite “attention budget.” Context is a resource with diminishing returns, not a bucket.

For an agent loop, this situation is even more critical. Each observation from a tool call lands in context and stays there, inexorably pushing the original instructions towards the middle of the window where recall degrades. Goal loss, therefore, isn’t just a model bug; it’s an expected outcome of unmanaged context on sufficiently long tasks.

Mechanism 1: Context Budgeting and Offloading

For example, The first line of defense in managing context is to prevent irrelevant or excessively large information from ever entering the LLM‘s active context window.

• Deep Agents: This framework employs strict offloading rules. For instance, tool responses exceeding 20,000 tokens are written to the filesystem and replaced by a file path alongside a brief preview of the first 10 lines. When the session context approaches 85% of the model’s window, older write and edit tool calls, whose full contents already reside on disk, are truncated to a pointer. Only when these offloading strategies are exhausted does the system resort to summarization.
• Claude Code: Applies similar budgeting techniques before the initial prompt. Auto-memory is capped at the first 200 lines or 25KB. Full tool schemas are deferred by default, with only tool names listed, and load on demand. After compaction, any re-read file over 5,000 tokens returns as a path reference rather than its full content. This approach proves highly effective; a research subagent might process 6,100 tokens of files but return a concise 420-token result to the parent.
• Architectural Budgeting with Subagents: The subagent pattern itself is a powerful form of budgeting. Each subagent can explore extensively, potentially burning tens of thousands of tokens, but returns only a distilled summary (often 1,000 to 2,000 tokens) to its parent. AWS AgentCore, for example, builds this by having a coordinator spawn multiple browser subagents in parallel MicroVMs, with an analyst subagent receiving only their structured findings, significantly reducing overall runtime compared to sequential processing.

Mechanism 2: Compaction

When offloading isn’t sufficient, summarization—or “compaction”—comes into play. This involves taking a conversation nearing the context window limit, summarizing it, and then reinitiating a new context with that summary. The primary challenge here is preventing “goal loss”—ensuring that critical constraints or objectives aren’t inadvertently dropped during the summarization process.

• Goal Preservation Strategies: Implementations differ in what they promise to keep. Claude Code’s compaction prompt focuses on preserving architectural decisions, unresolved bugs, and implementation details while discarding redundant tool outputs. Immediately after compaction, it re-reads up to five of the most recently modified files, reloads matching rules, and re-injects invoked skill bodies. Deep Agents, on the other hand, makes goal preservation a structural feature. Its summary is a structured document with dedicated fields for session intent, artifacts created, and next steps—a design choice that significantly improved performance in experiments. The full original transcript is also written to the filesystem, allowing recovery of any fact that might have been summarized away.
• Compaction at the API Layer: This mechanism is increasingly moving into API layers. OpenAI’s Responses API offers server-side compaction via context_management with a compact_threshold, returning an opaque encrypted compaction item for developers to pass into subsequent calls. OpenAI states that Codex relies on this for sustained long-running coding tasks. The Claude Developer Platform also exposes a compact_20260112 context-management edit with options for custom instructions and pausing after compaction to insert content.

Mechanism 3: Todo-State and Recitation

That said, While compaction protects the goal at the moment of summarization, todo-state and recitation protect it on every turn in between.

• Active Goal Maintenance: Manus agents, for instance, create and continuously rewrite a todo.md file step by step, checking items off as they’re completed. By rewriting this list, objectives are recited into the end of the context, pushing the global plan into the model’s recent attention span and actively reducing “lost in the middle” drift. This is a clever use of natural language to bias the model’s own attention without requiring complex architectural changes.
• Effectiveness and Considerations: The evidence on todo-state is not entirely one-sided. Deep Agents initially shipped a write_todos tool by default but later made TodoListMiddleware opt-in after evaluations showed slightly better reward and lower cost with todos disabled for certain task categories. However, it’s still recommended for long multi-step tasks, less capable models, and UIs that show progress. Claude Code maintains a todo list and re-injects the plan from disk after compaction. Anthropic’s guide describes the general pattern as structured note-taking, where the agent writes a NOTES.md or TODO file outside the window and reloads it. The underlying principle is that the goal exists as a mutable artifact, not only as a message in history that ages and gets summarized. A file rewritten every few turns is always recent, short, and survives any context reset.

Mechanism 4: Memory Strategy Across Sessions

The final piece of the puzzle is what persists after a task ends, enabling continuity and preventing redundant work across sessions.

• Persistent Context: Claude Code re-injects the project-root CLAUDE.md (containing project-level instructions) and auto-memory from disk after every compaction. AWS AgentCore Memory stores events and runs configured extraction strategies in the background, allowing a coordinator to call a “recall” tool on the next run instead of re-researching. Anthropic’s file-based memory tool serves a similar purpose on the Claude platform.
• Cost Considerations: However, persistent context is not without its costs. A study from ETH Zurich found that repository context files like AGENTS.md do not generally improve task success while raising inference costs significantly (LLM-generated files increased cost by 20-23%, developer-committed files by up to 19%). Memory that reloads every session acts as a standing tax on the attention budget. Therefore, it’s advised to keep such persistent files concise (e.g., CLAUDE.md under 200 lines) and move extensive reference material into skills or path-scoped rules that load only when needed.

Testing the Harness: Ensuring Goal Fidelity

Interestingly, A sophisticated context management system is only valuable if the agent can still complete its tasks and recover details it no longer explicitly “sees.”

• Targeted Evaluations: LangChain maintains targeted evaluations to test this crucial aspect. These include tests that trigger summarization mid-task and then check whether the agent continues accurately toward its objective. They also employ “needle-in-a-haystack” scenarios where a specific fact is summarized away but must be recovered through filesystem search. To generate enough events for comparing prompt variants, summarization is often triggered at much lower percentages (e.g., 10-20% of the window) than the default.
• Monitoring Goal Drift: The primary failure to watch for, in LangChain’s view, is goal drift: an agent that asks for clarification immediately after a summary, or incorrectly declares the task complete. AgentCore Evaluations ships a goal success rate evaluator that can score such traces. If you run a harness and haven’t forced a compaction in a test, you don’t yet know what your summary prompt might be inadvertently dropping.

Key Takeaways for Robust AI Agents

• The Harness is Key: Shallow agents fail from context overflow and goal loss; the solution primarily resides in the agent’s harness, not solely the LLM.
• Budget First: Implement aggressive offloading for large tool results (e.g., over 20,000 tokens) and evict old edits when the context window is nearing its limit (e.g., 85%).
• Smart Compaction: Summarization must explicitly name what it keeps, often requiring structured summaries with dedicated fields for session intent and next steps.
• Todo Recitation: Keeps the goal at the end of the context, but its benefit varies by task and model, sometimes incurring extra token costs. Evaluate its utility for your specific use case.
• Strategic Persistent Memory: Essential for cross-session continuity, but be mindful of the increased inference costs. Keep persistent configuration files concise and move extensive reference material into on-demand skills.

Expert Perspective

From an industry angle, the clearest signal around LLM Context Engineering is how it may influence context. The story reads less like a one-day spike and more like a marker of broader movement.

The next phase will depend on how quickly teams, regulators, or customers react. In practice, that gives LLM Context Engineering room to reshape expectations across tool over the near term.

For readers focused on practical impact, the best next step is to watch what changes around compaction once attention turns into execution.

Frequently Asked Questions

Why does LLM Context Engineering matter right now?

The Challenge of Long-Running AI Agent TasksFor readers tracking the shift, Large Language Models (LLMs) are incredibly powerful, yet their ability to maintain focus and context over extended, complex tasks presents a significant hurdle.

What broader change could LLM Context Engineering signal?

While they excel at short, defined jobs, asking an AI agent to perform an hour-long task involving hundreds of tool calls often leads to two predictable failures: context overflow and goal loss.

What should the market watch next around LLM Context Engineering?

This isn’t primarily a limitation of the model itself, but rather an issue with how its “harness”—the surrounding management layer—handles information.Meanwhile, This article looks at the ingenious mechanisms employed within this harness to transform shallow, easily distracted agents into robust, goal-oriented systems capable of tackling long-horizon tasks.Why Bigger Context Windows Aren’t the Ultimate SolutionIt might seem intuitive that a larger context window would solve the problem of context overflow.

Source: https://www.marktechpost.com/2026/09/12/context-engineering-inside-the-harness-4-mechanisms-that-beat-context-overflow-and-goal-loss-on-long-horizon-tasks/