Loop Engineering for Sub-Dime Agents: Cutting Token Costs in AI Coding Harnesses

Transitioning from single-turn chat to loop engineering creates an economic inflection point. Autonomous agents compound token consumption and latency across execution. Resolving a routine pull request can easily consume millions of tokens, turning a sub-dollar task into an unsustainable expense.

Author: Balaji Subramaniam, DevRel Engineer, Google Cloud

The core economics: Anatomy of a pull request

To quantify how unmanaged harness bloat compounds, consider five iterations of an agent resolving an issue in a Pull Request. 5-turn trajectory resolving an Object-Relational Mapping (ORM) regression. The table below contrasts a Naive Harness against an Optimized Harness leveraging prefix pinning, AST compaction, dynamic tool loading, sandbox auto-repair, and multi-tier routing.

*Pricing baseline: Frontier tier ($3.00/M prompt, $15.00/M completion); Workhorse tier ($0.30/M prompt, $1.50/M completion); 75% prompt cache discount. *

The cost-saving mechanisms

Sub-dime autonomous trajectories for coding agents depend on five coordinated mechanisms embedded in the coding agent harness runtime.

1. Strict prefix invariance (KV-cache pinning)

Server-side prompt caching hashes token prefixes. Mutating a single character at token index 0 invalidates the entire downstream KV cache.

Naive harnesses often prepend dynamic timestamps, turn counters (Turn 4 of 20), or commit hashes to the system prompt, causing a 100% cache miss rate.

The better approach: Enforce a stable prefix + append-only tail layout:

• Stable prefix (cached): Core instructions, repository conventions, invariant rules.

• Append-only tail (dynamic): User requests, assistant turns, and tool outputs.

• Reverse compaction: When approaching context limits, prune older intermediate tool outputs from the tail of the message history without mutating prefix tokens.

2. AST & syntax-aware tool output compaction

Reading full implementation files wastes thousands of tokens on internal logic irrelevant to module navigation.

The better approach: Implement an AST-aware reader using Tree-sitter or native syntax parsers. When an agent requests a file overview, the harness extracts an outline containing class definitions, method signatures, type annotations, and docstrings, truncating method bodies. Full source lines are loaded only when the agent explicitly requests a target line range for refactoring, reducing observation tokens.

3. Deterministic sandbox pre-flight & auto-repair

LLM inference is the most expensive compute primitive in the loop. Invoking a frontier model to fix indentation or unused imports wastes budget.

The better approach: All model-generated code edits are intercepted by an ephemeral container sandbox. The sandbox executes deterministic tools (ruff check, black, prettier, mypy). If formatters resolve the issue, the turn concludes without triggering an LLM inference call.

4. On-demand tool schema lazy loading & code-mode execution

In standard Model Context Protocol (MCP) implementations, registering multiple enterprise MCP servers (GitHub, Jira, databases) injects tokens of raw JSONSchema definitions on Turn 0.

The better approach: Replace static schema dumps with a 3-step progressive discovery protocol:

1. search_tools(query): Lightweight semantic search returning tool names and 1-line descriptions (~200 tokens).

2. describe_tools([names]): Full JSONSchemas returned only for tools the agent intends to call.

3. execute_tool(name, args): Sandboxed execution with response projection.

An implementation of this on-demand approach is the Google Cloud Developer Plugin for AI Coding Agents.

Code-mode sandbox execution: When interacting with data-heavy MCP tools (e.g., querying databases or searching logs), tool chaining in dynamic loops forces massive JSON arrays into the LLM context. In Code-Mode Execution, the agent writes a short Python or Bash script that executes inside the sandbox. The script queries the MCP endpoint, aggregates and filters data in memory, and prints only the final projected summary to stdout, reducing token consumption.

5. Governed model routing

Relying exclusively on frontier-tier models for every step of an autonomous trajectory is an anti-pattern. A typical 40-turn refactoring task consists predominantly of mechanical file edits, linter checks, and test executions. Harness developers can reduce costs by partitioning inference across two tiers:

• Workhorse tier (fast & efficient) handles default execution turns, file slicing, test runs, and mechanical diffs.

• Frontier tier (high reasoning) handles ambiguous architectures, initial decomposition, and root-cause analysis.

The 4 Deterministic Escalation Triggers:

1. Planning & decomposition phase: The initial turn requiring architectural decomposition and task graph planning.

2. Consecutive gate failures: A pre-commit test or linter gate has failed > 3 consecutive times on the same target file.

3. AST public signature mutation: An AST diff between original and modified files detects changes in public class names, function signatures, or parameter lists.

4. Schema validation failure: The Workhorse model's output fails structural JSON/Pydantic validation.

Operational safeguards & fleet governance:

• Avoid thrashing: Apply a sticky escalation lock that keeps the session in Frontier mode until the active file passes all verification gates, paired with an escalation cap per task.

• Progressive spending friction: Progressively downshift tasks from Frontier to Workhorse as budgets approach limits, or route to private endpoints (e.g., specialized Gemma deployments on Google Cloud Model Garden leveraging Scale-to-Zero and Spot VMs Infrastructure savings, reserving hard circuit breakers strictly for runaway cyclic loops.

Companion reference implementation

Try out the verified Python 3.11+ reference implementations for these core patterns and tests.

• CacheInvariantPromptBuilder: Isolates immutable system directives into a pinned prefix and maintains an append-only tail with reverse compaction to prune older tool outputs without altering prefix hash signatures.

• ASTAwareFileReader: Parses AST syntax trees to extract structural skeletons (class and method signatures, type hints, docstrings), replacing function bodies with ... to compress file observations.

• DynamicMCPRegistry: Implements lazy 3-step MCP discovery (search_tools -> describe_tools -> execute_tool) and filters response payloads via field projections to eliminate nulls and metadata bloat.

• GovernedTierRouter: Implements Workhorse-first Escalation Routing with the 4 deterministic triggers, sticky escalation locking, and budget-aware progressive downshifting.

Architecture for token efficiency

Optimizing token expenditure across autonomous AI coding fleets is fundamentally an exercise in systems architecture. By shifting from static context dumping to disciplined runtime orchestration, engineering organizations can unlock scalable developer velocity while keeping operational budgets under control:

1. Pin context prefixes: Enforce prefix invariance to maximize KV-cache hit rates across multi-turn trajectories.

2. Compact file observations: Use AST skeletal extraction to reduce file observation tokens.

3. Automate deterministic pre-flights: Offload syntax and formatting fixes to sandboxed tools without invoking LLM inference.

4. Adopt dynamic capabilities: Replace Turn 0 schema dumps with on-demand progressive discovery.

5. Route with governance: Default to the Workhorse tier, reserving Frontier escalations for deterministic triggers with sticky locks to prevent cascade thrash.