What Is Code Mode: How to Cut AI Agent Context by 99% Using Code Instead of JSON-RPC

If you have built autonomous AI agents over the past year, you have almost certainly embraced MCP (Model Context Protocol). It has quickly become the open industry standard for giving language models access to external tools, databases, and APIs.

However, as production agent architectures scale, a glaring flaw in raw tool calling has emerged: schema bloat, context rot, and skyrocketing inference costs.

When an agent connects to multiple MCP servers exposing dozens or hundreds of endpoints, your context window degrades before the user even finishes typing their prompt.

The industry's answer—pioneered and adopted by teams at Cloudflare, TanStack, Arize AI, Goose, and across the open-source ecosystem—is Code Mode.

In this deep dive, we’ll explore what Code Mode is, why traditional JSON-RPC tool calling breaks down at scale, and how treating code as the execution interface slashes token overhead by up to 99.9%.

The Bottleneck: Why Raw Tool Calling Doesn't Scale

In traditional tool calling, the LLM is treated as an RPC router. For every tool available, you pass an exhaustive JSON schema describing its name, description, and every nested parameter type.

This model encounters three critical failure points:

1. Schema Bloat & Context Rot

• Small scale: 50 tools across 4 internal servers consume around 9,400 tokens purely on schema definitions in every single turn.
• Enterprise scale: Exposing an entire enterprise surface—such as Cloudflare’s 2,500+ REST API endpoints—via traditional JSON schemas would require ~1.17 million tokens. That exceeds the context window of almost every production model, leaving zero room for conversation history or reasoning.

2. The N+1 Network Round-Trip Tax

Imagine asking an agent: "Find our top 20 customers, fetch their latest orders, and check shipment status."

Under raw tool calling, the agent must:

1. Call getTopCustomers() → wait for model inference → parse response.
2. Trigger 20 sequential or batched tool calls for getOrders() → wait → re-ingest all JSON.
3. Trigger another batch for getShipmentStatus().

Every intermediate payload—megabytes of raw JSON—is dumped directly into the conversation history. You pay for these tokens on every subsequent inference turn, and response latency balloons from seconds to minutes.

3. Arithmetic Hallucinations & Logic Failures

LLMs predict tokens; they do not compute. When raw tool calling forces the model to sum, filter, or join arrays returned from separate endpoints, calculation errors are virtually guaranteed.

What Exactly Is Code Mode?

> Code Mode is an architectural pattern where an agent interacts with tools not through declarative JSON-RPC schema arrays, but by writing and executing programs (typically TypeScript, JavaScript, or Python) against a typed client library inside a secure sandbox.

Instead of registering hundreds of individual function schemas with the model provider, the system exposes a minimal meta-interface—often just two tools:

• search(query): Dynamically discover available types and endpoints on demand (Progressive Disclosure).
• execute(code): Run a complete script in an isolated runtime (V8 isolate, QuickJS, or WASM).

Traditional Tool Calling:
[User Prompt] -> [LLM] -> [Tool Call Request] -> [Client Executes API] -> [Raw JSON to Context] -> [LLM] ... (Repeated N times)

Code Mode:
[User Prompt] -> [LLM Writes Typed Script] -> [Secure Sandbox Executes Loops/Fetches/Logic] -> [Pruned Summary back to LLM]

Why Models Excel at Code Mode

1. Pre-training bias: Modern frontier models have ingested hundreds of billions of lines of real-world code featuring loops, async handling (Promise.all), conditionals, and array manipulations. They are fundamentally better at writing idiomatic TypeScript than navigating bespoke JSON tool-call structures.
2. Context Pruning at the Edge: Filtering, pagination, sorting, and mathematical aggregations happen directly inside the execution sandbox. Only the final computed result is returned to the context window.

Code Comparison: The N+1 Problem in Action

Let’s look at a concrete workflow: "Fetch top 5 products and calculate their average review rating."

The Raw Tool Calling Way

Requires 7 conversational round trips, injects dozens of raw rating objects into context, and relies on the LLM to mentally compute averages.

The Code Mode Way (e.g., TanStack AI / Cloudflare Pattern)

The model generates and runs a single programmatic workflow:

// The agent writes this entire execution block at once
const topProducts = await storeApi.getTopProducts({ limit: 5 });

// Concurrent data retrieval directly inside the runtime
const ratingsData = await Promise.all(
  topProducts.map((product) => storeApi.getProductRatings({ productId: product.id }))
);

// Deterministic filtering and exact math executed in JS engine
return topProducts.map((product, idx) => {
  const scores = ratingsData[idx].ratings.map((r) => r.score);
  const avg = scores.reduce((acc, curr) => acc + curr, 0) / (scores.length || 1);

  return {
    id: product.id,
    name: product.name,
    totalSales: product.totalSales,
    averageRating: Number(avg.toFixed(2)), // 100% mathematical precision
  };
});

The difference:

• Model turns: 1 instead of 7+.
• Token footprint: Only the final 4-line summary enters the conversation context.
• Accuracy: Exact floating-point math handled natively by the JavaScript engine.

The Real-World Numbers: Token Reductions

Sandboxing & Security: Running Untrusted Agent Code

Allowing an LLM to generate dynamic code introduces immediate security requirements. Production Code Mode implementations address this through zero-trust runtime environments:

1. Lightweight Isolates (V8 / Cloudflare Dynamic Workers): Start in under 5ms without container startup overhead.
2. WebAssembly & QuickJS: Lightweight runtimes (like quickjs-emscripten) running directly in-process or in edge functions with strict RAM and CPU execution deadlines.
3. Capability Bindings over Raw Secrets: Scripts never receive raw API keys or database connection strings. Instead, the sandbox provides pre-authenticated proxy client instances where network requests are routed and validated by the host environment.
4. State Persistence Across Turns: Advanced runtimes (such as OpenClaw) allow scripts to store memory pointers and references across turns, letting agents interact with multi-megabyte datasets without repassing data into the LLM context.

When to Use Which Architecture

Final Thoughts

Code Mode is not a replacement for the Model Context Protocol; it is the architectural layer that makes MCP viable at scale.

Instead of forcing language models to act as fragile, latency-heavy RPC brokers, Code Mode meets LLMs where they are strongest: generating clear, typed, expressive code that executes safely in an isolated runtime.