The AI Agent Stack 2026 — Production Architecture

By mid-2026, a clear architectural consensus has emerged beyond framework wars: agents are not monoliths. They are layered systems, and the interfaces between layers are standardised on the Model Context Protocol (MCP). This article dissects the seven-layer production agent stack we've deployed across multiple systems in the past eighteen months.

1. Perception Layer

The perception layer normalises unstructured and semi-structured inputs — text, images, audio, JSON — into a representation the reasoning layer can consume. Responsibilities include schema inference (parse JSON dynamically or against a known schema), multimodal normalisation (transcribe audio, describe images, chunk documents), and exposing data sources as MCP resources with typed URI templates. The key metric is perception latency: budget under 100 ms for text-only, under 500 ms for multimodal.

2. Reasoning/Cognition Layer

This is the layer that selects which reasoning pattern to apply. It is not the execution itself — that belongs to the planning and tool layers — but the strategic choice of how to think about the current problem.

Pattern	Best For	Typical Latency	Complexity
ReAct (Reason + Act)	Simple Q&A, tool selection, single-step retrieval	1–3 LLM calls	Low
Plan-Execute	Multi-step tasks with clear dependencies	2–5 LLM calls	Medium
Reflexion	Tasks requiring self-correction (code gen, writing)	3–8 LLM calls	Medium
Tree-of-Thoughts	Search, constraint satisfaction, optimisation	5–20+ LLM calls	High
Multi-Agent Debate	Creative tasks, decision quality, fact-checking	5–15+ LLM calls	Very high

In production, we default to ReAct for 80 % of requests and escalate to Plan-Execute or Reflexion only when the router detects ambiguity or failure. This tiered approach reduces median latency by 60 % compared to running complex reasoning on every request.

3. Planning Layer

The planning layer decomposes a goal into subgoals and manages execution order — what to do next, as opposed to how to think about it. The planner emits a JSON plan of atomic steps referencing tools or sub-plans. Dynamic replanning triggers when a step errors or returns unexpected results.

interface PlanStep {
  id: string;
  type: "tool_call" | "sub_plan" | "llm_judgment" | "human_input";
  dependsOn: string[];
  maxRetries: number;
  timeoutMs: number;
}

interface ExecutionPlan {
  goal: string;
  steps: PlanStep[];
  fallback?: PlanStep[];
  maxParallel: number;
}

Plans must be bounded: set a maxSteps limit (15 by default) and a per-step timeout; if neither triggers, escalate via human-in-the-loop.

4. Memory Tier

Agent memory is categorised into three types:

Memory Type	Scope	Storage	Eviction
Working	Current conversation	In-memory buffer (10–50 turns)	LRU after turn limit
Episodic	Past sessions, user preferences	Vector DB + key-value store	TTL-based (30–90 days)
Semantic	Factual knowledge, learned patterns	Knowledge graph + RAG pipeline	Manual curation

Buffer management matters. We use a hierarchical compressor: frequently accessed episodic snapshots are promoted to working memory summaries, and the full history is demoted to vector storage. This is covered in more depth in our Introduction to GraphRAG article.

For the RAG pipeline, the retrieval step lives here; the actual tool call to the vector database lives in the tool layer — a clean separation of concerns.

5. Tool Layer (MCP)

The tool layer is where agents interact with the outside world, dominated in 2026 by the Model Context Protocol (MCP) — an open standard replacing ad-hoc tool APIs with a unified JSON-RPC 2.0 interface over three transport modes:

Transport	Use Case	Firewall Friendly	Latency
stdio	Local tools, development, CI/CD	No (requires stdout pipe)	Lowest
SSE (Server-Sent Events)	Server-to-agent streaming	Yes	Low
Streamable HTTP	Production deployments, load balancers	Yes	Medium

The following example shows an MCP client written in TypeScript that connects to a remote tool server over Streamable HTTP, calls the search_documents tool, and streams the result back:

import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StreamableHTTPTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";

interface ToolCallResult {
  content: Array<{ type: string; text?: string }>;
  isError: boolean;
}

async function callMcpTool(
  serverUrl: string,
  toolName: string,
  args: Record<string, unknown>
): Promise<ToolCallResult> {
  const transport = new StreamableHTTPTransport({
    url: serverUrl,
    headers: { Authorization: `Bearer ${process.env.MCP_API_KEY}` },
  });

  const client = new Client(
    { name: "agent-worker", version: "1.0.0" },
    { capabilities: { tools: {} } }
  );

  await client.connect(transport);

  const result = await client.request(
    {
      method: "tools/call",
      params: { name: toolName, arguments: args },
    },
    ToolCallResultSchema
  );

  await transport.close();
  return result;
}

// Usage
const docs = await callMcpTool("https://mcp.internal/search", "search_documents", {
  query: "production agent stack MCP",
  maxResults: 10,
  filters: { dateAfter: "2026-01-01" },
});

A notable open-source reference is FareedKhan-dev's production-grade-mcp-agentic-system, which organises a production MCP server into twelve components — transport, authentication, tool registry, validation, circuit breaker, rate limiting, caching, structured errors, observability, and governance — and ships a four-agent support copilot (Planner, Retriever, Synthesizer, Critic).

6. Orchestration Layer

The orchestration layer decides who calls what. It is the control plane that routes between the reasoning, planning, memory, and tool layers.

Single-Agent Architecture

One reasoner executes the full plan serially. Simple, debuggable, sufficient for most workloads. The Agentic AI Libraries Compared article details why the LLM router pattern — a single generalist agent with intelligent tool dispatch — outperforms multi-agent alternatives for 80 %+ of use cases.

Multi-Agent Architecture

Multiple specialised agents (researcher, coder, reviewer) collaborate under a supervisor. Useful when tools have incompatible security contexts, the workload requires parallel sub-tasks, or domain separation is a compliance requirement.

class Supervisor {
  private agents: Map<string, Agent>;

  async route(context: AgentContext): Promise<{
    nextAgent: string;
    requiresHumanApproval: boolean;
  }> {
    const response = await this.routerLlm.invoke(
      buildRouterPrompt(context)
    );
    if (response.requiresHumanApproval) {
      await this.publishApprovalRequest(response);
      return { nextAgent: "router", requiresHumanApproval: true };
    }
    return response;
  }
}

Human-in-the-Loop

Every orchestration layer needs an escalation path. We define three tiers: Inform (log and notify, no blocking), Approve (block until human approves), and Override (human provides a direct answer, short-circuiting the agent).

7. Guardrails & Observability

The final layer wraps everything in safety and visibility. Without it, the stack is not production-ready.

Three guardrail types are essential: input guardrails (reject prompt injection, policy violations, PII leakage before they reach the reasoner), output guardrails (validate responses against content policy and factual consistency), and tool-call guardrails (whitelist permitted tools and parameters per tenant).

For observability, export spans with a unified schema to OpenTelemetry:

interface AgentSpan {
  traceId: string;
  parentSpanId: string;
  layer: "perception" | "reasoning" | "planning" | "memory" | "tool" | "orchestration" | "guardrails";
  durationMs: number;
  metadata: {
    model?: string;
    tokensIn: number;
    tokensOut: number;
    toolName?: string;
    error?: string;
    decision?: string;
  };
}

Build dashboards for step-level latency, token budgets per request, pattern distribution (how often ReAct vs Reflexion fires), and guardrail false-positive rates.

Eval-driven development: the agent stack should be testable by automated evals. Define canonical queries, golden answers, and adversarial inputs. Run the suite on every deploy; if quality, latency, or cost regresses, the deploy fails.

Decision Table: When to Use What

Requirement	Recommended Layer/Pattern	Avoid
Simple Q&A, single tool call	ReAct + single agent	Multi-agent debate, ToT
Multi-step data pipeline	Plan-Execute + MCP tools	Naive ReAct loop
Code generation with self-correction	Reflexion + coder agent	Single-pass generation
Compliance-heavy enterprise	Supervisor + guardrails + Otel	Unbounded autonomous loops
Fast prototyping, low latency	LLM Router (single agent)	Full LangGraph state machine
Parallel independent tasks	Multi-agent under supervisor	Sequential single agent
High recall retrieval	MCP + RAG (GraphRAG)	Flat vector search only

The seven-layer model is not dogma — prioritise according to your workload and compliance surface. What matters is the separation of concerns that prevents the monolithic agent failure mode. Build each layer as an independent, testable, observable module, and the stack will survive the transition from prototype to production.

For a deeper comparison of the major frameworks implementing these layers, see Agentic AI Libraries Compared. For the knowledge graph infrastructure that powers the semantic memory tier, see Introduction to GraphRAG.