The AI Agent Stack 2026 — Production Architecture
AIThe AI Agent Stack 2026 — Production Architecture
By mid-2026, a clear architectural consensus has emerged beyond framework wars: agents are not monoliths. They are layered systems, and the interfaces between layers are standardised on the Model Context Protocol (MCP). This article dissects the seven-layer production agent stack we've deployed across multiple systems in the past eighteen months.
1. Perception Layer
The perception layer normalises unstructured and semi-structured inputs — text, images, audio, JSON — into a representation the reasoning layer can consume. Responsibilities include schema inference (parse JSON dynamically or against a known schema), multimodal normalisation (transcribe audio, describe images, chunk documents), and exposing data sources as MCP resources with typed URI templates. The key metric is perception latency: budget under 100 ms for text-only, under 500 ms for multimodal.
2. Reasoning/Cognition Layer
This is the layer that selects which reasoning pattern to apply. It is not the execution itself — that belongs to the planning and tool layers — but the strategic choice of how to think about the current problem.
| Pattern | Best For | Typical Latency | Complexity |
|---|---|---|---|
| ReAct (Reason + Act) | Simple Q&A, tool selection, single-step retrieval | 1–3 LLM calls | Low |
| Plan-Execute | Multi-step tasks with clear dependencies | 2–5 LLM calls | Medium |
| Reflexion | Tasks requiring self-correction (code gen, writing) | 3–8 LLM calls | Medium |
| Tree-of-Thoughts | Search, constraint satisfaction, optimisation | 5–20+ LLM calls | High |
| Multi-Agent Debate | Creative tasks, decision quality, fact-checking | 5–15+ LLM calls | Very high |
In production, we default to ReAct for 80 % of requests and escalate to Plan-Execute or Reflexion only when the router detects ambiguity or failure. This tiered approach reduces median latency by 60 % compared to running complex reasoning on every request.
3. Planning Layer
The planning layer decomposes a goal into subgoals and manages execution order — what to do next, as opposed to how to think about it. The planner emits a JSON plan of atomic steps referencing tools or sub-plans. Dynamic replanning triggers when a step errors or returns unexpected results.
interface PlanStep {
id: string;
type: "tool_call" | "sub_plan" | "llm_judgment" | "human_input";
dependsOn: string[];
maxRetries: number;
timeoutMs: number;
}
interface ExecutionPlan {
goal: string;
steps: PlanStep[];
fallback?: PlanStep[];
maxParallel: number;
}
Plans must be bounded: set a maxSteps limit (15 by default) and a per-step timeout; if neither triggers, escalate via human-in-the-loop.
4. Memory Tier
Agent memory is categorised into three types:
| Memory Type | Scope | Storage | Eviction |
|---|---|---|---|
| Working | Current conversation | In-memory buffer (10–50 turns) | LRU after turn limit |
| Episodic | Past sessions, user preferences | Vector DB + key-value store | TTL-based (30–90 days) |
| Semantic | Factual knowledge, learned patterns | Knowledge graph + RAG pipeline | Manual curation |
Buffer management matters. We use a hierarchical compressor: frequently accessed episodic snapshots are promoted to working memory summaries, and the full history is demoted to vector storage. This is covered in more depth in our Introduction to GraphRAG article.
For the RAG pipeline, the retrieval step lives here; the actual tool call to the vector database lives in the tool layer — a clean separation of concerns.
5. Tool Layer (MCP)
The tool layer is where agents interact with the outside world, dominated in 2026 by the Model Context Protocol (MCP) — an open standard replacing ad-hoc tool APIs with a unified JSON-RPC 2.0 interface over three transport modes:
| Transport | Use Case | Firewall Friendly | Latency |
|---|---|---|---|
| stdio | Local tools, development, CI/CD | No (requires stdout pipe) | Lowest |
| SSE (Server-Sent Events) | Server-to-agent streaming | Yes | Low |
| Streamable HTTP | Production deployments, load balancers | Yes | Medium |
The following example shows an MCP client written in TypeScript that connects to a remote tool server over Streamable HTTP, calls the search_documents tool, and streams the result back:
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StreamableHTTPTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";
interface ToolCallResult {
content: Array<{ type: string; text?: string }>;
isError: boolean;
}
async function callMcpTool(
serverUrl: string,
toolName: string,
args: Record<string, unknown>
): Promise<ToolCallResult> {
const transport = new StreamableHTTPTransport({
url: serverUrl,
headers: { Authorization: `Bearer ${process.env.MCP_API_KEY}` },
});
const client = new Client(
{ name: "agent-worker", version: "1.0.0" },
{ capabilities: { tools: {} } }
);
await client.connect(transport);
const result = await client.request(
{
method: "tools/call",
params: { name: toolName, arguments: args },
},
ToolCallResultSchema
);
await transport.close();
return result;
}
// Usage
const docs = await callMcpTool("https://mcp.internal/search", "search_documents", {
query: "production agent stack MCP",
maxResults: 10,
filters: { dateAfter: "2026-01-01" },
});
A notable open-source reference is FareedKhan-dev's production-grade-mcp-agentic-system, which organises a production MCP server into twelve components — transport, authentication, tool registry, validation, circuit breaker, rate limiting, caching, structured errors, observability, and governance — and ships a four-agent support copilot (Planner, Retriever, Synthesizer, Critic).
6. Orchestration Layer
The orchestration layer decides who calls what. It is the control plane that routes between the reasoning, planning, memory, and tool layers.
Single-Agent Architecture
One reasoner executes the full plan serially. Simple, debuggable, sufficient for most workloads. The Agentic AI Libraries Compared article details why the LLM router pattern — a single generalist agent with intelligent tool dispatch — outperforms multi-agent alternatives for 80 %+ of use cases.
Multi-Agent Architecture
Multiple specialised agents (researcher, coder, reviewer) collaborate under a supervisor. Useful when tools have incompatible security contexts, the workload requires parallel sub-tasks, or domain separation is a compliance requirement.
class Supervisor {
private agents: Map<string, Agent>;
async route(context: AgentContext): Promise<{
nextAgent: string;
requiresHumanApproval: boolean;
}> {
const response = await this.routerLlm.invoke(
buildRouterPrompt(context)
);
if (response.requiresHumanApproval) {
await this.publishApprovalRequest(response);
return { nextAgent: "router", requiresHumanApproval: true };
}
return response;
}
}
Human-in-the-Loop
Every orchestration layer needs an escalation path. We define three tiers: Inform (log and notify, no blocking), Approve (block until human approves), and Override (human provides a direct answer, short-circuiting the agent).
7. Guardrails & Observability
The final layer wraps everything in safety and visibility. Without it, the stack is not production-ready.
Three guardrail types are essential: input guardrails (reject prompt injection, policy violations, PII leakage before they reach the reasoner), output guardrails (validate responses against content policy and factual consistency), and tool-call guardrails (whitelist permitted tools and parameters per tenant).
For observability, export spans with a unified schema to OpenTelemetry:
interface AgentSpan {
traceId: string;
parentSpanId: string;
layer: "perception" | "reasoning" | "planning" | "memory" | "tool" | "orchestration" | "guardrails";
durationMs: number;
metadata: {
model?: string;
tokensIn: number;
tokensOut: number;
toolName?: string;
error?: string;
decision?: string;
};
}
Build dashboards for step-level latency, token budgets per request, pattern distribution (how often ReAct vs Reflexion fires), and guardrail false-positive rates.
Eval-driven development: the agent stack should be testable by automated evals. Define canonical queries, golden answers, and adversarial inputs. Run the suite on every deploy; if quality, latency, or cost regresses, the deploy fails.
Decision Table: When to Use What
| Requirement | Recommended Layer/Pattern | Avoid |
|---|---|---|
| Simple Q&A, single tool call | ReAct + single agent | Multi-agent debate, ToT |
| Multi-step data pipeline | Plan-Execute + MCP tools | Naive ReAct loop |
| Code generation with self-correction | Reflexion + coder agent | Single-pass generation |
| Compliance-heavy enterprise | Supervisor + guardrails + Otel | Unbounded autonomous loops |
| Fast prototyping, low latency | LLM Router (single agent) | Full LangGraph state machine |
| Parallel independent tasks | Multi-agent under supervisor | Sequential single agent |
| High recall retrieval | MCP + RAG (GraphRAG) | Flat vector search only |
The seven-layer model is not dogma — prioritise according to your workload and compliance surface. What matters is the separation of concerns that prevents the monolithic agent failure mode. Build each layer as an independent, testable, observable module, and the stack will survive the transition from prototype to production.
For a deeper comparison of the major frameworks implementing these layers, see Agentic AI Libraries Compared. For the knowledge graph infrastructure that powers the semantic memory tier, see Introduction to GraphRAG.