Enterprise AI Agent Architecture · DWG EAA-2026-01 · REV B
Architecture and operational discipline for AI agents in production.
An agentic system is justified when it automates work that previously required continuous human judgment, executes that work reliably without supervision, and keeps a human in control of decisions that carry meaningful risk.
of organizations had deployed AI agents
Gartner 2026 CIO survey
expect to deploy within two years
Gartner 2026 CIO survey
of agentic AI projects projected to be scrapped by 2027
cited causes: weak governance, unclear ROI — Gartner
Framing
The agentic spectrum
Defines the agentic spectrum, from deterministic scripts to multi-agent systems, and identifies which decisions are made by a human versus the system at each stage.
Who owns the decision
Read across: stages 00–01 are entirely author-decided — including retrieval-augmented generation, where a human wired the single retrieval and the single generation in advance. The first cell flips to SYS at stage 02, which is exactly where “agent” starts meaning something.
AUTONOMY — SINGLE-STEP DECISIONS
Model reasons, calls tools, stops
Example: “Look this up and summarize it”
MECHANISM — One reasoning pass selects a tool, observes the result, and returns — no replanning loop.
The critical distinction
RAG is not agentic. RAG retrieves and answers. An agent decides what to retrieve, what to do with it, and whether the result is good enough to stop.
Core design patterns
Single-agent, then multi-agent
Covers the three single-agent design patterns and six multi-agent topologies used in production, with their measured token-cost overhead.
Single-agent patterns — the default starting point
ReAct (Reason + Act)
Reflection / self-critique
Plan-and-execute
Six multi-agent shapes — select one to read it
Supervisor / orchestrator-worker
SHAPE — Coordinator delegates to specialists
Best for
The production default — roughly 70% of deployments
Watch out for
Coordinator becomes a bottleneck if under-specified
Tokens consumed for the same task — indexed to a tuned single agent
The dashed line is parity. Production measurements put peer-to-peer overhead at roughly +58% and centralized supervisor setups at roughly +285% against a well-tuned single agent doing the same task. Multi-agent is a decomposition tool, not a default — prove the bottleneck with a baseline before you pay this.
Cost, not just capability
Multi-agent systems have a measurable token cost. Production measurements put independent (peer-to-peer) overhead at roughly 58% more tokens than a well-tuned single agent doing the same task, and centralized (supervisor) setups around 285% more. Multi-agent architecture is a decomposition tool, not a default: use it when a task benefits from specialization, parallelism, or independent critique, and confirm that with a baseline comparison first. Start with one well-scoped agent and its tools; add coordination only when a measured bottleneck justifies it.
As frontier models increasingly handle planning and reflection natively — extended or interleaved thinking woven directly into the model's own turn, before and between tool calls — some of the hand-built ReAct/reflection scaffolding above is becoming the model's job rather than the framework's. The patterns still matter as a mental model; where they get implemented is shifting. More in Section 13.
The modern agent stack
Seven layers of the agent stack
Seven layers, from model to observability. Select a module to expand it with the current landscape for that layer.
The orchestration layer has consolidated over the past year around a handful of options — LangChain/LangGraph, Microsoft Agent Framework, Claude Agent SDK, Google ADK, and AWS Strands Agents — each suited to a different orchestration pattern and ecosystem. See Section 4.
Seven layers, cloud-agnostic — L01 model through L07 observability. Select a module to inspect the current landscape at that layer; the bus pulse tracks whichever one is open. Layers L04–L07 — memory, guardrails, identity, and observability — each get a dedicated deep dive later in this reference.
How to read this
The stack is deliberately cloud-agnostic. Every managed platform covered in sheet 10 packages some subset of these seven layers under its own naming. Understanding the layers independently makes it possible to evaluate a platform on its technical merits.
Where each layer is covered in depth
- L02
- Sheet 04 — frameworks
- L03
- Sheets 05–06 — tool design, interop
- L04
- Sheet 07 — memory and context
- L05–06
- Sheet 09 — security and governance
- L07
- Sheet 08 — traceability and evaluation
Framework deep-dive
LangChain, LangGraph, and the orchestration-layer landscape
Covers LangChain and LangGraph as the orchestration layer, and their relationship to Microsoft Agent Framework, Claude Agent SDK, Google ADK, and AWS Strands.
The relationship, precisely
LangGraph is the low-level runtime — a graph-based execution engine providing durable state (execution persists automatically and survives a server restart mid-conversation), streaming, checkpointing, and human-in-the-loop interrupts. LangChain is the high-level framework built on top of it: a standard tool-calling architecture, provider-agnostic model swapping, and — new in the 1.0 line — a middleware system for composable behavior. Drop to LangGraph directly for maximum control, or use LangChain's create_agent for a proven ReAct pattern with far less boilerplate.
LangChain — create_agent
LangGraph directly
LangSmith
Which one to reach for, in practice
What changed in the 1.0 line
Both reached a stable 1.0 release in October 2025, with a stated commitment to no breaking changes until 2.0. The older create_react_agent prebuilt (langgraph.prebuilt) is now deprecated in favor of LangChain's create_agent, which provides the same pattern plus the middleware system — composable hooks for human-approval gates, automatic conversation compression, and PII stripping, without touching the agent's core logic. In production at real scale: Uber, LinkedIn, Klarna, Replit, and Elastic are named public examples, across customer support, code migration, and threat-detection workloads.
Where this sits
LangChain/LangGraph is one orchestration-layer choice among several (Microsoft Agent Framework, Claude Agent SDK, Google ADK, AWS Strands). It leads on ecosystem breadth: the widest integration count, model-agnostic design, and — as of the 1.0 line — an explicit production-stability commitment, rather than the fast-moving research posture of earlier releases.
Tool design and usage
Tool design, selection, and the eight-tool ceiling
Tool design principles and the tool-selection ceiling. The interactive model below shows the effect of tool count on selection quality.
Tools registered on one agent
6
Selection quality
95%
Inside the working range
Selection stays reliable while the agent can hold every option distinctly. Keep tools few and high-leverage: one call that reconciles an invoice against a PO beats five granular endpoints the agent has to chain correctly.
Directional curve, load-bearing threshold. The shape illustrates a degradation mode that is well attested in practice; the number that matters is the inflection at roughly eight concurrently registered tools, not the exact percentage at any point.
The core reframe
A tool is a contract between a deterministic system (an API, a database, a file system) and a non-deterministic caller. Traditional API design optimizes for a human reading documentation once and writing code against it forever. Tool design has to optimize for a model reading the tool's description fresh on every call and deciding, in context, whether and how to use it — which makes the tool's name, description, and parameter docs load-bearing in a way a REST endpoint's docstring never was.
Five principles
- 01
Choose high-leverage tools, not thin API wrappers
A tool that consolidates a multi-step workflow into one call (“reconcile this invoice against the PO”) is worth more than exposing five granular endpoints and hoping the agent chains them correctly. Fewer, more capable tools beat many narrow ones.
- 02
Namespace clearly
Prefix or group tools by domain (finance_get_expenses, not get_expenses) so an agent choosing between similar-sounding tools across systems is not guessing.
- 03
Return meaningful context, not raw data
Natural-language identifiers over cryptic IDs, high-signal fields over a full record dump. If the agent needs a second call just to make sense of the first response, the tool is under-designed.
- 04
Optimize for token efficiency
Pagination, filtering, truncation, sensible defaults (a default result limit, not “return everything”), response-format options, and error messages that guide the agent toward a better query instead of just failing.
- 05
Treat tool descriptions as prompt engineering, not documentation
They are loaded directly into the model's context and materially steer behavior — Anthropic has reported that precise refinement of tool descriptions alone measurably improved task completion and reduced error rates on real coding benchmarks. Iterate the way a system prompt is iterated: prototype, test, identify exactly where the agent misuses a tool, refine, and re-test.
MCP as the delivery mechanism
In practice, most new tools now ship as MCP servers rather than bespoke per-framework function definitions — one server, callable from any MCP-compliant agent, with tool annotations that disclose which tools have open-world access or make destructive/irreversible changes (directly relevant to Section 9). Function-calling APIs remain the mechanism underneath; MCP standardizes discovery and packaging on top of them.
Test tools independently of the agent. A tool is regular software with an interface: it should carry unit tests, schema validation, and error-path coverage before an agent calls it. Agent-level evaluation (Section 8) determines whether the agent chooses and sequences tools well; it is not the layer that identifies a tool that is simply broken.
Interoperability
MCP and A2A
MCP and A2A, the two interoperability standards for tool connectivity and agent-to-agent communication.
Six agents, six resources. Without a shared protocol that is 36 bespoke integrations, each one written, tested and owned by someone. With MCP on the tool side and A2A between agents it is 12 — one server per resource, one card per agent. That is the argument for standardising inside a single organisation, not just across the industry.
MCP — MODEL CONTEXT PROTOCOL
Connects an agent to tools and data (design side in Section 5). Anthropic released it as an open standard in November 2024; OpenAI, Google, Microsoft, and most major platforms have since adopted it, and governance now sits with the Linux Foundation's Agentic AI Foundation. Monthly SDK downloads went from roughly 100,000 at launch to on the order of 97 million by March 2026.
The USB-C port for AI — one server per resource (Slack, a database, an internal API), any MCP-compliant agent can use it, instead of bespoke integration code per pairing.
A2A — AGENT2AGENT
Connects agents to each other. Google introduced it in April 2025 and donated it to the Linux Foundation two months later, joined by AWS, Cisco, Microsoft, Salesforce, SAP, and ServiceNow; IBM's competing ACP protocol folded into it later that year.
Agents publish “Agent Cards” — metadata describing what they can do and how to reach them — so a system built on one framework can delegate to an agent built on a completely different one.
Without a shared protocol, every new tool or agent-to-agent pairing requires bespoke integration work: N×M custom connections. With a shared protocol, that becomes N+M. This is the argument for standardizing on MCP/A2A even within a single organization, not only for external interoperability.
Memory and context engineering
Context engineering and memory engineering
Context engineering and memory engineering as related but distinct disciplines, and the four context-management levers, each with a tradeoff.
One inference call’s context window
100% FULL — OVERLOADED
Each lever has a cost. Compress trades fidelity for space, Select trades recall for precision, Isolate adds the multi-agent overhead described in sheet 02, and Write moves the problem into a store that must be pruned. Left unmanaged at 100%, context rot follows: quality degrades even though nothing is technically incorrect.
Context engineering
Memory engineering
Two failure modes that show up in production
Context poisoning
Bad or stale information gets written to memory and then contaminates every subsequent session.
Context rot / overload
Too much low-relevance material crowds out what the model needs, and quality degrades even though nothing is technically incorrect.
Traceability and evaluation
Tracing, evaluation, and production readiness
Tracing records what happened during execution; evaluation determines whether the outcome was correct. Covers OpenTelemetry conventions, trajectory evaluation, and LLM-as-judge calibration.
One invoke_agent trace — OTel GenAI conventions
4.20 s total
Where the time went: two tool calls cost 1.36s between them, and the final reconciliation call cost 1.52s on its own. A trace records that. It does not indicate whether the answer was correct — that is what the evaluation layers below are for.
Tracing is not evaluation
Tracing is not evaluation; it is the substrate evaluation runs on. A trace records what happened — which tools were called, in what order, how many tokens, how long — but not whether the outcome was correct.
The industry is converging on OpenTelemetry's GenAI semantic conventions as the standard schema — backed by the CNCF, natively supported by AWS, Azure, Google Cloud, Datadog, Honeycomb, and New Relic, and emitted automatically by LangChain, CrewAI, and other major frameworks. A newer, still-evolving set of conventions extends this specifically to agentic systems — tasks, sub-tasks, agent-to-agent handoffs, memory reads and writes.
Four layers, cheapest first
- 01
Deterministic checks first
Anything with an objectively correct answer — schema validity, exact match, whether the right tool was called, latency thresholds, token budgets — should be a code assertion, not a model call.
- 02
Trajectory evaluation, not just output evaluation
A correct final answer can still conceal three unnecessary API calls and a hallucinated fact underneath. Trajectory evaluation scores the full execution path — tool selection, arguments, redundant steps, recovery after an error — and is the more reliable way to distinguish a consistently correct agent from one that succeeded by chance.
- 03
LLM-as-judge, with real calibration discipline
For correctness, faithfulness, helpfulness, and safety, a second model scoring against an explicit rubric is standard — but it needs a few hundred human-labeled examples as ground truth (500+ comes up repeatedly), discrete rubric scales over open-ended ones, and re-calibration whenever the judge, prompts, or system under test changes. An uncalibrated judge is worse than no judge.
- 04
Verify against the environment, not just the transcript
As agents take real actions across real systems — updating a CRM, opening a pull request, modifying a cloud config — the newest approaches check actual downstream system state, not just whether the transcript looks right.
Offline and online, on a cadence
Close the loop
Route production failures into an annotation queue for expert review, then convert the reviewed case directly into a regression test — a specific trajectory that must not recur. Treat evaluation rubrics themselves as living, versioned artifacts updated from production feedback, not something written once before launch.
Representative tools
Security and governance
Defense in depth for autonomous agents
Prompt injection, defense in depth, agent identity, and liability under the EU AI Act. The interactive model below shows the effect of removing each defensive layer.
Arm or disarm each layer
Stopped at R1. No single control solves prompt injection; the architecture is unsolved and the mitigation is layers. Disarm them one at a time to observe how the remaining coverage narrows.
Prompt injection is still architecturally unsolved
LLMs process the system prompt, the user's input, and anything retrieved from a tool or document as one continuous token sequence — there is no built-in mechanism that reliably separates trusted instructions from untrusted content the agent has read. As agents gain tools and the ability to act, a successful injection stops producing a bad answer and starts producing a real-world action. OWASP maintains two relevant frameworks: the Top 10 for LLM Applications, and, more recently, the Top 10 for Agentic Applications (published December 2025).
Case in point — Replit, 2025
The Replit incident from 2025 — a coding agent deleted a production database and fabricated records despite being explicitly instructed not to touch anything, with no attacker involved — illustrates that AI safety failures and AI security failures are not separate categories once an agent can act on its own.
Agent identity is its own governance problem
Non-human identities already outnumber human identities by a wide margin in most enterprise environments, and agents are increasing that ratio quickly enough that traditional IAM — built around people who log in, hold a session, and are reviewed quarterly — does not map cleanly onto something that acts continuously and autonomously. The response is to treat agents as first-class identities: an owner, a scoped set of permissions, a kill switch, and the same lifecycle discipline applied to a human joiner-mover-leaver process. Microsoft (Entra Agent ID), AWS (AgentCore Identity), and Okta/Google have each shipped agent-specific identity primitives in the past year; a joint CISA/NSA/Five Eyes advisory on agentic AI adoption (May 2026) identifies unbounded privilege as the primary risk class.
Liability sits with the deployer
Legally, an autonomous agent is not its own liable party. Responsibility sits with the deployer — and, in ethics, with whoever designed, commissioned, and signed off on the system — under the EU AI Act and general liability principles alike; “the agent did it” is not a valid defense under any European legal order. On the tooling side this is beginning to appear as concrete controls: Yubico's YubiKey 5.8 line (mid-2026) added hardware security keys that sign off on specific agent actions, not just user login — an early instance of a human approval gate that is cryptographically enforced rather than a checkbox in a dashboard.
High-risk system obligations were originally set for 2 August 2026, but the “Digital Omnibus” agreement reached in May 2026 pushed most of those out — to December 2027 for standalone high-risk systems, August 2028 for AI embedded in already-regulated products — while general-purpose AI model transparency obligations remain in force from August 2025. Still moving through formal adoption as of mid-2026 — verify current status before citing a hard date.
Cloud reference architecture
AWS, Azure, and Google Cloud, compared
AWS, Azure, and Google Cloud each ship a managed runtime, a memory service, an MCP-speaking gateway, and native A2A support. Toggle between them for a direct comparison.
| Capability | AWS — Bedrock AgentCore | Azure — AI Foundry Agent Service | GCP — Gemini Enterprise |
|---|---|---|---|
| Managed runtimePARITY | AgentCore Runtime — serverless, 8-hour sessions, full session isolation | Foundry Agent Service — Connected Agents (delegation) + Multi-Agent Workflows (stateful, visual/YAML) | Agent Engine — managed, autoscaling |
| FrameworkDIFFERENTIATED | Framework-agnostic: Strands, LangGraph, CrewAI, LlamaIndex, and others run unmodified | Built on Microsoft Agent Framework (unified AutoGen + Semantic Kernel successor) | Agent Development Kit (ADK), stable v1.0; also runs LangChain / LangGraph / AG2 |
| Low-code optionDIFFERENTIATED | Not the focus — code-first by design | Portal / YAML workflow agents | Agent Studio (natural-language-to-config generation) |
| Tool connectivityPARITY | AgentCore Gateway — turns APIs/Lambda into MCP-compatible tools, connects to existing MCP servers | 1,400+ MCP-enabled tools; OpenAPI, Logic Apps, SharePoint, Bing | MCP-enabled Vertex AI Search + custom tools |
| MemoryPARITY | AgentCore Memory — session + long-term, self-managed extraction/consolidation | Managed memory built into Foundry | Agent Engine session/context management |
| Identity & accessPARITY | AgentCore Identity — OAuth + IAM, secure token vault | Microsoft Entra Agent ID | IAM service accounts + Gemini Enterprise agent registry |
| GovernanceDIFFERENTIATED | AgentCore Policy — centralized permission/behavior controls | Task-adherence guardrails, PII detection, prompt-injection defenses (opt-in via Foundry) | Gemini Enterprise governance & org-wide discovery layer |
| ObservabilityPARITY | AgentCore Observability — trajectory inspection, CloudWatch-native | AgentOps — accuracy/efficiency scoring built into Foundry | Built-in eval framework + Cloud Trace |
| Interop protocolsPARITY | MCP + A2A native | MCP + A2A (dedicated API head) | MCP + A2A |
| Model choiceDIFFERENTIATED | Any model, in or outside Bedrock (Anthropic, Meta, others) | Azure OpenAI plus a growing external catalog | 200+ models via Model Garden, including Gemini and Claude on Vertex |
Of ten capabilities
6
are at effective parity across all three clouds. The remaining 4 — framework, low-code surface, governance model, model catalogue — are where a real decision lives.
Net assessment
All three platforms are converging on the same shape: a managed runtime, a memory service, a gateway that speaks MCP, an identity layer, and native A2A support. By 2026, the differentiator is not capability — it is which ecosystem an organization has already standardized on, and which governance model its compliance team already trusts.
A build methodology
A methodology for building agentic systems
A ten-step build methodology, applicable across projects rather than specific to any one deployment.
STEP 01 / 10
Find a real candidate
High-volume and repetitive, bottlenecked by a human gathering context and making judgment calls, not by raw execution speed. Poor candidates: low-volume tasks, ambiguous success criteria, or irreversible high-stakes actions with zero tolerance for error.
A repeatable sequence, not one project’s story. Steps 01–03 decide whether to build at all; 04–07 are the build; 08–10 are what keeps it alive once it is running.
The two steps teams skip
07 — instrument before scaling. Tracing, trajectory evaluation, and cost dashboards are part of v1, not a post-incident addition.
09 — govern the agent like a managed identity. An owner, a documented purpose, an access review cadence, and a kill switch that is discoverable during an incident.
Both are inexpensive before launch and costly after an incident — a pattern consistent with the 40%+ project-cancellation figure cited on the title plate.
Applied example
From RAG to an agentic layer
The same parts, rearranged: how a RAG pipeline already in production evolves into an agentic layer.
Retrieval doesn’t disappear — it gets demoted. A RAG system already in production with 300+ users is credible ground to build on: the retrieval layer becomes one tool among several, a planning layer decides when to retrieve versus when to act, and the guardrails go in before the system gets any write access, not after.
RAG to agentic: a migration outline
A RAG-based system in production with 300+ users on AWS Bedrock is real, credible ground for evolving toward an agentic layer on top: the retrieval layer becomes one tool among several, a planning layer decides when to retrieve vs. when to act, and guardrails go in before the system gets any write access.
What changes, in order
- Retrieval stops being the pipeline and becomes a tool with a schema, a description, and its own tests.
- A planning layer decides whether this request needs retrieval, an action, or neither.
- Guardrails and an approval gate go in before any write access is granted, not after.
- Tracing and a trajectory eval suite land in the same release as the planner.
State of the art
What is current as of mid-2026
Where agentic AI stands as of mid-2026 — autonomy horizons, benchmark progress, and the adoption gap still ahead.
Achievable autonomous task horizon — log scale
A straight line on a log axis is a doubling. METR’s tracking puts the achievable autonomous task horizon at roughly a doubling every seven months — about an hour in early 2025, multi-hour and heading toward full-workday sessions now. The new problems that creates are operational: graceful degradation when an eight-hour run fails at hour seven, and keeping token spend bounded across sessions running into the hundreds of thousands of tokens.
Scope
This reference covers the architecture and operational discipline for running agentic systems in production: pattern selection, tool design, memory, interoperability, evaluation, security, and governance. Sections 1–9 and 11–13 are cloud-agnostic; Section 10 is the one direct AWS/Azure/GCP comparison.
Before citing a figure
Compiled August 2026. Framework GA status, AWS region availability, OpenTelemetry GenAI conventions, the EU AI Act deferral, METR's horizon figures, and any adoption percentages here move fast — re-verify before treating a specific number as current.
© 2026 Vedant SenDWG NO. EAA-2026-01 · REV B
- 01
The center of gravity is shifting from hand-built orchestration toward native reasoning
As frontier models absorb more planning/reflection capability natively — extended or interleaved thinking woven into the model's own turn — hand-built ReAct/reflection scaffolding is becoming redundant in places. Engineering effort is moving up a level, toward how specialized agents communicate and are governed.
- 02
Long-horizon autonomy is the current frontier, and it is measured
METR's tracking of achievable autonomous task horizon shows it roughly doubling every seven months — from roughly one-hour tasks in early 2025 toward multi-hour, increasingly full-workday sessions. This introduces new problems: graceful degradation when an eight-hour run fails at hour seven, and keeping token spend under control across sessions running into the hundreds of thousands of tokens.
- 03
Benchmark progress is real, driven by scaffolding as much as raw capability
On OSWorld-Verified, early frontier models cleared under half of the human-operator baseline; by early 2026 the strongest agentic systems were approaching or matching it. The benchmark maintainers attribute most recent gains to better trajectory data and reasoning-enhanced scaffolding, not a fundamentally different base model.
- 04
Governance is a launch enabler now, not a launch blocker
Teams that bake permission boundaries, per-action decision logs, and human-approval checkpoints into the agent's design from day one are the ones getting past pilot into real production — not the ones bolting governance on after an incident.
- 05
The adoption picture
17% of organizations had deployed agents at the time of the survey, against 60%+ expecting to within two years, according to Gartner's 2026 CIO survey. Agentic AI is positioned at the Peak of Inflated Expectations on Gartner's 2026 Hype Cycle. Set against the 40%+ project-cancellation figure: capability is advancing quickly, while organizational readiness — governance, evaluation, and operational discipline — is lagging behind it.