Securing agentic AI: a threat model and reference architecture

A traditional application does what its code says. An agent decides what to do next. It reads a goal, plans steps, calls tools and APIs, observes the result, and repeats — often with access to data, systems, and money. That decision loop is enormously useful, and it is also the reason agents cannot be secured like ordinary software.

This article lays out a threat model for agentic systems and a reference architecture that keeps the useful autonomy while containing the blast radius.

Why agents change the threat model

Classic AppSec assumes a fixed set of code paths an attacker tries to subvert. An agent generates its own path at runtime from untrusted input. Three properties make this hard:

• Instructions and data share a channel. A retrieved document, a tool result, or a web page can carry text the model treats as a new instruction.
• Actions are real. Tool calls send email, move files, run queries, and spend budget. A bad decision is not a log line — it is an effect.
• State compounds. Memory, scratchpads, and multi-step plans mean a single poisoned input can steer many later steps.

The agentic attack surface

It helps to enumerate where things go wrong rather than argue about a single "AI vulnerability."

Where agents get attacked

Prompt & context injection

Direct or indirect instructions that override the agent's goal, exfiltrate context, or trigger tools.

Tool & API abuse

Over-broad tool scopes let a hijacked agent read or mutate systems far beyond its task.

Memory & data poisoning

Malicious content written to long-term memory or a knowledge base influences future runs.

Excessive agency

Autonomy without approval gates turns a reasoning error into an irreversible action.

A reference architecture

The pattern that holds up in production is a mediated agent: the model never touches privileged systems directly. Everything routes through a controlled layer that authenticates, authorizes, validates, and logs.

Layers of a governed agent

Governance & evidence

Policy, audit log, framework mapping

Guardrails & policy engine

Input/output filters, allow-lists, approvals

Tool broker

Scoped credentials, schema validation, rate limits

Agent runtime

Planner, memory, orchestration

Model

LLM inference endpoint

The key inversion: capability lives in the tool broker, not the model. The agent can only request an action; the broker decides whether that request is allowed, with what credentials, and whether a human must approve it.

Controls that attach to the loop

Map defenses to the agent's actual lifecycle so nothing is left implicit.

Controls across the agent loop

1. Ingest

   Sanitize & tag untrusted content

   →
2. Plan

   Constrain goals; deny risky intents

   →
3. Act

   Broker validates & scopes tool calls

   →
4. Observe

   Filter tool output before re-entry

   →
5. Log

   Immutable trace for audit & replay

Practical rules that follow from this:

1. Treat every tool result as untrusted input on its way back into context — the same way you treat the initial prompt.
2. Scope credentials per task, not per agent. A support agent resolving one ticket should not hold standing write access to the whole datastore.
3. Gate irreversible or high-value actions (payments, deletes, external sends) behind explicit approval or a policy check.
4. Bound autonomy with step limits, budgets, and timeouts so a runaway loop fails closed.

Governance and evaluation

Controls are only credible if you can prove they work. Evaluate agents against safety and security criteria before launch, monitor them in production for anomalies and drift, and keep an audit trail mapped to the frameworks you answer to — the NIST AI RMF and the OWASP LLM Top 10 give you a shared, defensible vocabulary.

Autonomy is worth having. The job is to keep the reasoning free while making the actions accountable.

References

1. 1.OWASP Top 10 for LLM Applications
2. 2.MITRE ATLAS — adversarial threat landscape for AI systems
3. 3.NIST AI Risk Management Framework