Generative AI security risks: the controls and who owns them
What changes when assistants and agents read your email and files: six generative AI security risks, the control that reduces each one, and who should own it.
Securing generative AI differs from securing other software mainly because instructions and data arrive through the same channel. An assistant that reads an email, a PDF or a web page can treat text inside it as an order, and then act with whatever access you gave it. So the generative AI security risks that matter to an organisation sit less in the model and more in three places you control: what the assistant can read, what it can do, and what you record about both.
We recommend that organisations rolling out assistants and agents fix access first: scope the connectors, cut agent permissions and turn on logging before they buy filters or detection tools. Filters help, but the OWASP Top 10 for LLM Applications 2026 (August 2026) keeps prompt injection at number one and tells readers to build for the day it succeeds. Permissions decide how much damage that day does. The trade-off is real: narrower access makes assistants less useful in the first weeks, and you widen it as evidence comes in.
What is actually different about generative AI security?
Your existing controls still apply: identity, device management, patching, vendor review. Generative AI adds four new conditions on top of them.
- Content becomes instructions. A traditional application keeps code and data apart. A language model reads both as text, so every document, ticket, email and web page an assistant opens is input that someone outside your organisation may have written.
- The assistant acts with borrowed access. A connector to email, file shares or a CRM gives the assistant the permissions of the user or of a service account. Folders that were over-shared years ago can become searchable in seconds.
- Output varies and can trigger actions. The same request can produce different answers. When an answer is passed to a tool, a wrong answer becomes a wrong action.
- Old logs miss the new questions. Your logs may show that a file was sent. They may not show which prompt, which retrieved document and which approval led an agent to send it.
Six generative AI security risks for organisations that use assistants and agents
We group the risks by where they show up in daily use, and give the matching entry in the OWASP 2026 list where one exists (logging gaps have no entry of their own), so your security team can trace each one to the full guidance.
1. Prompt injection (LLM01:2026)
Someone hides instructions in content the assistant will read: a supplier email, a CV, a shared document, a web page. The assistant may follow them. Direct injection comes from the user; indirect injection comes from content the assistant retrieves, and it is the harder case for organisations because no one in the conversation wrote it.
2. Data leakage through prompts and connectors (LLM02:2026 Sensitive Information Disclosure)
Leakage runs two ways. Staff paste client files or salary data into tools the organisation never approved, which our guide to shadow AI discovery covers in detail. And approved assistants surface what their connectors can reach, so an HR folder shared with "everyone" years ago can appear in an answer to a junior employee.
3. Excessive agency (LLM03:2026)
An agent has more tools, permissions or freedom than its task needs: it can send email to anyone, approve refunds of any size, or delete records. OWASP moved this risk up to third place in 2026 because, in its words, "agentic deployments are where the damage is landing". Our article on AI agent governance and guardrails sets out autonomy levels for this.
4. Improper output handling (LLM10:2026)
Model output goes straight into another system without checks: a web page, a database query, an email template, or code that a developer merges without reading. The 2026 list widened this entry to cover insecure code that assistants generate at scale.
5. Model, plugin and connector supply chain (LLM04:2026 Supply Chain)
Your AI depends on parts you did not build: hosted models, open model files, connectors, MCP servers, browser extensions and third-party tool packages. Any of them can be tampered with or simply poorly built. OWASP's supply chain entry covers third-party models and tool packages; NIST lists the same concern as "Value Chain and Component Integration".
6. Logging gaps and hidden context
If you cannot reconstruct what an agent read, decided and did, you cannot investigate an incident or prove a control worked. A related failure is placing passwords, keys or business rules in system prompts: OWASP's LLM08:2026 Hidden Context Exposure tells developers to assume that hidden context is discoverable and never to store credentials in it.
Risk-to-control table: what reduces each risk and who owns it
The table maps each of the generative AI security risks above to a control and an owner. Owners are roles. In a mid-sized company one person may hold three of them; what matters is that each control has a name next to it.
| Risk | Control that reduces it | Who owns it |
|---|---|---|
| Prompt injection | Treat all retrieved content as untrusted; an assistant that reads outside content gets read-only tools; human confirmation before any action that sends, pays or deletes | Security team sets the rule; process owner approves actions |
| Data leakage through prompts | Approved tools with enterprise terms; data rules in the acceptable use policy; DLP on browser and endpoints | Data protection officer with IT |
| Data leakage through connectors | Fix file and mailbox permissions before connecting; connect named sites, not whole tenants; review connector scope every quarter | IT platform admin; data owner of each source |
| Excessive agency | Least privilege per tool; spending and volume limits; per-action approval for irreversible steps; a tested way to stop the agent | Business process owner with the platform team |
| Improper output handling | Validate output like user input before it reaches another system; code review for generated code | Application or engineering lead |
| Supply chain | Register of approved models, connectors and extensions; source and version check before install; vendor review | Security team with procurement |
| Logging gaps and hidden context | Log prompts, retrieved sources, tool calls and approvals; send to your security monitoring; no secrets in prompts; retention set by data rules | Security operations; data protection officer for retention |
Two rules sit behind the whole table. Permission checks live in code or in the target system, never in the model's judgement; OWASP calls this complete mediation. And every control above should be testable, so your review can show it works rather than assume it.
Two illustrative examples from Jordan and the GCC
A financial services company in Amman (illustrative). About 400 staff, an enterprise assistant connected to email and the document store. Before connecting, the IT team lists every folder shared with all staff and finds board papers and payroll exports among them. The company fixes those permissions first, connects three departments, and keeps the rest for a second phase. The rollout is slower, and the first answers are narrower, but payroll exports are no longer one chat question away from every employee.
A logistics firm in Riyadh (illustrative). A WhatsApp customer service agent handles delivery complaints in Arabic and English. Customer messages are untrusted input, so the agent can look up orders and issue store credit up to a fixed amount; any cash refund goes to a human. This follows the graduated approach OWASP describes, which uses a similar store-credit example, where recoverable actions run on their own and irreversible ones go to human review. Each conversation, lookup and credit is logged with the order number, so a disputed refund can be traced back to the message that caused it. For how our agentic AI service sets limits, approvals and audit logs for agents like this, see our agentic AI use case.
Which frameworks should you map your controls to?
Use one list for daily work and two for depth, rather than five overlapping ones.
- OWASP Top 10 for LLM Applications 2026 for the risk names your engineers and vendors will recognise. It succeeds the 2025 edition. OWASP published it on 3 August 2026, but its main LLM Top 10 page still showed the 2025 list when we checked on 19 October 2026, so confirm which edition a vendor's risk IDs refer to: LLM03 is Supply Chain in 2025 and Excessive Agency in 2026. For agents with tools and memory, OWASP points to its companion Top 10 for Agentic Applications (December 2025).
- NIST AI 600-1, the Generative AI Profile (July 2024) for governance. It lists 12 risks that generative AI creates or makes worse, including Information Security and Data Privacy, with suggested actions. It is voluntary guidance, not a law.
- MITRE ATLAS for your security team's threat modelling: a public knowledge base of attacker tactics and techniques against AI systems.
Where prompts or connectors carry personal data, the data protection law and sector rules that apply to your organisation also shape retention, transfer and access decisions. This article is general information, not legal advice; obligations depend on your sector and your data.
How to start
Start with an inventory of every assistant and agent in use, what each one connects to and what it can do. That list alone shows where the table above has empty owner cells. Our AI Security Assessment and Monitoring service reviews assistants and agents for prompt injection, data exposure and permissions, with an audit and usage dashboard. If you also need the policy, decision rights and risk register around them, the AI Governance and Compliance Foundation covers that, and both sit in our Governance Lab.
If you are not sure where to begin, the AI Exposure Scan is a free 45-minute conversation that ends in a written next step.
Frequently asked questions
What are the main generative AI security risks for an organisation?
For an organisation that uses assistants and agents, the main risks are prompt injection, data leakage through prompts and connectors, agents with more permissions than their task needs, model output passed to other systems without checks, untrusted models and connectors in the supply chain, and logs that cannot show what an agent read, decided and did. The OWASP Top 10 for LLM Applications 2026 names most of these.
Can prompt injection be fully prevented?
No current control removes it completely: OWASP's 2026 list says no reliable prevention exists today, keeps prompt injection at number one and advises building for the day it succeeds. Filters reduce it. Limiting what an assistant can do after reading outside content, requiring human confirmation for actions that send, pay or delete, and checking permissions outside the model limit the damage when an injection gets through.
Who should own generative AI security in a company?
No single team. The security team sets rules for untrusted content, logging and the supply chain. The data protection officer owns data rules and log retention. IT platform admins own connector scope. Business process owners approve what agents may do in their workflow. What matters is that every control has a named person next to it, even when one person holds several roles.
How is securing an AI agent different from securing a chatbot?
A chatbot that only answers can leak information or give wrong answers. An agent can also act: send email, issue refunds, change records. That adds excessive agency to the risk list. Agents need least privilege per tool, limits on amounts and volumes, approval for irreversible steps, a tested way to stop them, and logs of every tool call and approval.
Get new insights by email
One email when we publish something useful. No spam; leave any time.
Services in this article
Related lab: Governance Lab ↗
