Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
## Risk/Vuln Name
**Data poisoning**

**Author(s):**
Javier Dominguez

### Description
Data poisoning is the act of compromising the data an agent it relies on to solve the tasks. Although the data used to train the model can be poisoned, this item targets solely the data retrieved at runtime (RAG).

This poisoning can be either direct, by accessing the database where the information is stored, or indirect, by tricking a system to add to the database the poisoned data.

**This is partially covered by ASI01 Memory Poisoning**, as that item targets the same type of attack at memory level.


### Common Examples of Risk
1. Misinformation and data manipulation, as the model will answer questions based on the information available in the database.
2. Prompt manipulation, specially if the agent retrieves the system prompt from a database.
3. Direction to phishing or malicious sites added by the attacker in the database

### Prevention and Mitigation Strategies
1. **Ensure the integrity of the data** and review its content to detect anomalies.
2. **Implement strong controls around the database** to prevent an attacker to have unauthorised access to the data.
3. **Implementing continuous monitoring and auditing** to detect and respond to unusual or unauthorized access to the database.


### Example Attack Scenarios

- **Scenario 1: Phishing campaign (indirect)** – An agent reads content from an external website to recommend hotels. An attacker changes the content of that website with a phishing url, making the agent to show this malicious url to the users.
- **Scenario 2: Prompt injection** – An attacker replaces the dataset containing the system prompt in the database with its own instructions, modifying the behaviour of the agent.
- **Scenario 3: Misinformation** – An agent learns from the documents uploaded by the users. An attacker uploads a massive number of documents with misinformation, making the agent to believe this is true.



### Reference Links
1. [SEO Poisoning: How Threat Actors Are Tricking AI Models like ChatGPT, Gemini, and CoPilot](https://www.zerofox.com/blog/seo-poisoning-llms/)
2. [Microsoft Copilot: From Prompt Injection to Exfiltration of Personal Information](https://embracethered.com/blog/posts/2024/m365-copilot-prompt-injection-tool-invocation-and-data-exfil-using-ascii-smuggling/)
3. [Zero to hired. A CTF where the challenge is to upload a CV to the system to trigger a prompt injection] (https://github.com/javixeneize/zero-to-hired)
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
## Risk/Vuln Name
**Over-permissioning**

**Author(s):**
Javier Dominguez

### Description
Over-permissioning refers to assigning more permissions than necessary to an agent within an identity and access management system, resulting in unintended access rights or exposures. This violates the principle of least privilege by allowing agents more access than necessary, increasing the risk that they may access sensitive data or perform unauthorized actions.

**This complements ASI03 Privilege compromise**, as that item defines how the vulnerability defined here can be exploited.

The threat is partially covered by [LLM06:2025 Excessive Agency](https://genai.owasp.org/llmrisk/llm062025-excessive-agency/)



### Common Examples of Risk
1. Unauthorized privilege escalation to assume roles the agent is not supposed to.
2. Lateral movement to different environments.
3. Unauthorized resource creation, leading to creation of backdoors in the system.
4. Data exfiltration, if the agent has access to sensitive data.

### Prevention and Mitigation Strategies
1. **Assign granular privileges to the agent** only to the services needed, using the least privilege principle.
2. **Implement network isolation** to prevent a rogue agent from accessing other systems.
3. **Implement guardrails** to ensure the agents operates within defined boundaries.
4. **Implementing continuous monitoring and auditing** of agent activities and access patterns to detect and respond to unusual or unauthorized use.


### Example Attack Scenarios

- **Scenario 1: Access to sensitive information** – An agent aimed to analyse logs in an application is given full read permission, and can access any single log in the system.
- **Scenario 2: Data exfiltration** – An agent is given root access in the system, and is tricked to return the internal secrets, as passwords or keys.
- **Scenario 3: Create backdoor** – An agent is given write access in the system, and is tricked to create a backdoor to get remote access into the system.



### Reference Links
1. [Agentic AI - Threats and Mitigations](https://genai.owasp.org/resource/agentic-ai-threats-and-mitigations/)
2. [LLM06:2025 Excessive Agency](https://genai.owasp.org/llm-top-10/LLM06-excessive-agency)
3. [The 'Root Permissions' Problem: Why Agentic AI Poses Unique Data Security Risks] (https://www.rsaconference.com/library/blog/the-root-permissions-problem-why-agentic-ai-poses-unique-data-security-risks)
4. [Hacker Plants Computer 'Wiping' Commands in Amazon's AI Coding Agent](https://www.404media.co/hacker-plants-computer-wiping-commands-in-amazons-ai-coding-agent/)
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,14 @@ These risks are compounded in **heterogeneous, distributed agent environments**,
5. **Supply Chain Drift** where agent logic shifts unexpectedly due to unreviewed prompt, plugin, or orchestration updates.

### Prevention and Mitigation Strategies
1. **Digitally sign** agent cards, prompt templates, and model/tool definitions. Use verifiable SBOMs for agents and their runtime components.
1. **Digitally sign** agent cards, prompt templates, and model/tool definitions.
2. **Apply version control and peer review** for prompt/script repositories and memory definitions, just as for code.
3. **Enforce mutual authentication and attestation** in inter-agent protocols like A2A and MCP. Avoid open agent registration without validation.
4. **Lock down agent permissions** to prevent dynamic tool or plugin installation from untrusted locations.
5. **Use sandboxed execution environments** and privilege isolation for tools and agents.
6. **Continuously monitor** agent behavior and lineage for signs of drift, exfiltration, or sudden capability changes.
7. **Red-team agent behavior** by simulating poisoned supply chain components to assess security boundaries.
8. **Generate AIBOMs** to have a complete inventory of the components included in your AI system.

### Example Attack Scenarios

Expand Down Expand Up @@ -55,4 +56,6 @@ Researchers identify a server-side template injection (SSTI) bug in AutoGPT’s
3. https://www.trustwave.com/en-us/resources/blogs/spiderlabs-blog/agent-in-the-middle-abusing-agent-cards-in-the-agent-2-agent-protocol-to-win-all-the-tasks/
4. https://www.theregister.com/2025/07/21/vibe_coding_replit_ai_data_loss/
5. https://foraisec.medium.com/autogpt-remote-code-execution-ssti
6. https://blog.christianposta.com/understanding-mcp-and-a2a-attack-vectors-for-ai-agents/
6. https://blog.christianposta.com/understanding-mcp-and-a2a-attack-vectors-for-ai-agents/
7. https://github.com/sigstore/model-transparency
8. https://www.wiz.io/academy/ai-bom-ai-bill-of-materials