Skip to content

LLM06: Add structural authority separation mitigation #801

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
devongenerally-png wants to merge 1 commit into
base: main
Choose a base branch
from
+5 −0
Open

LLM06: Add structural authority separation mitigation #801

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions 2_0_vulns/LLM06_ExcessiveAgency.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,10 @@ The following actions can prevent Excessive Agency:

Follow secure coding best practice, such as applying OWASP’s recommendations in ASVS (Application Security Verification Standard), with a particularly strong focus on input sanitisation. Use Static Application Security Testing (SAST) and Dynamic and Interactive application testing (DAST, IAST) in development pipelines.

#### 9. Enforce structural authority separation

Interpose a deterministic policy evaluation layer between the LLM agent's tool selection and tool execution. Rather than relying on the LLM to self-govern which actions are appropriate, route every proposed tool call through a non-LLM policy engine that evaluates a structured intent (tool name, arguments, calling context) against deployer-defined rules and returns an approve/deny/escalate verdict. This ensures the entity that proposes an action is never the same entity that authorizes it — the agent operates in user-space while authorization is a system-level concern. Log every verdict to a tamper-evident audit trail for post-incident analysis.

The following options will not prevent Excessive Agency, but can limit the level of damage caused:

* Log and monitor the activity of LLM extensions and downstream systems to identify where undesirable actions are taking place, and respond accordingly.
Expand All @@ -104,3 +108,4 @@ Alternatively, the damage caused could be reduced by implementing rate limiting
4. [NeMo-Guardrails: Interface guidelines](https://github.com/NVIDIA/NeMo-Guardrails/blob/main/docs/security/guidelines.md): **NVIDIA Github**
5. [Simon Willison: Dual LLM Pattern](https://simonwillison.net/2023/Apr/25/dual-llm-pattern/): **Simon Willison**
6. [Sandboxing Agentic AI Workflows with WebAssembly](https://developer.nvidia.com/blog/sandboxing-agentic-ai-workflows-with-webassembly/) **NVIDIA, Joe Lucas**
7. [Governance-Guard: Structural Authority Separation for AI Agents](https://github.com/MetaCortex-Dynamics/governance-guard): **MetaCortex Dynamics**