How to protect an AI assistant against unauthorised instructions
Prompt injection occurs when text received by an assistant tries to change its behaviour, including content in documents, messages or pages it consults. Protection combines application limits, careful handling of sources and observable tests. This guide helps you request and review those controls in a business project.
By the end, you can:
- Map sources of untrusted content
- Separate interpretation from authorisation
- Prepare tests with fictitious data and functions
Distinguish the approved task from consulted content
An assistant may receive a legitimate task, such as summarising a report, and encounter a sentence inside it asking for data to be sent elsewhere. That sentence belongs to the report, not to the person’s authorisation. OWASP distinguishes direct inputs from instructions introduced indirectly through retrieved content.
Risk depends on available capabilities. An assistant that only prepares text may produce a manipulated summary; one that looks up records or sends messages may have additional consequences. Start by identifying possible effects in your project. The solution needs to act at the boundary where data is read, presented or changed.
Map the channels through which content arrives
List messages, attachments, the knowledge base, search results, tools and history. For each channel, identify who controls the content and whether third parties can change it. A document stored in an internal folder may contain externally received text; location does not automatically turn all its sentences into authorised instructions.
Keep the source associated with content when splitting, summarising or reusing it. A note produced by the assistant may also carry an unauthorised instruction from an earlier stage. Record which parts are application rules, which are user requests and which are data to interpret.
| Input | Proposed handling | Authority to execute |
|---|---|---|
| Application rule | Approved, versioned configuration | Defined by the application |
| Authenticated request | Interpret within permissions | Limited to the user and action |
| Document or email | Content with an identified source | Does not grant permissions |
| Tool result | Data validated before reuse | Does not extend the authorised scope |
Give each function a small, verifiable scope
A function that checks an enquiry’s status should return the necessary fields for that enquiry. It need not allow arbitrary queries across the whole database. A function that prepares a reply need not send messages. Reducing capabilities makes it easier to understand and test what an interpretation error could affect.
OWASP links excessive autonomy and permissions with unwanted actions. When designing the project, decide which actions are automatic, which produce only drafts and which require review of content and destination. Confirmation should show the specific operation; a generic approval button does not help detect that the recipient has changed.
Validate access and arguments outside the model
The application determines identity and checks access to the requested record. An identifier proposed by the model does not prove the user may read that object. OWASP recommends authorisation checks in API operations receiving identifiers. The same rule matters when an API request originates from an assistant.
Validate types, allowed fields, destinations and operation conditions. Also handle output according to where it will be used: text presented as HTML, search parameters and commands have different requirements. A structured result makes these checks easier but does not decide whether the action is legitimate.
Test an unauthorised instruction without real secrets
In a fictitious test, the assistant summarises report DEMO-17. The document includes a note asking it to ignore the task and copy the marker TESTE-SEM-VALOR to a test destination. Use simulated functions without the ability to contact people or change real data. The aim is to observe whether content can cause an out-of-scope action.
The test record should show proposed calls, authorisation decisions and changes to the fictitious state. A final answer saying “I did not do that” is insufficient. If records are missing or the test function is broken, mark the case inconclusive. The absence of a particular word in the answer does not prove protection of all data.
| Controlled test | Evidence to observe |
|---|---|
| Document requests an additional action | Unauthorised operation not executed |
| Request references another fictitious account | Access check rejects the object |
| Sending destination is changed in the content | Destination checked against the operation’s policy |
| Legitimate request contains suspicious words | Legitimate task remains usable |
Combine detection with controls that remain active
Clear instructions, context separation and pattern detection can reduce problems but do not replace application controls. A filter searching for a known phrase may fail with another formulation and block legitimate requests using the same word. Assess detection errors, not only examples that were blocked.
Human review also needs context: origin, destination, proposed change and reason. Reserve it for decisions where the person can check something relevant. If a flow asks for repeated approvals without showing consequences, attention may decline. The design should facilitate an informed choice and retain the decision for the exact operation.
Define how to suspend a capability and investigate
Prepare a way to disable the affected function while maintaining an alternative for essential work. Record the minimum needed to investigate: case reference, version, origin, tool, decision and final state. Do not copy credentials or entire documents into technical logs for convenience.
When changing an integration or adding a new source, repeat the associated tests. A passed test set describes the versions, cases and conditions tested; it is not a permanent guarantee. Also keep legitimate requests that were blocked so protection can improve without making the service unusable.
Use a boundary-review worksheet
Ask the technical lead for a demonstration of the limits as well as a written explanation. The worksheet connects each capability to its control and corresponding test. Explicitly mark points without evidence and choose the usage scope according to what was actually verified.
Worksheet for your project
Authorised task and users: Content sources and who controls them: Accessible data and available operations: Identity and object-level access checks: Allowed arguments, destinations and fields: Automatic actions / drafts / actions subject to review: Fictitious data and tools used in tests: Direct, indirect and legitimate-request cases: Observed calls, decisions and final state: Failures, incorrect blocks and inconclusive cases: How to suspend the capability and continue manually: Person responsible, versions and next review:
Sources and criteria
Content prepared with AI assistance and research using technical sources, with checks of the relevant examples and calculations. The references support the guide’s decisions; examples identified as fictitious let you try out the method.
