Building a Secure Internal AI Assistant

How an internal assistant usually works

Most knowledge assistants use retrieval-augmented generation (RAG). Connectors pull content from document libraries, wikis and ticketing systems; the content is split into passages and indexed for search; when someone asks a question, the most relevant passages are retrieved and sent to a language model with instructions to answer from them and cite sources. Some assistants can also take actions through tools, which raises the stakes considerably.

Where the model runs

Whichever option you choose, confirm in writing whether prompts and documents are retained, for how long, where they are processed and whether they are used to train models.

Permission-aware retrieval

• Synchronize access control lists from each source system and filter search results by the signed-in user's identity at query time.
• Re-synchronize promptly when permissions change or people leave.
• Test with accounts from different roles and confirm that restricted documents never appear in answers or citations.
• Clean up oversharing first. An assistant makes documents that were technically accessible but hard to find very easy to find.

Decide what goes in

• Start with sources that are useful, well maintained and not highly sensitive, such as policies, product documentation and support knowledge.
• Exclude or tightly restrict sources such as HR case files, medical information, payroll and legal matters unless there is a clear, approved purpose.
• Remove outdated and duplicate documents so the assistant does not quote superseded policies.
• Where documents contain personal data, apply the Data Privacy Act's principles of transparency, legitimate purpose and proportionality, and consider a privacy impact assessment. NPC Advisory No. 2024-04 explains how these obligations apply to AI systems.

Prompt injection and unsafe output

The OWASP Top 10 for LLM Applications lists prompt injection, sensitive information disclosure, improper output handling and excessive agency among the main risks. Practical defenses include:

• Treat retrieved text as data. A document that says "ignore previous instructions" must not change what the assistant is allowed to do.
• Give the assistant as few tools as possible, and require a person to confirm any action that changes data or sends messages.
• Do not put secrets or credentials in system prompts.
• Sanitize model output before rendering it, and block automatic loading of external images or links that could leak data.
• Limit request rates and response sizes to control cost and abuse.

Answer quality

• Instruct the model to answer only from retrieved sources and to say when it does not know.
• Show citations that link to the source documents so users can check.
• Build an evaluation set of real questions with expected answers and sources.
• Measure whether the right passages are retrieved, whether answers are correct and grounded, and whether the assistant declines appropriately.
• Re-run the evaluation whenever you change the model, prompts, chunking or sources.

Access, logging and monitoring

• Require single sign-on with MFA.
• Log questions, retrieved documents and answers for security and quality review, with a defined retention period and restricted access.
• Watch for unusual volumes, repeated attempts to extract restricted content and cost spikes.

Rollout and governance

1. Pilot with a defined group and a small set of sources.
2. Publish an acceptable use guide: what to ask, what not to paste in, and that people remain responsible for decisions.
3. Add a feedback button and review flagged answers every week during the pilot.
4. Name an owner who approves new sources and tools.
5. Expand gradually as evaluation results and feedback allow.

NIST's Generative AI Profile of the AI Risk Management Framework offers a structured list of risks and actions for organizations deploying systems like this.