A language model gets access to confidential data or internal systems and has to stand up to an audit. I make sure it only sees what it may see, and only does what it may do.

When you need AI security

A chatbot or assistant answers questions from internal documents (RAG).
An agent may call tools: send emails, create tickets, query databases, run code.
Models are connected to internal systems through MCP servers.
The application has to pass a review by internal audit, data protection or a regulator before go-live.
Customers or partners ask how the AI feature is secured.

The typical risks

Prompt injection

A language model does not reliably tell instructions from content. A hidden instruction in an email, a web page or a document can steer the model. That becomes dangerous as soon as the model can use tools or send data out.

Data poisoning

Whoever can put content into the knowledge base of a RAG pipeline influences the answers. I check where the data comes from, who may change it and how manipulated content gets noticed.

Missing permission checks at retrieval

The most common mistake: the search finds every document, and the model shows it to anyone who asks. Permissions have to be checked at retrieval time, with the rights of the person asking, not those of the system.

Overly broad tool access

An agent with write access to everything is a privileged account that anyone can steer with a text input. I limit tools and MCP access to what is needed and add confirmations for consequential actions.

Approach

Threat model: Which data, tools and user groups exist, and where are the trust boundaries?
Review: Architecture, prompts, retrieval, permissions, logging and runtime.
Targeted tests: Prompt injection and data exfiltration scenarios against the agreed attack surfaces.
Hardening: Prioritised measures, implemented with your team if you wish.
Report: Findings, proof that the measures work and a closing review.

Experience

I have worked in software engineering for over 20 years and have secured AI applications up to the high-security zone of a bank. If you also want the application built or extended: AI enablement and engineering are on my sister site 0x68.de.

Related services: IT security & IAM for the identities and permissions beneath the application, IT forensics when something has already happened.

AI security for LLM and agent applications