Skip to main content

Glossary P

Glossary · Cross-cutting

Prompt injection

Area
Both strands

Prompt injection is a vulnerability in which inputs or injected content change the behaviour or output of a language model in unintended ways.

According to the OWASP Gen AI Security Project, manipulated input can take effect even when nobody can perceive it. The BSI's study on generative AI models distinguishes two routes. In the direct route, users modify their input to bypass safety mechanisms or make the model break out of its predefined role. Depending on where those mechanisms sit, the study calls this jailbreaking or prompt injection. In the indirect route, the manipulation hides in unchecked third-party sources such as websites or documents.

According to the study, if a language model application has execution and access capabilities and can act autonomously on the model's output, unauthorised actions can follow. Examples are unwanted purchases or e-mails sent and deleted. Such an application could be an AI agent with access to orders or a mailbox. According to the BSI's 2025 report, there is currently no single highly effective measure against indirect prompt injection, so several are combined in practice. The generative AI study names one option: making critical actions depend on the user's explicit consent, for example via a confirmation button.

The AI agents we build produce drafts and previews; nothing is published until a person has approved it. In a knowledge platform we built for pharma logistics, the chat assistant's own access key reaches only content released to all signed-in users.

Where the term appears

Sources

  1. LLM01:2025 Prompt Injection (opens in a new tab)

    OWASP Gen AI Security Project (OWASP Foundation) accessed

  2. Generative KI-Modelle: Chancen und Risiken für Industrie und Behörden, Version 2.0 vom 17.01.2025 (opens in a new tab)

    Bundesamt für Sicherheit in der Informationstechnik (BSI) accessed

  3. Die Lage der IT-Sicherheit in Deutschland, Kapitel Resiliente Gestaltung von LLM-Systemen (opens in a new tab)

    Bundesamt für Sicherheit in der Informationstechnik (BSI) accessed

Share on LinkedIn

Let's talk about your project.

If this term is on your desk right now, a conversation usually helps faster than any definition.

Search the website

Popular topics

All results