Hallucination

Germany's Federal Office for Information Security (BSI) covers hallucinations in its study on generative AI models (in German). Many models invent content when they process inputs on current topics. Hallucinations are a major problem with large language models Large language model A large language model (LLM) is an AI model, trained on large amounts of text, that processes text inputs called prompts and generates text outputs from them. Read in the glossary → , because the outputs usually look credible, especially when they cite references that may themselves be entirely made up.

According to the BSI, giving a model a knowledge base through RAG RAG Retrieval-augmented generation (RAG) is a method that lets a generative AI model draw on documents held in a knowledge base and use them to enrich its output. Read in the glossary → is a common way to avoid hallucinations, and showing users the text excerpts behind an answer can mitigate the effects, though not eliminate them. Where the impact may be critical, outputs should be checked and, if required, edited by hand before further use.

Checks are built in from the start, because we assume an AI agent AI agent An AI agent is a software system, built on a large language model, that carries out actions on its own and uses interfaces to other programs to reach its goal. Read in the glossary → will get something wrong sooner or later. Take the knowledge platform we built for a pharmaceutical logistics Pharma logistics Pharma logistics is the term we use for storing, transporting and delivering medicinal products and other pharma-sector goods, such as promotional material and supplies for patients. Read in the glossary → company: the model behind its analysis software must not invent or guess, and where abbreviations, business context or technical details are unclear, it has to ask. The platform's interface Interface An interface is the point at which two systems, programs or components meet and exchange data; the word also covers the user interface between a person and software. Read in the glossary → , not the model, assigns values such as the next free case number. In one research pipeline, a second agent on a different model checks the work. A statement counts as a fact only once a script has found the supporting quotation word for word in the source.