# Context grounding: limiting hallucinations with prompts

[Skip to content](#lm-inhoud)Network/[NL](/en/voorkomen-hallucinaties-context-grounding-prompts)EN[Hubhub.llmnet.nlCompare models on task, language, cost and licence.](https://hub.llmnet.nl/en/)[Communitycommunity.llmnet.nlPrompt techniques, patterns and system prompts.](https://community.llmnet.nl/en/)[APIapi.llmnet.nlLLMs in production: rate limits, routing, structured output.](https://api.llmnet.nl/en/)[Consultancyconsultancy.llmnet.nlRolling out AI in an organisation, pilot to production.](https://consultancy.llmnet.nl/en/)[Newsnieuws.llmnet.nlAI developments, explained for the Netherlands.](https://nieuws.llmnet.nl/en/)[Benchmarkbenchmark.llmnet.nlMeasure AI quality yourself, on your own tasks.](https://benchmark.llmnet.nl/en/)[Careersvacatures.llmnet.nlAI roles, salaries and career paths in the Netherlands.](https://vacatures.llmnet.nl/en/)[Learnleren.llmnet.nlAI concepts in plain language, beginner to builder.](https://leren.llmnet.nl/en/)[Guidegids.llmnet.nlRun AI privately on your own Mac, PC, NAS or home server.](https://gids.llmnet.nl/en/)[Directorydirectory.llmnet.nlMapping the AI ecosystem: tools, models, companies.](https://directory.llmnet.nl/en/)[Radarradar.llmnet.nlSignals from X, research and communities for indie developers.](https://radar.llmnet.nl/en/)[Appsapps.llmnet.nlReviews of AI apps and open-source repos, with tips for builders.](https://apps.llmnet.nl/en/)[llmnet.nl — main site](https://llmnet.nl/en/)[](https://x.com/intent/post?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fvoorkomen-hallucinaties-context-grounding-prompts&text=Context%20grounding%3A%20limiting%20hallucinations%20with%20prompts)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fvoorkomen-hallucinaties-context-grounding-prompts)[](https://www.reddit.com/submit?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fvoorkomen-hallucinaties-context-grounding-prompts&title=Context%20grounding%3A%20limiting%20hallucinations%20with%20prompts)[](#)[](https://x.com/intent/post?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fvoorkomen-hallucinaties-context-grounding-prompts&text=Context%20grounding%3A%20limiting%20hallucinations%20with%20prompts)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fvoorkomen-hallucinaties-context-grounding-prompts)[](https://www.reddit.com/submit?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fvoorkomen-hallucinaties-context-grounding-prompts&title=Context%20grounding%3A%20limiting%20hallucinations%20with%20prompts)[](#)

# Limiting hallucinations with context grounding

By Ivo Donker — compiled with AI support (Claude & Gemini) · Last updated: 6 August 2026

## What you can and cannot solve with the prompt

The emergence of incorrect or fabricated claims in language models, also known as hallucinations, has several causes. For developers and prompt engineers, it's crucial to distinguish between two types of hallucinations: parametric errors and contextual derailments. To understand why this distinction is fundamental, we refer to the theoretical analysis on [understanding hallucinations](https://leren.llmnet.nl/en/hallucinaties-begrijpen).

Parametric hallucinations occur when a language model consults its internal weights (the memory built up during pre-training) and fallibly combines outdated, incomplete, or distorted facts found there. When you ask a model to retrieve general knowledge without providing source text, you can influence this behavior through prompts only indirectly at best. After all, you're asking the model to draw from a probabilistic store of patterns.

Contextual hallucinations, by contrast, arise in situations where you explicitly provide a model with documents, data, or snippets to answer a question. If the model makes statements in the generated text that cannot be demonstrably traced back to the provided context, that's a prompt and grounding problem. This type of hallucination can indeed be tackled directly through structured prompt engineering and context grounding.

Context grounding is the process by which the model is strictly anchored to the provided sources. The goal is to switch off the model's parametric memory for factual claims and have the model function purely as a processor, editor, and formulator of the provided context.

## The basic pattern for grounded prompts

The fundamental pattern of a grounded prompt rests on a clear separation of roles and boundaries. A standard grounded prompt consists of three basic components: the system instruction with behavioral rules, the explicitly delimited source context, and the user's question.

In the basic pattern, you instruct the model that the provided sources are the only permitted source of truth. The instruction forces the model to disregard any assumption that falls outside the provided text. This prevents the model from filling logical gaps with assumptions from its pre-training.

[SYSTEEMINSTRUCTIE]
Je bent een feitelijke assistent. Beantwoord de vraag van de gebruiker UITSLUITEND op basis van de onderstaande brondocumenten.
Gebruik geen externe kennis of aannames die niet expliciet in de brontekst worden genoemd.

[BRONDOCUMENTEN]
--- BRON 1 ---
De maximale verwerkingstijd voor aanvragen bedraagt vijf werkdagen. 

--- BRON 2 ---
Spoedaanvragen worden binnen 24 uur verwerkt, mits ingediend voor 12:00 uur.

[GEBRUIKERSVRAAG]
Wat is de verwerkingstijd voor een standaardaanvraag?

This basic pattern enforces a framework, but without additional measures, there's a good chance the model will still guess when in doubt. Adding an explicit escape hatch is therefore an indispensable part of the instruction.

## The escape hatch: the most important sentence in your prompt

Language models are trained via Reinforcement Learning from Human Feedback (RLHF) to answer as usefully and helpfully as possible. This eagerness-to-please behavior is the primary driver behind contextual hallucinations. When the provided source text doesn't contain the answer to a question, or only partially contains it, the model will tend toward extrapolation: it guesses the most likely answer just so it can give a response.

The most important rule in a grounded prompt is therefore the explicitly prescribed escape hatch (also called the fallback clause or abstention instruction ). You must explicitly give the model permission to say that the answer isn't in the sources, and you must also prescribe exactly what that refusal should look like.

Without an escape hatch: The model experiences the instruction as a task that must be completed at all costs. If the source doesn't contain the answer, guessing is the only option left.

With an escape hatch: The model gets a defined escape route. Recognizing a lack of information thereby becomes a correct, approved outcome.

An effective wording for this escape hatch is, for example:

Als de meegegeven brondocumenten onvoldoende informatie bevatten om de vraag volledig te beantwoorden, antwoord dan exact: "Op basis van de verstrekte bronnen kan deze vraag niet worden beantwoord." Vul geen ontbrekende informatie aan met eigen kennis.

Adding this sentence transforms the model's decision process. The answer "I don't know based on the sources" gets a higher probability value than a speculative answer, because the model is explicitly rewarded for recognizing the boundaries of the context.

## Source delimiting and mandatory citations

To enforce context grounding and make it verifiable, source texts must be presented in a structured and recognizable way. Loose chunks of text placed in the prompt without clear boundaries blur, for the model, the line between instruction and data.

It's wise to give sources unique identifiers (such as [DOC-1], [DOC-2] or XML tags <source id="1">) and requiring the model to reference the specific fragment for every factual statement. An effective technique for increasing accuracy is requiring direct quotes.

[INSTRUCTIE]
Beantwoord de vraag. Onderbouw elke feitelijke bewering in je antwoord met een direct citaat en de bijbehorende bron-ID. Formatteer je antwoord als volgt:
- Bewering [Bron ID]: "Exact geciteerde tekst"

[BRONNEN]
<doc id="A1">De garantieperiode op elektronica bedraagt 24 maanden vanaf de aankoopdatum.</doc>
<doc id="A2">Schade door waterschade valt buiten de standaard garantiedekking.</doc>

[VRAAG]
Geldt de garantie als de telefoon in het water is gevallen?

Why does mandatory citation make the behavior measurable? When a model is forced to first copy the exact text fragment before drawing a conclusion, a form of Chain-of-Thoughtreasoning kicks in. The model first processes its grasp of the source in its output tokens, which reduces the chance of misinterpretation. On top of that, this makes the answer programmatically verifiable.

## The pitfall of fake citations and sampling checks

Requiring source references offers no absolute guarantee. In practice, developers often run into the pitfall of the 'fake citation' (or misattribution). This happens when a language model cites a literally correct quote from the source but attaches a conclusion or claim to it that isn't supported by that quote.

A model might, for example, cite: "The building is open from Monday through Friday", and then go on to claim: "The building is closed on public holidays [DOC-1]". The quote is accurate, but the conclusion drawn about public holidays appears nowhere in the source. The model uses the quote as a kind of decoration to lend authority to a hallucinated claim.

To detect and prevent fake citations, it's necessary to regularly sample the generated answers. You can set up an automated check or human evaluation that verifies two criteria:

- Is the cited quote literally present in the provided source text? (Substring matching)

- Does the cited quote logically imply the claim being made about it? (NLI - Natural Language Inference)

If a sample reveals that the claim and the citation diverge, that's an indication that the prompt instruction needs tightening, or that the logical leap being asked of the model is too large.

## Structure, order, and separation of instructions and sources

The way information is ordered in the prompt has a major influence on how closely the model sticks to the context. Language models tend to give more weight to information placed at the beginning or end of the prompt (the so-called primacy and recency biases).

A proven effective order for grounded prompts is:

- System role and strict behavioral rules: Define what the model may and may not do, including the escape hatch.

- Contextual data / source documents: Present the sources in a structured way with clear boundaries.

- User question and final instruction: Place the user's question right at the bottom, followed by a repetition of the format.

Clearly marking the user's question also prevents the model from confusing the content of the source text with instructions. When source text is entered unstructured, a malicious user or an unsafe source text can try to bypass the model's rules. Although this overlaps with the security aspects from the guide on [defending against prompt injection](https://community.llmnet.nl/en/prompt-injection-verdedigen), a clear separation between instructions, sources, and questions also serves a purely functional purpose here: it prevents the model from 'executing' source text instead of summarizing or analyzing it.

## Handling conflicting sources

In complex applications, such as searching through multiple company documents or knowledge systems, it regularly happens that sources contradict each other. Think of an old policy document from 2022 and a more recent memo from 2025 that list different reimbursement amounts.

If a prompt contains no specific instruction for contradictions, an LLM behaves unpredictably. It randomly picks one of the two sources, tries to invent a compromise that appears in neither source, or gets confused and generates an incoherent answer.

You need to explicitly equip the prompt with a protocol for conflicting information:

[INSTRUCTIE VOOR TEGENSTRIJDIGHEDEN]
Als de verstrekte bronnen elkaar tegenspreken op een punt dat nodig is om de vraag te beantwoorden:
1. Kies NIET zelf welke bron juist is.
2. Benoem expliciet de tegenstrijdigheid.
3. Citeer beide bronnen met hun respectievelijke identifiers.

Voorbeeld van een correct antwoord bij tegenstrijdigheid:
"De bronnen spreken elkaar tegen. Volgens [BRON-A] bedraagt de opzegtermijn één maand, terwijl [BRON-B] vermeldt dat de opzegtermijn twee maanden bedraagt."

By instructing the model to report contradictions rather than resolve them, you prevent the AI from creating a false sense of certainty that is factually wrong.

## Context size: why less is often more

A common misconception is that enlarging the context window automatically leads to better answers. Adding dozens of pages of documents in the hope that the model will find the right answer often backfires in practice.

When a prompt is overloaded with irrelevant material, the effect known as lost in the middleoccurs. Models process information at the core of a large context less accurately than information at the beginning or end. Moreover, noise increases the chance of faulty assimilation: the model draws connections between unrelated paragraphs and hallucinates conclusions.

For optimal grounding results: pre-select only the most relevant passages. Applying tight filtering, as discussed in the article on [context management](https://community.llmnet.nl/en/context-management), is essential for preserving precision. If it's necessary to process large volumes of documents, it's advisable to first condense the texts with specific [prompts for summarization](https://community.llmnet.nl/en/prompts-voor-samenvatten) before offering them as context to the final grounding prompt.

## Second-line verification: the extra check step

For critical applications — such as medical, legal, or financial assistants — a single prompt call is often not enough to guarantee a zero-tolerance level for hallucinations. In such situations, a second-line verification process (a verifier pipeline) can be deployed.

In this pattern, a first model call (the Generator) generates an answer based on the grounded prompt. Next, a second, independent model call (the Verifier) is carried out. The Verifier gets one very specific task: it receives only the original sources and the generated answer, and must check whether every claim is factually supported.

[VERIFIER PROMPT]
Je bent een onafhankelijke auditor. Je taak is het valideren van een gegenereerd antwoord tegen de brontekst.

BRONTEKST:
{oorspronkelijke_bronnen}

GEDENEREERD ANTWOORD:
{antwoord_van_generator}

OPDRACHT:
Controleer elke bewering in het gegenereerde antwoord. 
Geef als output 'GOEDGEKEURD' als alle bewering 100% gedekt zijn door de brontekst.
Als er ook maar één bewering staat die niet in de brontekst staat, geef dan 'AFGEKEURD' met een lijst van niet-onderbouwde beweringen.

This extra check step naturally comes with additional latency and API costs. The decision to deploy second-line verification depends on the application's risk profile. For customer-service chatbots handling general questions, the extra cost often doesn't outweigh the benefit. For automated contract analyses or policy comparisons, on the other hand, this second check is a worthwhile investment to prevent faulty information disclosure.

## Measuring whether it works: evaluation on a fixed test set

Prompt engineering for context grounding is an iterative process. A change to the instructions that reduces hallucinations in scenario A can unintentionally lead to an increase in errors in scenario B (for example, because the model becomes too cautious and refuses to answer questions that are actually in the source).

To determine whether a prompt change is actually effective, you need to measure against a fixed test set (an evaluation benchmark). This set should consist of three types of test cases:

- Directly answerable questions: Questions whose answer is literally and unambiguously present in the context.

- Unanswerable questions: Questions that seem related to the topic, but whose facts have deliberately been left out of the context. Test whether the escape hatch is correctly triggered.

- Misleading / conflicting questions: Questions that assume something false, or where the sources contradict each other.

The primary metric you track here is the ungrounded claim ratio: the percentage of generated answers that contain at least one factual claim not supported by the provided context. For a deeper dive into setting up these measurement methods and quality checks, you can consult the articles on [measuring hallucinations](https://benchmark.llmnet.nl/en/hallucinaties-meten) and setting up a structured [RAG evaluation](https://benchmark.llmnet.nl/en/rag-evaluatie).

## Comparison of grounding techniques

The table below gives an overview of the various grounding techniques, their mechanism, and the associated trade-offs when implementing them.

Technique | 
Mechanism | 
Advantage | 
Point of Attention | 

Explicit escape hatch | 
Allowing the model to answer "unknown" when sources are missing. | 
Prevents the model from filling gaps by guessing. | 
Model may give up too quickly if the wording is too strict. | 

Mandatory citation | 
Requiring the model to show the source fragment for every claim. | 
Makes the answer directly verifiable and auditable. | 
Risk of fake citations that don't support the claim. | 

Conflict protocol | 
Instruction to flag contradictory sources. | 
Prevents the AI from arbitrarily picking a source. | 
Requires the model to fully analyze both sources. | 

Second-line verification | 
A second LLM call validates whether the context supports the claims. | 
Maximum reduction of unfounded claims. | 
Significantly increases processing time and API costs. | 

## Summary guidelines for developers

Limiting hallucinations through context grounding is one of the most effective methods for building reliable LLM applications. By consciously bounding the model's parametric knowledge, giving the model an explicit escape hatch for missing information, strictly structuring sources, and continuously evaluating the process on a fixed test set, you bring processing quality to a manageable and controllable level.

## Further reading

- [Context management for developers](https://community.llmnet.nl/en/context-management)

- [Defending against prompt injection in production](https://community.llmnet.nl/en/prompt-injection-verdedigen)

- [Prompts for summarizing large documents](https://community.llmnet.nl/en/prompts-voor-samenvatten)

- [Understanding hallucinations: causes and mechanisms](https://leren.llmnet.nl/en/hallucinaties-begrijpen)

- [Measuring hallucinations in language models](https://benchmark.llmnet.nl/en/hallucinaties-meten)

- [RAG evaluation: benchmarks and metrics](https://benchmark.llmnet.nl/en/rag-evaluatie)

llmnet.nl - prompt engineering and developer community