# Dynamic Few-Shot Selection with Vector Context

[Skip to content](#lm-inhoud)Network/[NL](/en/dynamische-few-shot-selectie-met-vector-context)EN[Hubhub.llmnet.nlCompare models on task, language, cost and license.](https://hub.llmnet.nl/en/)[Communitycommunity.llmnet.nlPrompt techniques, patterns and system prompts.](https://community.llmnet.nl/en/)[APIapi.llmnet.nlLLMs in production: rate limits, routing, structured output.](https://api.llmnet.nl/en/)[Consultancyconsultancy.llmnet.nlRolling out AI in an organization, pilot to production.](https://consultancy.llmnet.nl/en/)[Newsnieuws.llmnet.nlAI developments, explained for the Netherlands.](https://nieuws.llmnet.nl/en/)[Benchmarkbenchmark.llmnet.nlMeasure AI quality yourself, on your own tasks.](https://benchmark.llmnet.nl/en/)[Careersvacatures.llmnet.nlAI roles, salaries and career paths in the Netherlands.](https://vacatures.llmnet.nl/en/)[Learnleren.llmnet.nlAI concepts in plain language, beginner to builder.](https://leren.llmnet.nl/en/)[Guidegids.llmnet.nlRun AI privately on your own Mac, PC, NAS or home server.](https://gids.llmnet.nl/en/)[Directorydirectory.llmnet.nlMapping the AI ecosystem: tools, models, companies.](https://directory.llmnet.nl/en/)[Radarradar.llmnet.nlSignals from X, research and communities for indie developers.](https://radar.llmnet.nl/en/)[Appsapps.llmnet.nlReviews of AI apps and open-source repos, with tips for builders.](https://apps.llmnet.nl/en/)[llmnet.nl — main site](https://llmnet.nl/en/)[](https://x.com/intent/post?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fdynamische-few-shot-selectie-met-vector-context&text=Dynamic%20Few-Shot%20Selection%20with%20Vector%20Context)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fdynamische-few-shot-selectie-met-vector-context)[](https://www.reddit.com/submit?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fdynamische-few-shot-selectie-met-vector-context&title=Dynamic%20Few-Shot%20Selection%20with%20Vector%20Context)[](#)[](https://x.com/intent/post?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fdynamische-few-shot-selectie-met-vector-context&text=Dynamic%20Few-Shot%20Selection%20with%20Vector%20Context)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fdynamische-few-shot-selectie-met-vector-context)[](https://www.reddit.com/submit?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fdynamische-few-shot-selectie-met-vector-context&title=Dynamic%20Few-Shot%20Selection%20with%20Vector%20Context)[](#)

 
# Dynamic Few-Shot Selection with Vector Context

 By Ivo Donker — compiled with AI assistance (Claude & Gemini)

 Static prompts with fixed demonstrations quickly hit their limits in complex production applications. When an application handles dozens of different intents, edge cases, or input structures, a fixed set of three or four examples rarely provides optimal guidance for every specific question. Dynamic few-shot selection solves this bottleneck by retrieving relevant demonstrations in real time from a vector store based on the semantic context of the incoming query. Instead of a trade-off between breadth and depth, the model gets served exactly the examples that match the task in terms of syntax, domain logic, and boundary conditions.

 In this article, we look at the architecture behind dynamic few-shot selection. We examine how vector embeddings are used to index demonstrations, how similarity matching and diversity selection balance each other, and how you build these components into a robust pipeline. The structural downsides, such as latency overhead, embedding noise, and context pollution, are also analyzed in detail.

 
## The limitations of static few-shot prompts

 Anyone who hardcodes examples directly into a system prompt inevitably makes a trade-off between context budget and representativeness. Anyone who wants to revisit the fundamentals of [how examples drive in-context learning](https://community.llmnet.nl/en/few-shot-prompting)knows that a handful of demonstrations often dictates the reasoning path and the desired output format. In a diverse production domain, however, a fixed set of demonstrations falls short as soon as a user raises a rare edge case that isn't covered by those fixed examples.

 Simply raising the number of static examples to ten or twenty causes three structural complications. First, the fixed token cost of every API call rises linearly. Second, a phenomenon occurs where earlier or intermediate demonstrations receive less attention from the model's attention mechanism. Third, there's a risk of over-steering: irrelevant examples can introduce patterns that the model blindly copies onto inputs where those patterns don't apply at all.

 When the decision needs to be made about the right approach, consulting the guide on [the trade-off between zero-shot, few-shot, and chain-of-thought](https://community.llmnet.nl/en/prompttechniek-kiezen) offers a clear framework for when demonstrations are even necessary in the first place. Dynamic selection forms the logical evolution once a fixed prompt falls short but a full fine-tuning process is too rigid or too expensive.

 
## Architecture of dynamic vector retrieval

 A dynamic few-shot pipeline consists of three functional layers: the storage layer (a database of validated input-output pairs), the embedding and indexing layer, and the prompt assembly layer. When a query enters the runtime, an embedding model generates a vector representation of the user input. The vector store then computes the cosine similarity or dot-product distance between the query vector and the indexed input examples.

 Crucial to the architecture is exactly what gets embedded. A common mistake is embedding the full input-output pair. Because the incoming query only contains the input, embedding the output in the index leads to asymmetric vectors. After all, the embedding model then ends up searching for similarities between a standalone question and a combination of question plus answer. Best practice is to generate vectors based solely on the input-key, while the corresponding output and optional metadata are stored as payload.

 
 
 
 
 Component | 
 Task in Pipeline | 
 Point of Attention | 
 

 
 
 
 Example Bank | 
 Validated JSON documents with input, reasoning, and output | 
 Requires strict schema validation and periodic quality audits | 
 

 
 Embedding Service | 
 Converting text query into a d-dimensional vector | 
 Must be identical to the model that built the index | 
 

 
 Vector Store | 
 Performing K-Nearest Neighbor (k-NN) or HNSW searches | 
 Index tuning for low latency at high QPS | 
 

 
 Prompt Assembler | 
 Injecting top-k matches into the template | 
 Validating total token limit before API dispatch | 
 

 
 
 

 The performance of this search system depends directly on the representational quality of the chosen vectors. The overview evaluating [models for semantic search and RAG](https://hub.llmnet.nl/en/embeddingmodellen-vergeleken)makes clear that small differences in dimension size and domain training have a major impact on the ability to distinguish between subtly different query structures.

 
## Selection strategies: cosine similarity versus diversity

 The simplest selection method is retrieving the k examples with the highest cosine similarity. While this works excellently for simple classifications, pure similarity creates a redundancy problem for more complex tasks. If the user's question closely resembles a cluster of three nearly identical examples in the database, the model receives the exact same demonstration three times. This wastes context space and denies the model the chance to observe broader edge cases.

 To counter redundancy, advanced pipelines combine semantic relevance with diversity selection. A proven method for this is Maximal Marginal Relevance (MMR). MMR iteratively computes the score of a candidate example based on two factors: similarity to the user query and dissimilarity to already-selected examples. The formula balances these with a parameter $\lambda$:

 By setting $\lambda$ to, say, 0.7, you prioritize relevance while still preventing duplicate examples from being added. An alternative method is k-means clustering in the embedding space, where one representative example is chosen from each relevant cluster.

 Structurally assembling these dynamic prompt components fits seamlessly with the principle of modular systems. See how [prompts are built modularly from separate building blocks](https://community.llmnet.nl/en/prompt-modulariteit-herbruikbare-componenten) to see how dynamic injection sections stay neatly separated from static system instructions.

 
## Implementation example in Python

 The implementation below demonstrates a complete pipeline that uses an in-memory vector store, cosine similarity, and automated prompt assembly. The code uses typed data structures and ensures that only the input fields get vectorized.

import math
from dataclasses import dataclass
from typing import List, Dict, Any

@dataclass
class Example:
 id: str
 input_text: str
 output_text: str
 metadata: Dict[str, Any]
 vector: List[float] = None

def dot_product(v1: List[float], v2: List[float]) -> float:
 return sum(a * b for a, b in zip(v1, v2))

def magnitude(v: List[float]) -> float:
 return math.sqrt(sum(a * a for a in v))

def cosine_similarity(v1: List[float], v2: List[float]) -> float:
 mag1 = magnitude(v1)
 mag2 = magnitude(v2)
 if mag1 == 0.0 or mag2 == 0.0:
 return 0.0
 return dot_product(v1, v2) / (mag1 * mag2)

class DynamicFewShotSelector:
 def __init__(self, examples: List[Example]):
 self.examples = examples

 def select_top_k(self, query_vector: List[float], k: int = 3) -> List[Example]:
 scored = []
 for ex in self.examples:
 sim = cosine_similarity(query_vector, ex.vector)
 scored.append((sim, ex))
 
 # Sorteer aflopend op similariteitsscore
 scored.sort(key=lambda item: item[0], reverse=True)
 return [item[1] for item in scored[:k]]

 def assemble_prompt(self, system_instruction: str, 
 query_text: str, 
 query_vector: List[float], 
 k: int = 3) -> str:
 selected_examples = self.select_top_k(query_vector, k=k)
 
 prompt_parts = [system_instruction, "\n--- VOORBEELDEN ---"]
 for idx, ex in enumerate(selected_examples, 1):
 prompt_parts.append(f"\n[Voorbeeld {idx}]")
 prompt_parts.append(f"Invoer: {ex.input_text}")
 prompt_parts.append(f"Uitvoer: {ex.output_text}")
 
 prompt_parts.append("\n--- HUIDIGE TAAK ---")
 prompt_parts.append(f"Invoer: {query_text}")
 prompt_parts.append("Uitvoer:")
 
 return "\n".join(prompt_parts)

 In a production environment, you'd replace the in-memory functions with a specialized database client (such as Qdrant, Milvus, or pgvector) and an asynchronous embedding call. The underlying selection pattern stays identical.

 
## Integration within Context Engineering

 Dynamic few-shot selection is not an isolated trick; it's an integral part of a broader data and context regime. Where the initial prompt engineering focused on static wording and sentence structure, context engineering operationalizes the entire context space as a dynamically assembled runtime environment. Anyone who wants to understand why the shift toward [context engineering and dynamic injections](https://leren.llmnet.nl/en/context-engineering-uitgelegd) has become necessary will see few-shot retrieval as the archetype of this transition.

 Within this engineering discipline, order matters a great deal. When examples are inserted into the prompt, the position of the demonstrations directly affects the model's behavior. Empirical observations show that models give disproportionate weight to the last example placed directly before the user input (recency bias). If the most relevant example is placed first and two less relevant examples follow it, the effectiveness of the most relevant demonstration can get diluted.

 A best practice for integration is therefore reversing the selection list: place the candidate example with the highest similarity score as the last demonstration, right before the actual question. This way, the most accurate reference point acts as a direct 'springboard' for the model's generation process.

 
## Pitfalls, risks, and context management

 While dynamic few-shot selection can significantly increase the accuracy of LLMs, it also introduces specific failure modes that need to be actively monitored and mitigated.

 
 
### Key operational risks

 1. Semantic interference (false positives): A question can look lexically very similar to an example but have an opposite semantic intent. Injecting a syntactically similar but substantively incorrect demonstration sends the model straight down the wrong path.

 2. Latency stack: Generating a query embedding and querying the vector store adds between 30 and 150 milliseconds to time-to-first-token (TTFT). For real-time interactions, this step must be strictly asynchronous or tightly cached.

 3. Outlier distortion: When the user input falls completely outside the domain of the example bank, the vector store still selects the 'least bad' k matches. These forced matches can actively drive hallucinations.

 

 To prevent outlier distortion, a minimum similarity threshold should be set. If no example in the database reaches a cosine similarity of at least 0.75, the system automatically falls back to a zero-shot prompt or a fixed generic fallback instruction. This prevents the model from being 'poisoned' with irrelevant patterns.

 In addition, the total token volume needs to be tightly managed. Uncontrolled injection of long examples can crowd out other vital context parts. For methods to guard the overall context space and prevent degradation, the article on [context window management in practice](https://community.llmnet.nl/en/context-management) offers valuable strategies for context compaction and budgeting.

 
## Evaluation and continuous curation of the example bank

 A dynamic few-shot system is only as reliable as the dataset it rests on. Maintaining the example bank requires a structured curation process. In a mature development environment, examples aren't added manually and ad hoc but instead go through a pipeline with automated validations.

 The lifecycle of a dynamic demonstration set consists of four steps:

 1. Gap detection: Production logs are analyzed for questions where the top-1 similarity score stayed below the cutoff, or where validation errors occurred in the model output. These are the indicators of missing knowledge in the vector store.

 2. Synthesis and validation: Concrete input-output pairs are drafted for new scenarios. These pairs are checked by human domain experts or automated evaluation models for factual accuracy, format consistency, and the absence of superfluous tokens.

 3. Regression testing: Before a new example is added to the live vector store, a test suite runs against a benchmark set. This prevents the new example from unintentionally 'hijacking' earlier, well-functioning queries through an overly broad embedding.

 4. Deduplication and pruning: Examples that are rarely hit, or that have nearly identical vectors to newer, better examples, get archived to keep search time low and the index sharp.

 
## Conclusion and implementation checklist

 Dynamic few-shot selection transforms in-context learning from a static configuration into an adaptive runtime layer. By treating examples as structured data that gets retrieved dynamically, LLM applications can scale to complex domains without prompts becoming unmanageably long or expensive.

 
 
 
 
 Phase | 
 Checkpoint | 
 Status | 
 

 
 
 
 Data | 
 Only the input text is vectorized (not the output) | 
 Required | 
 

 
 Retrieval | 
 Similarity threshold set as fallback to zero-shot | 
 Required | 
 

 
 Diversity | 
 MMR or clustering applied to filter out duplicate examples | 
 Recommended | 
 

 
 Prompting | 
 Most relevant example placed last (recency optimization) | 
 Recommended | 
 

 
 Monitoring | 
 Logging similarity scores and retrieval latency | 
 Required | 
 

 
 
 

 By structurally building these checks into the orchestration layer, you get a resilient system that consistently generates high-quality responses, regardless of how varied the incoming requests are.