# Maintaining Long Conversations [Skip to content](#lm-inhoud)Network/[NL](/en/context-management)EN[Hubhub.llmnet.nlCompare models on task, language, cost and license.](https://hub.llmnet.nl/en/)[Communitycommunity.llmnet.nlPrompt techniques, patterns and system prompts.](https://community.llmnet.nl/en/)[APIapi.llmnet.nlLLMs in production: rate limits, routing, structured output.](https://api.llmnet.nl/en/)[Consultancyconsultancy.llmnet.nlRolling out AI in an organization, pilot to production.](https://consultancy.llmnet.nl/en/)[Newsnieuws.llmnet.nlAI developments, explained for the Netherlands.](https://nieuws.llmnet.nl/en/)[Benchmarkbenchmark.llmnet.nlMeasure AI quality yourself, on your own tasks.](https://benchmark.llmnet.nl/en/)[Careersvacatures.llmnet.nlAI roles, salaries and career paths in the Netherlands.](https://vacatures.llmnet.nl/en/)[Learnleren.llmnet.nlAI concepts in plain language, beginner to builder.](https://leren.llmnet.nl/en/)[Guidegids.llmnet.nlRun AI privately on your own Mac, PC, NAS or home server.](https://gids.llmnet.nl/en/)[Directorydirectory.llmnet.nlMapping the AI ecosystem: tools, models, companies.](https://directory.llmnet.nl/en/)[Radarradar.llmnet.nlSignals from X, research and communities for indie developers.](https://radar.llmnet.nl/en/)[Appsapps.llmnet.nlReviews of AI apps and open-source repos, with tips for builders.](https://apps.llmnet.nl/en/)[llmnet.nl — main site](https://llmnet.nl/en/)[](https://x.com/intent/post?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fcontext-management&text=Maintaining%20Long%20Conversations)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fcontext-management)[](https://www.reddit.com/submit?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fcontext-management&title=Maintaining%20Long%20Conversations)[](#)[](https://x.com/intent/post?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fcontext-management&text=Maintaining%20Long%20Conversations)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fcontext-management)[](https://www.reddit.com/submit?url=https%3A%2F%2Fcommunity.llmnet.nl%2Fen%2Fcontext-management&title=Maintaining%20Long%20Conversations)[](#)By Ivo Donker — created with AI assistance (Claude & Gemini) · Last updated: July 27, 2026 Developer & Prompt-Engineering Community # Maintaining Long Conversations: Context Window Management in Practice Published by llmnet.nl • Reading time: approx. 6 minutes Every developer working with Large Language Models (LLMs) runs into it sooner or later: the context window gets full. As a conversation or agent loop progresses, the prompt history grows exponentially. This not only leads to higher latency and increasing API costs, but also causes the model to lose track of relevant instructions (the infamous lost-in-the-middle phenomenon). In this article, we dive deep into practical techniques for context window management. No theoretical talk, but directly applicable strategies such as smart summarization, chunking, and state management for robust applications. ## Why the Context Window Crashes Many developers think that a model with a window of 128k or even 1 million tokens can go on indefinitely. In practice, however, the attention layer (attention mechanism) struggles to extract precise details as the noise in the context increases. In addition, with stateless APIs, you pay for the entire history with every request. Management is therefore twofold: cost savings and reliability. ## Technique 1: Rolling Summarization Instead of endlessly sending all raw chat messages, you maintain a compact status state or a rolling summary. As soon as the history exceeds a threshold (for example, 10,000 tokens), you trigger a background task that compresses the oldest messages. Practical Example: Keep the last 4 interactions in full for immediate context, and replace all previous interactions with a single updated paragraph containing decisions, choices made, and active variables. ## Technique 2: Chunking and Retrieval-Augmented Generation (RAG) With long documents or codebases, sending the entire source code in the prompt is impractical. This is where chunking comes in. You split large documents into manageable pieces (chunks) of, for example, 512 to 1024 tokens, store them in a vector database, and retrieve only what is relevant at that specific moment. Want to know more about how to set up and optimize RAG architectures for your own applications? Check out our comprehensive guide on the [llmnet.nl learning modules](https://leren.llmnet.nl/en/). ## Technique 3: What to Do When the Window is Full? Sometimes a conversation inevitably gets stuck because the user pastes a huge dataset. A robust architecture handles this with automatic degradation strategies: - Hard Truncation while retaining System Prompt: Immediately remove the oldest user-assistant pairs, but always retain the primary system prompt and the active task definition. - State Extraction: Just before a reset, ask the model to generate a JSON structure containing the current state of the application. Load this JSON into the clean, new session as a starting point. - Sliding Window with prioritization: Assign weights to messages. Critical code snippets or decisions get high priority and are retained longer than general greetings or minor corrections. ## Code Example: Simple Token Counter and Truncation Logic Below you can see a simple Python example demonstrating how to shorten messages based on an estimated token limit: def manage_context(messages, max_tokens=4000): current_tokens = sum(len(m["content"]) // 4 for m in messages) # If we are under the limit, no action is needed if current_tokens <= max_tokens: return messages # Always retain the system prompt (index 0) and the latest interaction system_prompt = messages[0] recent_history = messages[-2:] # Merge older messages or drop them in a controlled manner trimmed_messages = [system_prompt] + messages[1:-2] while sum(len(m["content"]) // 4 for m in trimmed_messages) > (max_tokens - 500): if len(trimmed_messages) > 3: trimmed_messages.pop(1) # Remove oldest non-system message else: break trimmed_messages.extend(recent_history) return trimmed_messages ## Conclusion Context window management is not an optional 'nice-to-have', but a fundamental part of modern LLM development. By summarizing smartly, extracting state, and relying on targeted chunking via RAG, you build applications that scale without performance or budgets exploding. © 2026 llmnet.nl • [Back to the main portal](https://llmnet.nl)