Deterministic Prompt Pruner Cuts LLM Context Overhead, Preserves Dependencies
Independent developer Emmimal has introduced a deterministic prompt-pruning layer designed to mitigate token bloat in long-running Large Language Model conversations. As multi-turn agent sessions accumulate stale tool outputs, duplicate retrieval chunks, and irrelevant history, prompts expand into append-only logs that increase inference costs and degrade model reasoning performance. Conventional truncation methods fail in production environments because they silently discard implicit dependencies required by subsequent turns. To address this, Emmimal engineered a three-pass pruning pipeline that operates entirely on standard Python libraries, eliminating reliance on external embeddings or LLM calls to guarantee predictable, reproducible output. The architecture executes three sequential functions before prompt compilation. The first pass identifies and removes expired context, such as superseded tool calls sharing identical keys. The second pass eliminates duplicate retrieved documents by normalizing whitespace and tracking prior occurrences. Crucially, the third pass implements dependency restoration by scanning for explicit DEFINE and REF markers, ensuring that no later turn loses access to a required fact simply because it was flagged for removal in earlier passes. During development, the author identified a critical testing oversight where the dependency restoration logic remained entirely untested due to synthetic data constraints. Correcting this by allowing tool outputs to define dependencies revealed that the system actively preserves critical context across varying conversation lengths. Rigorous benchmarking validated the pipeline across fifteen configurations spanning plain chat, RAG-assisted workflows, and tool-heavy agent operations at five distinct conversation sizes. Results demonstrate workload-dependent token reduction ranging from two to four percent for standard dialogue to twenty-seven to thirty-four percent for retrieval-augmented and heavily tool-dependent applications. Across all test parameters, the system preserved one hundred percent of labeled required facts and achieved idempotent behavior, meaning repeated execution yields identical state without compounding data loss. Preprocessing overhead remained consistent and below fifty milliseconds even when processing 131,000 tokens across two thousand turns. While the implementation prioritizes absolute determinism and zero external dependencies, it acknowledges inherent limitations. Dependency detection relies on literal identifier matching rather than semantic analysis, meaning paraphrased references without explicit tags will not trigger restoration. The developer notes that future iterations should integrate the deterministic layer as a foundational safety stage preceding semantic compression tools. Additionally, validating the architecture against actual production telemetry rather than synthetic benchmarks remains a priority to capture real-world traffic patterns. By treating prompt expansion as a structured memory management challenge, the pruning layer offers a reliable, cost-effective mechanism for maintaining long-context integrity. The complete source code, benchmark harness, and validation tests are publicly available for independent verification and production integration.
