New 1.5B Parameter Model, Arch-Router, Achieves 93% Accuracy in Preference-Aligned LLM Routing Without Retraining

Researchers at Katanemo Labs have unveiled Arch-Router, a novel routing model and framework that intelligently maps user queries to the most suitable large language model (LLM) without the need for costly retraining. Announced on July 7, 2025, this development addresses a critical challenge faced by enterprises that use multiple LLMs in their products, ensuring that each query is directed to the best model for the specific task at hand. The Challenges of LLM Routing As the number of LLMs continues to grow, developers are increasingly transitioning from single-model setups to multi-model systems. These systems leverage the unique capabilities of each LLM for specific tasks, such as code generation, text summarization, and image editing. However, this shift introduces significant challenges in routing user queries efficiently and accurately. Existing Routing Methods: - Task-Based Routing: Directs queries based on predefined tasks but struggles with ambiguous or evolving user intentions, especially in multi-turn conversations. - Performance-Based Routing: Optimizes for cost and performance metrics but often neglects real-world user preferences and requires expensive retraining to adapt to new models. Introducing Arch-Router Katanemo Labs proposes a "preference-aligned routing" framework that aligns with subjective human preferences, offers transparency, and remains adaptable as models and use cases evolve. This framework uses a "Domain-Action Taxonomy," a two-level hierarchy where the domain represents a broad topic (e.g., legal, finance) and the action specifies a particular task (e.g., summarization, code generation). Two-Stage Routing Process: 1. Policy Selection: A preference-aligned router model, Arch-Router, takes the user query and a set of policies defined in natural language, selecting the most appropriate policy. 2. Model Mapping: A mapping function links the chosen policy to the corresponding LLM. Key Features of Arch-Router Arch-Router is a compact 1.5B parameter language model fine-tuned for preference-aligned routing. By including the routing policies as part of the input, the system can adapt to new or modified routes during inference via in-context learning, eliminating the need for retraining. The model's efficiency is maintained even with longer policies, as its latency is primarily determined by the short identifiers of the selected policies. Testing and Performance The researchers fine-tuned Arch-Router on a curated dataset of 43,000 examples and evaluated its performance against state-of-the-art proprietary models from OpenAI, Anthropic, and Google. Results from four public datasets designed to assess conversational AI systems showed that Arch-Router achieved an overall routing accuracy of 93.17%, outperforming all other models by an average of 7.71%. Its advantage was particularly pronounced in multi-turn conversations, indicating its strong ability to maintain context over extended interactions. Real-World Applications Arch-Router is already finding practical applications in various domains. In open-source coding tools, it directs different stages of the development workflow to the most suitable LLMs. For example, tasks such as "code design," "code understanding," and "code generation" can be routed to the most effective models for each step. Similarly, enterprises can route document creation requests to one LLM and image editing tasks to another, enhancing the overall user experience. For personal assistants, Arch-Router helps manage diverse tasks like text summarization and factoid queries, making the interaction feel seamless and cohesive to the end user. The integration with Arch, Katanemo Labs' AI-native proxy server, enables developers to implement and test sophisticated traffic-shaping rules for new LLMs, ensuring a smooth transition and continuous optimization. Industry Insights and Company Profile Salman Paracha, the founder and CEO of Katanemo Labs, emphasizes that Arch-Router and the broader Arch framework aim to move beyond fragmented LLM implementations. “Our goal is to unify and improve the overall user experience, especially in scenarios where user tasks are diverse,” he said. Katanemo Labs is further integrating its tools with evaluation platforms to streamline the process for enterprise developers. The success of Arch-Router underscores the growing importance of flexible and preference-aligned AI architectures. With large tech companies and startups alike racing to develop more sophisticated LLMs, solutions like Arch-Router will be crucial for maximizing the effectiveness and efficiency of AI systems in real-world applications. Evaluation by Industry Insiders Industry experts have praised Arch-Router for its innovative approach to LLM routing. They note that its ability to adapt to user-defined preferences and maintain high performance without retraining represents a significant advancement in the field. This framework could help bridge the gap between academic benchmarks and practical user experiences, enabling more intuitive and flexible AI deployments in a wide range of industries. Katanemo Labs, known for its cutting-edge research in AI infrastructure, continues to push the boundaries of what is possible with language models and routing systems. The company’s focus on practical, user-centric solutions positions it as a leader in the rapidly evolving AI landscape.