New AI Models Make Decisions Without Words

New open decision models let AI agents make choices without generating text, significantly cutting token costs and latency for agentic systems.

AI agents often "think out loud" by generating text for their internal decision-making processes, a step that consumes significant computing resources, incurs high costs, and adds unnecessary latency. This internal monologue, where an agent formulates its next step in natural language, is a major bottleneck for building scalable systems. According to reporting from The New Stack, this inefficiency is drawing increasing scrutiny from across the industry. To solve this, a new family of open-source "decision models" called Kev has been released. These models are designed specifically to handle an agent's internal choices, allowing them to select the right tool or path without generating a single word of text, fundamentally re-architecting how these systems operate and targeting the wasteful overhead that even researchers at major AI labs have identified as a critical issue.

The core innovation behind Kev is its shift from conversational reasoning to structured, non-textual outputs for internal tasks. A typical AI agent might prompt a large language model with a question like, "Given the user's request, should I use the search tool or the calculator tool next?" It then has to wait for a full-sentence reply, such as "You should use the search tool," and parse this text to extract the actual decision. Kev models replace this entire inefficient loop. They are much smaller and specifically trained to understand the context of a decision and output a simple, machine-readable response, such as a single token or a numerical index that corresponds to a specific action. This completely bypasses the resource-intensive steps of text generation and parsing, resulting in a decision-making process that is orders of magnitude faster and cheaper.

This development is part of a broader industry trend focused on optimizing AI for practical, real-world deployment. As the initial novelty of massive LLMs gives way to the realities of operational costs, the focus is shifting towards efficiency, cost-effectiveness, and specialization. The Kev model is a prime example of a smaller, purpose-built tool designed to perform one function exceptionally well within a larger, more complex agentic system. This approach mirrors the evolution from monolithic software architectures to more nimble microservices and aligns with advanced concepts like Mixture of Experts (MoE), where different specialized models collaborate to solve a problem. By unbundling an agent's internal decision logic from its external language generation capabilities, developers can build more sophisticated, multi-step automated systems that were previously economically unviable.

For developers, founders, and CTOs, the practical takeaway is that relying on a single, giant LLM for every part of an agent's workflow is an increasingly outdated and expensive strategy. The release of open models like Kev provides a tangible, production-ready alternative for optimizing the internal mechanics of agentic systems. Teams building these applications should now evaluate where their agents' decision points can be offloaded to a more efficient, non-textual model, reserving the powerful but costly generative models for tasks that actually require natural language, such as summarizing information or communicating with a human user. Looking ahead, the key developments to watch will be the rate of adoption for these specialized decision models and whether incumbent AI platforms like OpenAI or Anthropic begin to offer similar low-cost, non-generative API endpoints for agent tool-use and internal reasoning.

For developers and engineers building AI agents, this offers a direct path to reducing operational costs and improving performance. By replacing text-based internal monologues with efficient decision models, agentic systems can become faster, cheaper, and more scalable, unlocking previously impractical use cases.

This innovation directly lowers the cost of running AI agent applications, making them more commercially viable. Companies can now build and deploy more complex automated workflows at scale, potentially creating new revenue streams or improving operational efficiency without incurring prohibitive token fees.