Why a Cheaper AI Can Double Your Bill
TL;DR: Alibaba's new Qwen4-Max model is cheaper per token than competitors like Claude. But independent tests show it generates twice the output, potentially doubling your final bill and revealing a key trade-off in AI model selection.
Key facts
- Category
- AI
- Impact
- High
- Published
- Source
- The New Stack
Full summary
Alibaba's new Qwen4-Max model is cheaper per token, but tests show it generates twice the output, potentially doubling your final bill.
A new large language model from Alibaba is highlighting a critical lesson for anyone building with AI: the sticker price isn't the whole story. According to reporting from The New Stack, independent testing by the firm Artificial Analysis found that Alibaba’s new Qwen4-Max model is exceptionally verbose. In a standard test suite, the model generated roughly twice the number of output tokens as the median for comparable models. While Qwen4-Max may boast a lower cost per token than rivals like Anthropic's Claude Opus, its tendency to be overly talkative can lead to significantly higher final costs for the same task. This analysis reveals a crucial trade-off between a model's price and its operational behavior, forcing teams to look beyond simple price charts when choosing an AI partner.
This discrepancy between per-token price and total cost stems from the fundamental business model of large language models. Companies typically pay for API access based on the number of tokens they process, counting both the input (your prompt) and the output (the model's response). A model with a low output token price seems like a bargain. However, if that model uses twice as many tokens to give you an answer, it can easily erase those initial savings. For example, a simple query that might elicit a 200-token response from one model could generate a 400-token response from a more verbose one like Qwen4-Max. This means developers could end up paying the same amount or even more, despite choosing the model with the lower advertised rate. This hidden cost of verbosity is a critical factor that can lead to unexpected budget overruns.
The findings fit into a broader trend of the AI market beginning to mature beyond simple leaderboard rankings and price-per-token comparisons. In the early days of the LLM race, performance on academic benchmarks and raw pricing were the primary differentiators. Now, as more companies deploy these models in real-world production environments, operational characteristics are becoming just as important. Factors like latency, consistency, and, as this case shows, verbosity directly impact both user experience and the total cost of ownership. This shift forces a more sophisticated evaluation process, where engineering and finance teams must collaborate to understand how a model's specific behaviors will affect their bottom line. It's no longer just about which model is smartest or cheapest on paper, but which is most efficient and cost-effective for a specific job.
For CTOs, developers, and founders, the practical takeaway is clear: you must benchmark models against your own use cases. Relying on third-party pricing lists or performance metrics alone is a recipe for financial surprises. The ideal model for a creative content generation task, where verbosity might be a feature, could be a costly mistake for a structured data extraction or a conversational chatbot application where concise answers are preferred. The key is to calculate the total cost per task, not the cost per token. As the AI landscape becomes more crowded and competitive, expect to see more nuanced analyses focusing on these operational efficiencies. The ability to conduct these internal tests and understand the true cost of an AI-powered feature will become a significant competitive advantage.
Why it matters
This analysis highlights a critical, often-overlooked metric in LLM evaluation: output verbosity. For developers and engineers, it proves that headline per-token pricing can be misleading. Real-world cost-effectiveness depends on how a model's behavior impacts total token consumption for a given task.
Business impact
Companies choosing an LLM based solely on low per-token costs risk significant budget overruns. The Qwen4-Max case demonstrates that a "cheaper" model can lead to higher operational expenses if its verbosity doubles output. This forces a more sophisticated TCO analysis for all AI investments.
Tags
Related on Notifire
Related stories
Primary source: The New Stack
