Cognition CEO Scott Wu Criticizes Token Leaderboards, Urges Output Focus
Prominent artificial intelligence executives are warning against the growing trend of evaluating software engineers based on artificial intelligence token consumption, arguing that such leaderboards incentivize wasteful usage rather than actual productivity. Scott Wu, chief executive of the San Francisco-based AI coding startup Cognition, recently highlighted the flaw in this approach during a podcast appearance. While acknowledging that tracking AI tool usage is directionally sound, Wu cautioned that companies frequently misapply the metric by rewarding engineers who burn the most tokens. He advocated for outcome-driven evaluations, such as tracking resolved engineering tickets, accelerated project timelines, and reduced development costs. Wu noted that if AI enables teams to triple their output, the compute expense remains justified, provided the rewards align with tangible results rather than raw usage. Wu’s critique aligns with broader industry pushback against what has been termed tokenmaxxing. Earlier this month, Legora chief technology officer Jacob Lauritzen warned that incorporating token counts into performance reviews encourages employees to artificially inflate usage to appear productive. Similarly, Andrew Feldman, CEO of Cerebras Systems, dismissed the practice of granting developers unlimited AI tokens during a Bloomberg conference, comparing it to renting a luxury vehicle for a simple errand. Feldman emphasized the need for computational efficiency, advising organizations to adopt cost-effective open-source models for routine tasks rather than relying exclusively on premium proprietary systems. The push for refined AI adoption metrics arrives as Cognition and its competitors navigate the financial realities of scaling large language models. Cognition, which develops the autonomous AI software engineer Devin, recently secured more than one billion dollars in funding at a twenty-six-billion-dollar valuation, underscoring the intense capital demand of the sector. Industry leaders now stress that sustainable AI integration requires balancing innovation with fiscal responsibility. By shifting performance incentives toward measurable business outcomes and optimizing model selection, companies can harness AI productivity benefits without succumbing to inefficient token consumption. This strategic pivot is expected to reshape engineering evaluation frameworks across the rapidly evolving artificial intelligence landscape.
