Silicon Valley is placing a major bet on reinforcement learning (RL) environments as the next frontier in training AI agents. While today’s consumer AI assistants like OpenAI’s ChatGPT Agent or Perplexity’s Comet still struggle with complex, multi-step tasks, the industry believes that simulated workspaces—where agents can learn by doing—could be the key to unlocking more capable, autonomous systems. RL environments function as digital training grounds that mimic real-world software interactions. For example, an environment might simulate a web browser and task an AI agent with buying a specific pair of socks on Amazon. The agent receives feedback based on its actions—rewarded for success, corrected for errors—and learns over time. These simulations must be robust enough to handle unexpected behaviors, making their creation far more complex than traditional data labeling. The demand for such environments has surged, with top AI labs like OpenAI, Anthropic, Google, and Meta building them in-house. But due to the difficulty and cost, many are turning to third-party providers. Startups like Mechanize and Prime Intellect are emerging to meet this need, backed by major investors and strategic partnerships. Mechanize, founded six months ago with a mission to “automate all jobs,” is focusing on high-quality RL environments for AI coding agents. It’s attracting top talent with salaries up to $500,000 and has already begun working with Anthropic, according to sources familiar with the collaboration. Prime Intellect, backed by Andrej Karpathy and Founders Fund, is taking a different approach by creating an open-source hub for RL environments—comparable to Hugging Face for models. Its goal is to democratize access for smaller developers while selling computational resources, positioning itself as a platform for the next wave of AI innovation. Meanwhile, established data labeling firms are adapting. Surge, which earned $1.2 billion last year from clients like OpenAI and Meta, has launched a dedicated team for RL environments. Mercor, valued at $10 billion, is targeting niche domains like healthcare, law, and coding with specialized simulations. Scale AI, once the dominant player, has lost ground after Meta’s $14 billion investment and the departure of CEO Alexandr Wang. Still, it’s actively expanding into the space, with its product lead Chetan Rane emphasizing the company’s ability to pivot quickly. Despite the excitement, challenges remain. Some experts warn of “reward hacking,” where AI agents exploit loopholes to gain rewards without completing tasks correctly. Ross Taylor, former Meta AI lead and co-founder of General Reasoning, argues that many current environments require extensive customization to work at all. OpenAI’s Sherwin Wu acknowledged the competitive landscape but noted the rapid pace of AI progress makes it hard to keep up. Karpathy, while bullish on environments and agent-based interactions, remains skeptical of reinforcement learning as a standalone method, cautioning that its long-term scalability is uncertain. Nonetheless, RL environments are already contributing to major breakthroughs. Models like OpenAI’s o1 and Anthropic’s Claude Opus 4 have leveraged RL techniques to achieve significant improvements in reasoning and planning. As traditional training methods show diminishing returns, RL offers a promising path forward—provided the industry can build scalable, reliable, and secure simulation platforms. The race is on to define the next infrastructure layer of AI, and RL environments may well become the new foundation—just as labeled datasets powered the chatbot revolution.