From GPT-3 Leader to Anthropic CTO, Tom Brown Discusses Entrepreneurial Experience, Scaling Laws, and Chip Supply Chain dependence.
Amid the rapid evolution of global artificial intelligence, Tom Brown's name has stood out. As the core leader of OpenAI's GPT-3 development, he oversaw and drove the pivotal leap of large language models from the lab to industrialization. Now, as co-founder and Chief Technology Officer (CTO) of Anthropic, he leads the team in launching the Claude series of models.From a certain perspective, Tom Brown's experience reflects the key nodes in the development of AI.
Compared to most conventional engineers, Tom Brown's career path has been more circuitous. He began his entrepreneurial career, self-taught in mathematics and machine learning, and gradually entered the world of cutting-edge research. Leveraging his cross-disciplinary experience in engineering and organization, he joined OpenAI to lead core R&D and played a key role in the GPT-3 project.The nonlinear growth from entrepreneurship to scientific research has given Tom Brown a unique perspective on both technological exploration and strategic judgment.
Recently, in an interview with Y Combinator, Tom Brown shared his transformation journey from entrepreneur to researcher, and talked about the construction of Claude Code, the lessons learned from the design of GPT-3 and LLM systems, and also touched upon his thoughts on challenges such as computing power, energy and future AI security. HyperAI has compiled some of the key points from the interview without violating the original intention. The following is the transcript of the interview.
From OpenAI to Anthropic, Tom Brown's early entrepreneurial journey
Q: What did you learn early on in your entrepreneurial journey?
Tom Brown: I realized very early on thatIn an entrepreneurial environment, you must be a wolf, not a dog.This meant not waiting for tasks or following instructions. When you're starting a business, no one tells you exactly what to do. You face a variety of ambiguous problems and life-or-death challenges every day. That experience instilled in me the habit of proactively trying to solve problems, a shift I believe has been valuable.
Q: How did you transition from entrepreneurship to AI?
Tom Brown: At that time, I felt that I had accumulated some intuition and execution ability in the process of starting a business, but lacked hard-core mathematics and machine learning knowledge.These are necessary if we are to create revolutionary artificial intelligence.I spent 6 months teaching myself, tackling topics that I had previously completely avoided.I need to spend a long time on my own to ensure that I will not become a burden to the team, but a role that can truly help the team.
Q: What was your deepest impression in the early days of OpenAI?
Tom Brown: OpenAI back then was a completely different company than it is today. We were operating out of a small, makeshift office in San Francisco that looked like a chocolate factory, but we had already received $1 billion in funding.The contrast is very strong: on the one hand, there are huge expectations in terms of resources, and on the other hand, we are solving the most basic problems every day.We've tried many things, like building a StarCraft environment and training reinforcement learning agents, but the language model was truly impressive. When GPT-3 came out, everyone had a hunch: this wasn't just a scientific demo, but the beginnings of some kind of general intelligence.
Q: Why did you leave OpenAI to start Anthropic?
Tom Brown: OpenAI was where I grew up, but then there were different ideas within the team. Some felt we needed an organization that considered safety and long-term risk. I was drawn to that atmosphere, so I joined a small group of people to start Anthropic.To be honest, we didn’t have a completely clear roadmap at that time and were still groping in many areas.
Q: What difficulties did Claude encounter during its development?
Tom Brown: Honestly, we didn't have a smooth start. With Claude 1, the results didn't live up to expectations. It was often just a "good enough prototype" rather than a breakthrough. Coupled with delays in infrastructure development, we fell a bit behind schedule.The turning point came with Claude 3.5 Sonnet, where our coding capabilities suddenly became very strong. This not only gave us hope but also improved the external evaluation of our model.
Q: Could you please introduce the launch of Claude Code and Agent tools?
Tom Brown: This is actually quite interesting.We initially created Claude Code purely for internal use. We had absolutely no idea it would be a success in the market; up until that point, we had placed our entire bet on the API.The market response was even better than we expected, with developers in particular immediately grasping its value and becoming the key to opening up the market.
Tom Brown's two intuitive shocks: "Demand Matching" and "Scaling Laws"
Q: What differences have you seen from Grouper to Tinder?
Tom Brown: Grouper essentially brought three couples together to meet at a bar, which was an interesting idea but had limited growth. Tinder was launched when we were working on Grouper. It solved some core problems in online dating by using the "both parties click 'like' before matching" mechanism.For example, it reduces the pain of being rejected and makes the experience safer for girls.This better meets the needs of users, so Tinder's solution is successful.
Grouper and Tinder are two dating apps. Here, Tom Brown talks about how he met Grouper founder Michael Waxman, and then discusses how Tinder offers a superior solution to social problems compared to Grouper.
Q: What inspiration did Scaling Laws give you?
Tom Brown:I think Scaling Laws are the most "clumsy but effective" discovery in our field.The original scaling law was like a straight line, about 12 orders of magnitude, which is a very large number, I have never seen it go beyond 12 orders of magnitude. So I was convinced thatI wanted to focus all my efforts on scaling. Many people were dissatisfied at the time, saying that this was a waste of money on GPUs, but the results of the Scaling Laws were stable. I believe we should dare to do "stupid but useful" things.
Current AI Strategies and Future Challenges
Q: What challenges does Anthropic face in terms of computing power and infrastructure?
Tom Brown: The challenges we face now are completely different from those we faced a few years ago. In the past, we were concerned about whether we had good model ideas.The real bottleneck now is computing power and energy. The number of GPUs and electricity required for AGI-level training is increasing by three times each year, a rate that is almost unparalleled in the history of technology. You have to build huge clusters and optimize energy usage.
Q: How does Anthropic plan its chips and strategy?
Tom Brown: We don't want to bet our fortunes on just one vendor, and we're also trying Google TPU and Amazon's Tranium. This creates more complexity for the engineering team—we need to spread our performance engineering team across all of these platforms.But in the long term, this increases our flexibility: on the one hand, it allows us to absorb additional capacity because there is more capacity overall; and secondly, it allows us to use the right chip for the right job.
Reference Links:
1.https://www.youtube.com/watch?v=JdT78t1Offo&list=PLynUse_piAuGeHpk87iKkQKvZHlcMbNrE&index=6