AI agents are evolving from simple "tools" into systems that can operate independently for weeks, tackling complex tasks. The primary bottleneck hindering their widespread adoption is shifting from model capability to the efficiency of the hardware that runs them. Jeff Dean, Google's Chief Scientist and a deep learning pioneer, shared his insights on the current state and future direction of AI in an interview with Y Combinator, pinpointing what he sees as the next major technological breakthrough.

Dean stated that his prediction from last year, that "AI has reached the level of a junior engineer," has largely been validated. He noted that the speed of progress in models completing more complex tasks has actually exceeded his expectations. He specifically emphasized that AI agents are beginning to show their capabilities in areas beyond coding, marking a significant future trend.

When looking ahead to the next big breakthrough, Dean focused on inference hardware. He suggested that if inference latency could be reduced by a factor of 50, it would fundamentally change the product boundaries of AI systems, ushering in entirely new application forms that are currently impossible to create. At the same time, he sent a clear signal to entrepreneurs: context engineering, specialized domain models, and multi-agent systems that integrate inference-time compute are the primary windows for small teams to achieve differentiation.

AI Agent Capabilities Surpass Expectations, Can Run for Weeks

Dean confirmed during the conversation that his assessment from May 2025 that "AI has reached the level of a junior engineer" was largely accurate. He pointed out that actual progress has, in some ways, exceeded his expectations. "The pace at which models can accomplish more complex tasks has been faster than I anticipated," he said. "And we're starting to see these agent-based systems demonstrate their potential in areas beyond just programming."

He believes there is a common blind spot in the market's understanding of AI agent capabilities: these systems can now operate for more than just a couple of hours. In certain problem domains and with sufficiently powerful models, they can run for days or even weeks, completing extremely complex tasks. "Some people are beginning to vaguely sense this, but I don't think it's widely accepted yet. This is going to be a big deal."

On a practical level, Dean gave an example of optimizing the performance of a low-level library with his colleague Sanjay. By writing a "skill" for the agent, they taught the model to complete a full cycle of "measure baseline performance, improve code, measure performance gain, and iterate." This created a highly efficient, automated loop. He also noted that current models are quite effective at translating software from one programming language to another, as the original source code itself serves as an extremely clear specification.

Inference Hardware: The Next Critical Battlefield

Dean defined the efficiency of inference hardware as the core constraint for the next phase of large-scale AI deployment. He drew a parallel to a historic decision he led in 2001—moving Google's search index from hard drives to memory—to illustrate how a leap in hardware architecture can be decisive for product capabilities.

"We're going to see more and more high-performance, low-energy inference hardware systems," he stated. "Inference is the key to making agent-based systems accessible to more people. Latency is extremely important, and specialization of hardware is the key path to improving energy efficiency and reducing latency, far more effective than general-purpose compute devices like GPUs or TPUs."

Dean further explained a fundamental constraint in current AI system design that many entrepreneurs overlook: the energy required to move data from an accelerator's HBM to the processor for computation is thousands of times greater than the energy needed to perform a single multiplication operation. This disparity profoundly affects the necessity of batching, system architecture design, and ultimately, product latency. "Without this thousand-fold difference, you wouldn't need batching. But because it exists, you have to process many samples or many tokens at once to amortize the cost of that data movement."

Regarding the optimization of inference hardware, he highlighted two key points: minimizing data movement and experimenting with extremely low-precision arithmetic, building the required precision directly into the hardware rather than supporting redundant precision options.

The Rise of Context Engineering, a Window for Small Teams to Differentiate

Dean pointed out that the driving force behind AI progress is undergoing a structural shift. It's moving from "bigger models, more data" towards system engineering capabilities beyond the model itself, including retrieval, tool use, memory management, and agent orchestration—what the industry calls "context engineering."

"The model is just one part of the overall system you need to build," he said. "This involves how the model uses various tools, how it retrieves relevant information, and potentially how it retains information retrieved from solving past problems and puts that information into the model's context."

He sees context engineering as a major opportunity for small teams, precisely because of the change in the barrier to entry. "Before, training a model required vast resources, GPUs, and data. But for context engineering, you just need an API like Gemini, and then you build your own retrieval system, tool calling, and so on."

For the common problem of agents "derailing" after more than thirty or forty steps, Dean offered two strategies: one is to provide models with skills and prompts to keep them on familiar operational paths. The other is to use a multi-agent architecture, searching for feasible solution paths through inference-time compute. Another model or agent can then evaluate and filter these paths, retaining effective solutions and discarding those that have gone off track.

A Framework for Entrepreneurs: Look for Problems with Near-Zero Model Success Rates

On topics directly relevant to entrepreneurs, Dean provided a specific framework for identifying opportunities.

He noted that general-purpose models are continuously getting better at more and more things. This means entrepreneurs must carefully assess whether their chosen direction has lasting value. "Is what you're doing 'durable'?—in the next six or twelve months, will the frontier models be able to do it?"

When identifying effective entry points, he gave a clear signal: you should look for problems where current general-purpose models have a success rate near 0% or 1%, not 20%. A 20% success rate often indicates a capability is just emerging. With more training data or increased model scale, general-purpose models are likely to catch up quickly.

He believes two types of scenarios have strong structural advantages. First, products that can access special data that general-purpose models cannot, such as personal user information. Second, by training a specialized small model for an extremely difficult and specific problem, you can achieve high-precision solutions at a low cost. He cited AlphaFold as an example—a model specifically for protein folding, not a general-purpose system, which achieved remarkable results in its domain. He sees similar opportunities in fields like materials science and chip design.

Automating the Scientific Method: AI Accelerating Its Own Evolution

Dean's bold prediction for 2027 points towards a deeper evolutionary path: a significant increase in the automation of machine learning systems themselves, forming a loop of "using AI to optimize AI."

The path he describes involves breaking down a problem into sub-problems, running them in a tight, automated experimental loop, and then integrating the results to continuously iterate towards an improved system. "This applies not just to machine learning, but to other scientific and engineering fields as well—basically any area with a measurable goal can now make significant progress."

Dean also mentioned that a key to accelerating this cycle is building faster verification models. He cited a case in quantum chemistry: a Density Functional Theory (DFT) simulation that used to run overnight was approximated by a neural network, achieving a 300,000-fold speedup. This allowed a screening of ten million molecular configurations, which would have taken six months, to be completed in a lunch break, fundamentally changing the rhythm of scientific research.

He also presented a challenging thought experiment: for the past 60 years, the semiconductor industry has assumed transistors must be extremely reliable. But what if you introduced redundancy and fault tolerance at a higher system level, could you build a system using transistors that can produce errors 20 times a day? This kind of re-examination of fundamental assumptions is, in his view, the kind of thinking that can lead to next-generation breakthrough systems.

The Future Scarcity: Judgment, Not Execution

When asked what would be truly scarce once agents can write all the code, Dean gave a clear answer: taste—the ability to know what problem to ask the agent to solve.

"Models aren't necessarily good at making that kind of judgment," he said. "The future will be people guiding massive amounts of AI-assisted computation to achieve great results faster—but that core of 'what you want the model to do' is the key thing you should focus on."

He proposed a quantitative way to train this judgment: periodically write down a list of things that might be important in the next 12 months. A year later, look back and check which ones actually became important, which were done by others, and which are still open. He also emphasized the value of "first-principles" thinking—not getting bogged down in how a problem is solved today, but squinting at the problem's essence to find a completely different approach that could yield a one to two order of magnitude improvement.

Interview Transcript Excerpts

Host: Jeff, welcome, thank you so much for coming.

Jeff Dean: My voice is a bit hoarse, but we'll manage.

Host: You led the development of MapReduce, Bigtable, TensorFlow, TPUs, and Gemini. A year ago at the AI Ascent conference, you said AI had reached the level of a junior engineer. How has that prediction aged?

Jeff Dean: I feel models have improved rapidly on agentic, long-horizon programming tasks. They are quite capable now. Depending on how you define "junior engineer," I think the prediction was basically correct.

Host: What did you underestimate in that prediction?

Jeff Dean: The ability of models to complete more complex tasks grew faster than I anticipated. And I see these agent-based systems starting to emerge in areas beyond programming, which I think will be an important trend.

Host: Give us another bold prediction for 2027.

Jeff Dean: I think the degree of automation of machine learning systems themselves will increase dramatically—having machine learning systems improve their own capabilities by running lots of experiments, breaking problems into subproblems, running them in a tight automated experimental loop, and integrating results. This full automation of problem decomposition and experimentation will be very exciting, and it applies to many scientific and engineering fields.

Host: Looking back at history, in 2001, you and Sanjay calculated that the entire search index could fit into the memory of all servers. That was a key moment. What is the equivalent "fits-in-memory" moment for 2026?

Jeff Dean: I think we'll see more high-performance, low-energy inference hardware. People realize inference is key to making agent-based systems accessible to many. Latency is very important, and hardware specialization is the key to improving energy efficiency and reducing latency, more so than general-purpose GPUs or TPUs.

Host: So you're saying, what if we didn't have to wait?

Jeff Dean: Yes. Imagine if latency were 50 times lower, what could you do with that?

Host: What's a common assumption about AI that is no longer true?

Jeff Dean: That agent-based systems can only run for an hour or two. In some problem domains, with a sufficiently powerful model, they can run for days or weeks on very complex tasks. Some people are starting to see this, but I don't think it's widely accepted, and it's going to be a big deal.

Host: Can you give an example of a task you've had an agent work on for weeks?

Jeff Dean: Having an agent create a new version of software from scratch in a different language, perhaps with better security or performance, and have it really work hard to complete that.

Host: You're famous for napkin math. In 2013, you calculated that if every Google user did three minutes of voice recognition daily, it would require doubling the entire server fleet. That led to the TPU. What kind of napkin math should future entrepreneurs do tonight?

Jeff Dean: Think about what problems and bottlenecks exist in your area of interest. Is there a completely different approach that could lead to a one-to-two order of magnitude improvement in performance or capability? Sometimes, squinting at a problem from first principles yields good ideas others haven't thought of.

Host: You wrote a famous list of latency numbers for a systems engineer. Can you give us a 2026 AI version of important numbers?

Jeff Dean: You'd want to know the bandwidth from accelerator main memory to on-chip memory to the multiplier units, the energy cost of a multiplication, the interconnect bandwidth between chips, and how that degrades as you scale to many chips. These are important numbers that affect how you think about and solve problems.

Host: You mentioned the energy to move data is a thousand times more than the energy to do a computation. What problems do entrepreneurs mistake for "model problems" that are actually energy or data I/O problems?

Jeff Dean: That thousand-fold gap is a key example that shapes many aspects of machine learning. Without it, you wouldn't need batching. Because of it, you must process many samples or tokens at once to amortize the cost of data movement. This deeply affects system design and product latency.

Host: You're known for spending a long weekend hacking out a clever solution. Could you spend a few weeks making batch-size-1 training possible?

Jeff Dean: I've been thinking more about inference lately. Inference requires very low latency, unlike training. I think there's a lot of room for more specialized hardware for inference, and it's not as well explored.

Host: What interesting things are you thinking about for inference?

Jeff Dean: Minimizing data movement. Trying very low-precision arithmetic. If you know what precision you need, build it into the hardware and don't support many other options.

Host: AI progress is moving from "just better models" to things outside the model: retrieval, tools, memory, agents. This is becoming "context engineering."

Jeff Dean: Yes. The model is just one part of a system that uses tools, retrieves information, and has memory. This context is very clear to the model, unlike the fuzzy training data. The orchestration of these complex agent and multi-agent systems will become very important. The interesting thing is that everyone here can participate in context engineering, because it just requires an API, not massive resources to train a model.

Host: How should people get better at context engineering?

Jeff Dean: Use the models, scaffolding, and tools to solve problems. See where they fail. Often, you can write better specifications or "skills" to teach the model how to use tools, significantly improving performance. This is a great way to understand what additional information the model needs.

Host: Can you give an example of a skill you wrote?

Jeff Dean: A few weeks ago, Sanjay and I were optimizing a low-level library. We wrote a "skill" to teach the model the complete cycle of measuring a baseline, making a change, re-measuring, and iterating. This automated a process we would normally do as engineers, and it worked quite well for certain types of problems.

Host: What's the bottleneck when agents start to "derail" after 30 or 40 steps?

Jeff Dean: One approach is to provide "skills" and prompts to keep the agent on familiar paths. Another is to use a multi-agent system, where multiple agents try different paths, and another model evaluates the promising ones. This is essentially searching the solution space using inference-time compute, which can greatly improve reliability.

Host: Where can a two or three-person team win against a company like Google?

Jeff Dean: Google builds general-purpose models. If someone builds a great interface and a specialized model for a specific domain they are passionate about, they can make a fantastic product. But you must ask yourself if the problem is "durable"—will frontier models be able to do it in six or twelve months?

Host: How should an entrepreneur judge which areas are worth investing in?

Jeff Dean: First, pick a problem you're excited about that you believe will be useful. Second, test general-purpose models. If they have a 0% or 1% success rate, that's a good sign. If they have a 20% success rate, it's a dangerous sign because that capability is likely emerging and will improve quickly.

Host: What do these problems look like?

Jeff Dean: Sometimes, your product can access unique data, like personal user information. Or it could be a very hard problem where a specialized small model, trained with the right data, can be very accurate and cheap to run. Think of AlphaFold—a very specialized model for a specific domain.

Host: What will be the scarce skill when all code is written by agents?

Jeff Dean: Having good taste. Knowing what to ask the agent to do. That high-level judgment of "what to work on" is incredibly important. The future will be people guiding AI computation, but that core of "what you want the model to do" is the key focus.

Host: How do you cultivate good taste?

Jeff Dean: Partly from experience working on many different problems. Also, write down a list of things you think will be important in the next 12 months, and review it a year later. Another way is to do crazy thought experiments, questioning fundamental assumptions, like whether we could build a system using transistors that err 20 times a day if we introduced redundancy at a higher level.

Host: What's a crazy assumption you broke that led to a significant system?

Jeff Dean: The TPU is good example. MapReduce was another. We were writing lots of hand-parallelized and fault-tolerant code. We squinted and realized we could create an abstraction layer like MapReduce, hiding the complexity of parallelism and reliability in a reusable library. That started from a thought experiment.