Everyone measures how many problems AI solves. The constraint is that solving isn’t understanding.
The mechanism is the same one I see every time a field falls in love with a metric.
A famous problem—let’s take Riemann or Navier-Stokes as an example, a Millennium Prize—didn’t matter for the answer. It mattered as a measuring tool: if someone solved it, that was the sign that new methods had been born. Then the slow process would start: seminars, discussions, simplifications, a textbook readable by a third-year student. After thirty years, that method would end up inside a compression algorithm or a cryptographic protocol.
The problem was the thermometer, not the fever.
When an AI company uses that thermometer as a product benchmark, exactly what the mathematicians describe happens: the mass production of “true/false” statements at an increasing pace can destroy fertile ground instead of giving it life. You optimize the proxy and lose the thing the proxy was measuring.
———
A lot of people are missing Terence Tao’s point and thinking “mathematicians are upset that AI is better than them.” That’s not what he’s saying, and some people are forgetting that Tao is one of the most AI-pilled mathematicians out there.
His point is that when people work on discovering something, along the way they invent new concepts. Those concepts later become useful far beyond the original goal, and enables further inventions. Finding a solution does matter, but the intermediate idea is often what makes the field richer, because other people can share it and build the next thing from it.
In tech, we can use the analogy of collaborative software. We started with algorithms for merging changes in a Word document, and evolved that to concepts about versions, diffs, and merges, and later to real-time collaboration tools like Git, Google Docs, and Figma. Humans built upon these concepts and developed more powerful solution
———
This 2009 (!) University of Wisconsin-Madison Placement Office study of “reform vs traditional math” outcomes came to mind when reading Corrinne Hess’s 7 July 2026 article: “Fewer than half of Wisconsin students are proficient at math“.
21% of University of Wisconsin System Freshman Require Remedial Math
How One Woman Rewrote Math in Corvallis
Singapore Math
Discovery Math
Math Forum 2007
—-
The data clearly indicate that being able to read is not a requirement for graduation at (Madison) East, especially if you are black or Hispanic.
