I just finished reading your book. Loved it. Here’s another example of AI predictive modeling snake oil:
You probably know about Zillow, the real estate platform. They started as an alternative to the Realtor MLS service, which was only open to agents. It worked, and Zillow (and a couple of others) really opened up the market. It was a good business.
Sometime in the teens, Zillow started adding estimated house prices to their site. They do this for all homes, not just the ones on the market. The pricing algorithm is proprietary, but probably takes nearby sales into account, as well as the size and physical characteristics of the home (How many bedrooms, etc.). My neighbor is in real estate, and she says the Zillow estimate is a fairly good first order approximation. I know some people who check their home’s Zillow estimate regularly.
In 2018 they decided to eat their own cooking. They started a business flipping houses. If a home was for sale for less than what their model predicted, they would buy it. I believe they make a site visit as well as using their models. Then they would try to sell it.
The flipping business was a total flop. Basically they overpaid for a lot of houses. Even though the overall house market was strong during Covid, they still managed to lose money. I believe the problem was that there are a lot of idiosyncratic variables that go into an individual house’s value, and many of them were not in their AI. Anyway, they started exiting the flipping business in 2021.
Because Zillow’s basic business is sound, the company was able to ride it out. However the stockholders took a substantial bath. The company lost over a billion dollars during this period. The stock fell from $137 to $30. It still has not recovered that loss.
Emergent capabilities based solely on scaling are largely a myth. While scaling does extend capacity, such as with "one-shot" or "few-shot" learning, this is primarily due to a larger pattern-matching basis, inductive mechanisms (not deductive reasoning), and combinatorial effects. Currently, LLM builders are attempting to compensate for significant shortcomings with human intervention (hordes of poorly paid science students and some PhDs) to create the illusion of progress (patching AI). This is not a viable path toward Artificial General Intelligence (AGI).
Historically speaking, progress is slow, but it adds up.
There's likely a lot of mechanisms to understand and modeling to do. Each time we make advances it becomes more clear what problems remain and what to do next.
AI agents calling third-party logic looks like a very promising direction. As much as possible AI should be a glue connecting various parts that are already shown to work well.
Simple thought: all new tech advances quickly with the hardware that’s currently available. When bottlenecks are discovered we have a slowdown until new hardware arises.
Really appreciated this breakdown of the shifting narratives around model scaling—it highlights how much of AI forecasting is shaped by commercial incentives rather than pure technical reality. The discussion on inference scaling was especially interesting, as it underscores how improvements may now come from efficiency and application rather than just brute-force scaling.
It also makes me wonder: with model scaling hitting practical limits, are we about to see a shift where RAG, multi-modal learning, and domain-specific reasoning take center stage? Instead of ever-larger models, will the next breakthroughs come from better integration with structured data and real-world applications?
Are we entering a phase where business strategy and applied AI matter more than raw research breakthroughs?
"Another potential reason to give more weight to insiders is their technical expertise. We don’t think this is a strong reason: there is just as much AI expertise in academia as in industry."
I just finished reading your book. Loved it. Here’s another example of AI predictive modeling snake oil:
You probably know about Zillow, the real estate platform. They started as an alternative to the Realtor MLS service, which was only open to agents. It worked, and Zillow (and a couple of others) really opened up the market. It was a good business.
Sometime in the teens, Zillow started adding estimated house prices to their site. They do this for all homes, not just the ones on the market. The pricing algorithm is proprietary, but probably takes nearby sales into account, as well as the size and physical characteristics of the home (How many bedrooms, etc.). My neighbor is in real estate, and she says the Zillow estimate is a fairly good first order approximation. I know some people who check their home’s Zillow estimate regularly.
In 2018 they decided to eat their own cooking. They started a business flipping houses. If a home was for sale for less than what their model predicted, they would buy it. I believe they make a site visit as well as using their models. Then they would try to sell it.
The flipping business was a total flop. Basically they overpaid for a lot of houses. Even though the overall house market was strong during Covid, they still managed to lose money. I believe the problem was that there are a lot of idiosyncratic variables that go into an individual house’s value, and many of them were not in their AI. Anyway, they started exiting the flipping business in 2021.
Because Zillow’s basic business is sound, the company was able to ride it out. However the stockholders took a substantial bath. The company lost over a billion dollars during this period. The stock fell from $137 to $30. It still has not recovered that loss.
Thanks for the note! Yes, Zestimate is a great example. We actually considered including it in the book but there were already so many examples to pick from! I really liked this explanation of what went wrong: https://ryxcommar.com/2021/11/06/zillow-prophet-time-series-and-prices/
Mistaking an adversarial process for a random one.
Schadenfreude here:
https://www.reddit.com/r/RealEstate/comments/nepv7p/zillow_offers_unbelievable_offer_already/
Emergent capabilities based solely on scaling are largely a myth. While scaling does extend capacity, such as with "one-shot" or "few-shot" learning, this is primarily due to a larger pattern-matching basis, inductive mechanisms (not deductive reasoning), and combinatorial effects. Currently, LLM builders are attempting to compensate for significant shortcomings with human intervention (hordes of poorly paid science students and some PhDs) to create the illusion of progress (patching AI). This is not a viable path toward Artificial General Intelligence (AGI).
Historically speaking, progress is slow, but it adds up.
There's likely a lot of mechanisms to understand and modeling to do. Each time we make advances it becomes more clear what problems remain and what to do next.
AI agents calling third-party logic looks like a very promising direction. As much as possible AI should be a glue connecting various parts that are already shown to work well.
AGI is not a faster horse.
The AIME graph demonstrates conclusively the plateauing of capability with compute.
Replot the graph with a traditional linear x-axis and you will see that capability has plateaued.
Much, much smaller improvements for each order of magnitude of compute.
Hello Sayash,
I hope this communique finds you in a moment of stillness.
Have huge respect for your work, specially the unique reflections.
We’ve just opened the first door of something we’ve been quietly crafting for years—
A work not meant for markets, but for reflection and memory.
Not designed to perform, but to endure.
It’s called The Silent Treasury.
A place where judgment is kept like firewood: dry, sacred, and meant for long winters.
Where trust, patience, and self-stewardship are treated as capital—more rare, perhaps, than liquidity itself.
This first piece speaks to a quiet truth we’ve long sat with:
Why many modern PE, VC, Hedge, Alt funds, SPAC, and rollups fracture before they truly root.
And what it means to build something meant to be left, not merely exited.
It’s not short. Or viral. But it’s built to last.
And if it speaks to something you’ve always known but rarely seen expressed,
then perhaps this work belongs in your world.
The publication link is enclosed, should you wish to experience it.
https://helloin.substack.com/p/built-to-be-left?r=5i8pez
Warmly,
The Silent Treasury
A vault where wisdom echoes in stillness, and eternity breathes.
If you like reading instead of listening, check this out. An A.I. essay was written up from a podcast about the dangers of A.I.…. hmmmm. Irony?:
https://open.substack.com/pub/soberchristiangentlemanpodcast/p/essay-the-ai-deception-2025-unraveling?utm_source=share&utm_medium=android&r=31s3eo
yes it is
Simple thought: all new tech advances quickly with the hardware that’s currently available. When bottlenecks are discovered we have a slowdown until new hardware arises.
Really appreciated this breakdown of the shifting narratives around model scaling—it highlights how much of AI forecasting is shaped by commercial incentives rather than pure technical reality. The discussion on inference scaling was especially interesting, as it underscores how improvements may now come from efficiency and application rather than just brute-force scaling.
It also makes me wonder: with model scaling hitting practical limits, are we about to see a shift where RAG, multi-modal learning, and domain-specific reasoning take center stage? Instead of ever-larger models, will the next breakthroughs come from better integration with structured data and real-world applications?
Are we entering a phase where business strategy and applied AI matter more than raw research breakthroughs?
Incredible article - very level-headed and thorough. A joy to read. Thanks for sharing.
Nakasone ...
"Another potential reason to give more weight to insiders is their technical expertise. We don’t think this is a strong reason: there is just as much AI expertise in academia as in industry."
What's the basis for this claim?
The appalling lack of accurate predictions by techno-nerds within the industry probably plays a part