AI Scaling Risks: Why the Industry’s Growth May Be Overheated
- A new study from the Massachusetts Institute of Technology (MIT) suggests that larger, more computationally intensive AI models could soon offer diminishing returns compared to smaller models.
- "In the next five to 10 years, it's very likely that things will start to narrow," says neil Thompson, a computer scientist adn MIT professor who participated in...
- Leaps in efficiency, such as those observed with DeepSeek's exceptionally low-cost model in january, have already served as a blow of reality for the AI industry, accustomed to...
“`html
Diminishing Returns for Large AI Models: MIT Study Predicts Shift Towards Efficiency
A new study from the Massachusetts Institute of Technology (MIT) suggests that larger, more computationally intensive AI models could soon offer diminishing returns compared to smaller models. Comparing the scaling laws wiht continued improvements in model efficiency, the researchers found that it might very well be more tough to get higher performance from giant models, while increasing efficiency could make models running with hardware more modest were increasingly capable in the next decade.
“In the next five to 10 years, it’s very likely that things will start to narrow,” says neil Thompson, a computer scientist adn MIT professor who participated in the study.
Leaps in efficiency, such as those observed with DeepSeek’s exceptionally low-cost model in january, have already served as a blow of reality for the AI industry, accustomed to consuming enormous amounts of computing.
As things stand, a frontier model from a company like OpenAI is currently much better than a model trained with a fraction of the computation from an academic lab. Even though the MIT team’s prediction might not come true if, such as, new training methods such as reinforcement learning produce surprising results, they suggest that large AI companies will have less of an advantage in the future.
hans Gundlach, a research scientist at MIT who led the analysis, became interested in the question because of the unwieldy nature of state-of-the-art models. Together with thompson and Jayson Lynch, another MIT researcher, he mapped the future performance of cutting-edge models compared to those built with more modest computational means. Gundlach says the predicted trend is especially pronounced for the reasoning models now in vogue,which rely more on additional computation during inference.
Thompson says the results demonstrate the value of refining an algorithm and increasing computing power. “If a lot of money is spent training these models, a portion should be dedicated to developing more efficient algorithms, because that can be very important,” he adds.
The study is especially fascinating…