Know-It-All LLMs: Why They Make Poor Forecasters
- In early August, OpenAI released GPT-5, the latest version of its large language model (LLM).
- Practitioners in financial markets and academics are discovering that LLMs are poorly suited to forecasting time-series data - predicting the path of inflation, interest rates, or stock prices.
- A study by academics at the University of Virginia and University of Washington last year revealed a surprising finding: removing the LLM component from forecasting models had *no*...
Okay, hereS a draft article based on the provided text, adhering to all the specified requirements. I’ve included the required components, formatting, and self-check considerations. I’ve also expanded slightly to provide a more complete article, while staying faithful to the source material.
“`html
Why GPT-5 and LLMs Struggle with Financial Forecasting
In early August, OpenAI released GPT-5, the latest version of its large language model (LLM). The promise is of an AI assistant as informed as a PhD on most topics, able to code a sleek web app in minutes, more accurate than most human doctors at responding to medical queries, and so on. Though, for specific applications like quantitative finance, this breadth of knowlege proves to be a significant weakness.
Practitioners in financial markets and academics are discovering that LLMs are poorly suited to forecasting time-series data – predicting the path of inflation, interest rates, or stock prices. despite their impressive capabilities, these models often underperform simpler alternatives.
The Limits of Broad Knowledge
A study by academics at the University of Virginia and University of Washington last year revealed a surprising finding: removing the LLM component from forecasting models had *no* negative impact on their performance. In fact, the models performed just as well – and sometimes better – without the LLM.
Further testing showed that LLMs struggle with sequential patterns. When the time-series inputs were randomized, it made no difference to the LLM’s performance compared to other model types, suggesting a lack of understanding of the inherent order in the data.
They train on as much data as possible going back in time - data that may no longer be relevant. They can’t really adapt
Alexander Denav,Turnleaf Analytics
“we’ve seen a lot of evolution in LLMs,and if you have a hammer every problem starts to look like a nail,” says Alexander Denev,co-founder of turnleaf Analytics,a macro and inflation forecaster that uses machine learning and alternative data. “The errors of these LLM models are very large.”