A recent study by Stanford researchers highlights the fast-changing landscape of large language models (LLMs). Testing 50 widely cited claims against 20 current models, the study found that 44 percent of these claims are already outdated, with another 34 percent significantly weakened, according to The Decoder.
The research also notes that the average lifespan of claims about LLMs is just 15.7 months, with statements regarding what models cannot do becoming obsolete the quickest. This rapid evolution underscores the challenges in relying on static assessments of AI capabilities.
For Japanese investors and tech companies monitoring AI developments in FX, crypto, and equities markets, this study emphasizes the need for continuous evaluation of AI tools as their performance and limitations can shift rapidly.