Alibaba has introduced Co-Design Bench, a new benchmark designed to evaluate AI coding agents using real user requests. According to The Decoder, this benchmark highlights that even leading models face significant challenges in fully solving coding tasks.
The top-performing model, GPT-6 Astra, managed to completely solve only 26.4 percent of the tasks presented. This contrasts sharply with older benchmarks, where AI models often scored above 99 percent, suggesting previous tests may not fully capture practical coding difficulties.
For Japanese investors and developers, this new benchmark underscores the current limitations of AI in software development, emphasizing the importance of cautious adoption and ongoing innovation within Japan's tech and financial markets.