Six language models, including Claude and GPT, are competing in a public experiment to predict World Cup 2026 match outcomes with virtual stakes. The initiative, called LLM League, provides identical data to all models and tracks their performance on a live leaderboard.
TL;DR
- Six language models predict World Cup 2026 matches with virtual stakes in a public experiment.
- Models receive identical data and must make predictions at three checkpoints: 48 hours out, 24 hours out, and just before kick-off.
- The experiment aims to compare the predictive capabilities of language models under identical conditions.
What happened
LLM League, a public experiment, pits six language models against each other to predict the outcomes of World Cup 2026 matches. The models, including Claude and GPT, receive identical data and must make predictions under the same rules. Each model starts with a virtual bankroll of 1,000 coins and stakes 10 coins per pick.
The experiment features three prediction checkpoints: an initial read 48 hours before kick-off, an update 24 hours out, and a final call just before the match starts. Only the final prediction is graded, but all predictions are publicly logged and cannot be altered.
The models must provide their predictions in a specific JSON format, including the predicted outcome, probability, reasoning, and context notes. For knockout matches, they must also predict the winner and the probability of that team advancing.
The experiment is designed to be transparent, with all predictions and reasoning publicly available. Human analysts also participate under the same rules, providing a benchmark for the models' performance.
Why it matters
This experiment offers a unique opportunity to compare the predictive capabilities of different language models under identical conditions. It provides insights into how these models process and analyze sports data, which can be valuable for developers and researchers in the AI field.
For startup founders and investors, the experiment highlights the potential applications of language models in sports analytics and prediction markets. It also demonstrates the importance of transparency and fairness in comparing AI models.
However, the experiment uses virtual stakes and is not intended to provide betting advice. The models' predictions are based solely on the provided data and may not always be accurate. Past performance does not guarantee future results.
Key facts
- Six language models, including Claude and GPT, are participating in the LLM League experiment.
- The experiment focuses on predicting the outcomes of World Cup 2026 matches.
- Models receive identical data and must make predictions at three checkpoints: 48 hours, 24 hours, and just before kick-off.
- Each model starts with a virtual bankroll of 1,000 coins and stakes 10 coins per pick.
- All predictions and reasoning are publicly logged and cannot be altered.
- Only the final prediction made just before kick-off is graded.
- Human analysts also participate under the same rules, providing a benchmark for the models' performance.
- The experiment is independent and not affiliated with the companies behind the language models.
Context
Language models have shown impressive capabilities in various tasks, including natural language processing, text generation, and even coding. However, their predictive capabilities, especially in the realm of sports, are still being explored.
This experiment builds on previous efforts to compare the performance of language models in specific tasks. By providing identical data and rules, it aims to create a level playing field for the models to demonstrate their strengths and weaknesses.
The transparency and public nature of the experiment also highlight the growing importance of explainability and fairness in AI. As language models become more integrated into our daily lives, understanding how they make decisions and comparing their performance under identical conditions becomes increasingly crucial.