This is the second AIMO Progress Prize competition. It builds upon the first AIMO Progress Prize competition, which was won in July 2024 by Project Numina. This second competition has an increased prize pool, a new dataset of problems, increased compute for participants and updated rules for using open-source LLMs.
The ability to reason mathematically is a critical milestone for AI. Mathematical reasoning is the foundation for solving many complex problems, from engineering marvels to intricate financial models. However, current AI capabilities are limited in this area.
The AI Mathematical Olympiad (AIMO) Prize is a $10mn fund to spur the open development of AI models capable of performing as well as top human participants in the International Mathematical Olympiad (IMO).
This second AIMO Progress Prize competition has 110 math problems in algebra, combinatorics, geometry and number theory. The difficulty has been increased from the first competition, and the problems are now around the National Olympiad level. The problems have also been designed to be ‘AI hard’ in terms of the mathematical reasoning required, which was tested against current open LLMs’ capabilities.
To address the challenge of train-test leakage, the competition uses novel math problems created by an international team of problem solvers. Using this transparent and fair evaluation framework, the competition will help to strengthen the benchmarks for assessing AI models’ mathematical reasoning skills, without the risk of contamination from training data.
This latest AIMO Progress Prize competition offers an exciting opportunity to drive innovation in the field of AI for Math, while also fostering healthy competition and supporting open science.
Join us as we work towards a future where AI models’ mathematical reasoning skills are accurately and reliably assessed, driving progress and innovation.
Submissions are evaluated on the accuracy between their predicted labels and the ground-truth labels. In other words, submissions are ranked by the fraction of predicted labels that exactly match the ground-truth labels.
In this competition, every ground-truth label is an integer between 0 and 999, inclusive.
You should arrive at this number by taking the problem solution modulo 1000. If, for instance, you believe the solution to a problem is 65521 should be reported as 521 and -900 should be reported as 100. To be clear, for positive integers larger than 1000, this means: report the last three digits, discarding any initial zero(s). Thus 1009 should be reported as 9.
If a question asks for an answer aa to be calculated modulo mm where mm is specified (not all questions are of this type), then calculate the residue aa modulo mm which is a′a′ with 0≤a′<m0≤a′<m and then report this answer modulo 10001000. For example, if asked to calculate the positive integer 20252025 modulo 999999, the final answer should be 2727. However, if asked to calculate 20252025 modulo 10131013, the final answer should be 1212.
Answers may require basic computations, e.g., ⌊1002–√⌋=141⌊1002⌋=141.
You must submit to this competition using the provided Python evaluation API, which serves test set instances one-by-one in random order. To use the API, follow the template in this notebook.
Note the requirement that the prediction server must be called within 15 minutes of the submission notebook starting, and that each prediction must be returned within 30 minutes.
TOTAL FUND FOR PROGRESS PRIZE 2: $2,117,152
Prizes for Top-Ranking Teams in this Competition:
1st Place: $262,144
2nd Place: $131,072
3rd Place: $65,536
4th Place: $32,768
5th Place: $16,384
Overall Progress Prize Winner: The Overall Progress Prize Winner shall be the highest ranking team that achieves a score of at least 47/50 on both public and private test sets. After any prizes for the five top-ranking teams have been awarded, the remainder of the total fund shall be awarded to the Overall Progress Prize Winner.
If a team is named the Overall Progress Prize Winner in this competition, the prize will be at least $1,589,248. If no team is named the Overall Progress Prize Winner in this competition, the remainder of the total fund shall roll over to the next competition, where the same prize allocation will apply.
Deadline:- 25-03-2025
