In this competition, you’ll develop AI systems to efficiently learn new skills and solve open-ended problems, rather than depend exclusively on AI systems trained with extensive datasets. The top submissions will show improvement toward human reasoning benchmarks.
Current AI systems can not generalize to new problems outside their training data, despite extensive training on large datasets. LLMs have brought AI to the mainstream for a large selection of known tasks. However, progress towards Artificial General Intelligence (AGI) has stalled. Improvements in AGI could enable AI systems that think and invent alongside humans.
The Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI) benchmark measures an AI system’s ability to efficiently learn new skills. Humans easily score 85% in ARC, whereas the best AI systems only score 34%. The ARC Prize competition encourages researchers to explore ideas beyond LLMs, which depend heavily on large datasets and struggle with novel problems.
This competition includes several components. The competition as described here carries a prize of $100,000 with an additional $500,000 available if any team can beat a score of 85% on the leaderboard. Further opportunities outside of Kaggle are also available with associated prizes- to learn more visit ARCprize.org.
Your work could contribute to new AI problem-solving applicable across industries. Vastly improved AGI will likely reshape human-machine interactions. Winning solutions will be open-sourced to promote transparency and collaboration in the field of AGI.
This competition evaluates submissions on the percentage of correct predictions. For each task, you should predict exactly 2 outputs for every test input grid contained in the task. (Tasks can have more than one test input that needs a predicted output.) Each task test output has one ground truth. For a given task output, any of the 2 predicted outputs matches the ground truth exactly, you score 1 for that task test output, otherwise 0. The final score is the sum averaged of the highest score per task output divided by the total number of task test outputs.
The submission file for this competition must be a json named submission.json.
For each task output in the evaluation set, you should make exactly 2 predictions (attempt_1, attempt_2). The structure of predictions is shown below. Most tasks only have a single output (a single dictionary enclosed in a list), although some tasks have multiple outputs that must be predicted. These should contain two dictionaries of predictions enclosed in a list, as is shown by the example below. When a task has multiple test outputs that need to be predicted (e.g., task 12997ef3 below), they must be in the same order as the corresponding test inputs.
Awards:-
TOTAL PRIZES AVAILABLE: $1,100,000
- 2024 Progress Prizes: $100,000
- Grand Prize: $500,000
- To Be Announced Prizes (on ARCprize.org): $500,000
2024 Progress Prizes
- Prizes for Top-Ranking Teams in this Competition: $50,000
- First Prize: $25,000
- Second Prize: $10,000
- Third Prize: $5,000
- Fourth Prize: $5,000
- Fifth Prize: $5,000
- Paper Award Prizes: $50,000
- Winner: $45,000
- Runner Up: $5,000
Deadline:- 03-11-2024
