Testing Frontier LLMs (GPT4) on ARC-AGI

Template: www.kaggle.com/code/gregkamra...
arcprize.org/leaderboard
arcprize.org/arc-agi-pub
ARC Prize is a $1,000,000+ public competition to beat and open source a solution to the ARC-AGI benchmark.
Hosted by Mike Knoop (Co-founder, Zapier) and François Chollet (Creator of ARC-AGI, Keras).
--
Website: arcprize.org/
Twitter/X: / arcprize
Newsletter: Signup @ arcprize.org/
Discord: / discord
Try your first ARC-AGI tasks: arcprize.org/play

[@jackq2331]

Excellent.

[@MarkoTManninen]

I understand retries, but I am confuced with the two attempts. Do you always need to provide two? In which case they would have different data and both would be required for 100% correct prediction? I also missed the part in which the prediction and correct answers are matched and prounounced.

[@ARCprize]

Sorry this isn't more clear on the video!
You get two tried at each task. Old competitions had 3 tries. So you can basically give two attempts. If either are correct you pass the task.
Under scoring methodology there is more information: arcprize.org/guide#submissions

[@LimeTubeH]

I'm confused...what are we supposed to attach with our API add-on secret?

[@ARCprize]

What do you mean attach? That’s where you put your API key and then reference it in your code

[@conformist]

first.

[@cyb3rvoid]

That was unreal!

[@conformist]

@@cyb3rvoid for my next magic trick, i will solve the agi price first

[@wwkk4964]

​@@conformistsolve it backwards!

[@filipgara3444]

Ensure diversity in your model

[@johnkintner]

third since no one called it :kappa:

[@aluphshahim5808]

Second 😂