Superhuman Trackmania AI Demo | map5
Vložit
- čas přidán 4. 07. 2024
- Linesight project.
We used reinforcement learning to train an AI that plays Trackmania. On its training track, the AI is faster than the current human world record. We put this video out as a challenge: can a human still overcome our AI?
Game: Trackmania Nations Forever (TMNF)
Map: tmnf.exchange/trackshow/10460245
Replay (.gbx file): drive.google.com/file/d/1jfOy...
Wirtual attempt: • Can Wirtual beat an AI...
Github: github.com/Linesight-RL/lines...
00:00 Intro
00:13 Best run (silent)
02:17 Improvement over previous AI
04:29 Previous development maps
05:06 Discussion & 40 run swarm
Training method: Reinforcement learning - Hry
But then... the Ai got *this run*
Haha. Believe it or not, but since publishing this video it has improved three times already 😂
I'm pretty sure one day you'll be in a Wirtual video with the Ai because this is pretty interesting 😮
But I doubt an Ai will beat difficult tracks or even use tricks like bugslide 😅
@@Patrik2166 he is already no? Wirtual made a WirtualTV video about beating it I thought
@@travisallan4637 it was someone else's Ai I believe
@@Patrik2166 no same one, same map just the previous version of this one
This is absolutely insane, good job guys! Beating the world record by a second is just astonishing and really shows how much effort you put into this.
crizpy :D
The AI is coming for your job man
@@dozergaming69420 fr
Welp this is scary to see such a good level for an AI but super impressive ! A lot of ppl tried to do AI but this one is the one that impressed me the most.
Glad you enjoyed it!
This is incredible, looking forward to more progress.
It's amazing seeing how previous AI times earlier in the lap were actually better than the best time. Plenty of room for improvement.
hâte de voir le résultat sur d'autres circuits! bien joué
this is insainly awesome keep up the awesome work
This AI will break sub 2 minutes on this map on 27 February 2024
Fingers crossed, but I don't think there's that much tile left on the table 😅
65 hours of training time on normal pc to get better than pros is incredible.
it's actually scary
Keep on training guys! ❤
Great work, keep it up! Can't wait to see your further progress ;) and maybe one day combination with other TAS techniques
How would you combine this with other TAS techniques ?
you just do! duh..
@@linesight-rl Maybe as an alternative to bruteforcing a segment? Rather than random brue force, you would make the ai take over for the segment, +give it some noise so it will have variation.
In theory it could result in an improvement faster than a purely random brute force.
that's incredible !!
Stupid good project and effort, really entertaining and well presented
Thanks a lot :)
++ insane video insane ia really good
Huge congratz! This is a good step forward! Interested to see it learn more tracks. It is crazy to me how pro players can sometimes get clean runs on complex tracks on a first attempt - wonder how long itll take AIs to beat humans in 'first attempts'.
a good step forward for cheaters.
@@FOWST Oh no, anyways.
It's amazing what you've done ! What are you planning to do now ? As you said it would be interesting to make it play on other types of map
Probably try various improvements on this same map, then move onto other tyoes of maps. Is there an interesting map you could suggest for future tests ?
@@linesight-rl It would be fun to see what it could do on the campaign maps. Going further classics like Hakkalicious and if we are dreaming really big 128 Deep Fear and Oachkatzlschwoaf :D but I guess it has a lot to learn until then. Anyways, awesome work on this already, curious to see how far it can go!
Campaign maps would be incredible to watch. This could potentially unveil new strategies, reviving competition on these maps.
@@linesight-rl Should try it with some tech maps
@@kv-5 They've done Hokalicious now! (in case you weren't aware already)
I agree deep fear would be really cool to see, but maybe start with a shorter FS map?
I feel like that would take weeks to train. But I could be completely wrong.
this is nuts
Imagine watching ai play kacky. Would be the pinnacle of trackmania!
Or ice
this is so f*cking cool
It can't go up or down hills, but it's a start.
This is an amazing, nice project ... It must take hundreds of hours... Can I ask you how do you describe map environment and car state as input to the neuron network?
It doesn't, it just gives screen data as an image
I really would like to see the ai try some tech maps, since tech requires precision, I wonder how much precise the ai can become
I’m sending this to the ceo of algorithm
Very cool! I just wonder if a human could beat the Time if he/she spent the same time on the Map
I have a question, does the ai know the whole track while driving ? Or does it only have his frontal view like a player would ? Because that would mean it’s faster on a discovery run than a human who learned the map fully
Imagine in the future, everyone in div 1 cotd is just an AI model lol.
5:40 A question here: Is that some sort of fault in the "refresh rate" of the neural network? The neural network should be deterministic so the same input with the same weights should result in the same outputs, so the same run. Or am I missing something?
The neural network is indeed deterministic, but there is some intrinsic variability due to frame generation. We might take a screenshot 1ms before or 1ms after. The different frame sometimes results in a different action.
Also I have not tested whether clouds and flags in the decor are deterministic. If not, they might cause minimal perturbation that would also explain why runs are not fully repeatable.
@@linesight-rl Isn't image recognition with AI not only good at picking up queues, but also ignoring them? I'd love to see a heatmap.
Don't think shadows or flag veawing should have any effect on an AI, but I'm no AI expert.
Surfaces is "just" an extra node i.e. what surface am I on? Checks color/shape of the blocc, adjusts the inputs.
What I really wonder is how would it fare with obstacles and scenery blocking the view and profile of the next corner.
Leather meatbags have memory for that. But once you give AI that sort of memory it becomes TAS, maybe a selective memory based on bonks?
The problem I see with leaving it to learn the hidden track, is that it may end up thinking all hidden corners are the same and it will fail miserably as soon as you change what you hide.
is there a limit to inputs ? or he can do like riolu
now we wait for the AI to get a sub 2 minute run
Can the same AI be run on other racing games too that only rely on this kind of simplistic inputs? No gearing, no analogue values for any input?
КАК ЖЕ ДОЛГО Я ЭТОГО ЖДАЛ!
когда я только услышал про TAS, изначально, я подумал, что он так и работает - компьютер самостоятельно ищет кратчайший путь, но какого было моё удивление, когда я узнал, что этим занимаются люди (огромное им уважение), но с технологией автообучения ИИ это выйдет на совершенно новый, недостижимый для человека уровень
It would be sick if this AI gets so good that it can do all 60 laps of the E05 TAS-only cut. In any case, I'm interested about the future of the project.
ML bots are much more human-like than you'd think. I don't think they'd be able to reliably pull off TAS only strats.
What's that strategy ?
@@linesight-rl If you search up TASes of E05, you'll find it. There's an uberbug and some chained noseboosts in there iirc.
Oh yeah right, no chance it finds this strategy unless we HEAVILY guide it.
Can you run an experiment with a higher frame rate? Does this impact performance or is 20 FPS sufficient for this game?
when you say that it takes screenshots and can react live, how many screenshots per seconds does it take? i mostly wonder if it's comparable to a movie with 24 pictures per second or something unimaginable like a screenshot every hundred milliseconds :)
meilleures salutations from switzerland, to switzerland ;)
We're playing at 20 FPS, with low-res grayscale screenshots (160x120 pixels).
@@linesight-rl ah wow! didn't expect it to be so "slow" and low-res, and it still manages to easily beat humans :D
Would more FPS and higher res improve the AI?
Higher res: that would definitely help, but we're facing hardware limitations here. We need to store these screenshots in RAM. So there's a compromise to be made between "better screenshots" and "more diverse screenshots". It also drastically impacts training speed.
More FPS: There's a similar compromise to be made. If you can control at a higher frequency, you CAN reach better precision. But it also means that the impact of any action becomes smaller. At some point it would become too difficult for the AI to differentiate between good and bad actions.
@@linesight-rl Okay, I see :) Thanks for the replies!
Sike, it's actually just Carljr
I'm more interested in how fast it can learn tracks it's never seen. Learning one track isn't that big a deal - you can basically just automate brute-force TAS or train with only time as an input and eventually arrive at the same result. Learning visually only, I'm not convinced that it's actually learning to drive a track, more that it's just recognising where it is on the track and being trained into the right response for each moment. I think this must be the case, because it's making tactical decisions about the line it wants based on what's around the corner, which may not be on screen yet - that means the outputs aren't really based on driving skill in response to the track in front of it, it's really just playing from memory.
How you efficiently develop that memory is more interesting. That's how human players play. They see a new track, play it, learn it, get good. I want to see AI doing THAT bit.
The way top level humans in any field play is by developing a large muscle memory base from which they only need to make slight changes when encountering something new to have their internal mind state line up with the stimulus they are getting to play better. Developing that muscle memory bank is the hard part at least for humans thanks to how slow muscle memory develops, just making slight changes to play a new yet highly similar scenario to those you developed muscle memory for is easy, at least for humans.
is an AI run considered a TAS? or is it just a TS?
How can you do this?
how did you learn to create a program like this?
Are you working with the youtuber yosh?
Do you plan on publishing a paper on this?
This is not something we've discussed yet.
Look out Hockolitous
Please do ice😊
i want to see an AI do a run with infinite (or close enough) grip
Hi! Fun challenge. Put this AI into press forward map. What will happen?
no one mentioning the rammstein at the end of map lamo
Do you think it helps ? I'm not sure it's intentional.
@@linesight-rl this can definitely help getting the turn more quickly. it does look like it gained a bit of time there. i dont think it intentionally did that though, its a very lucky trick. it has reacted to it perfectly
I think eventually the idea of ai vs tas will become quite interesting, as ai wouldnt be limited by time consumption or what inputs a human will even attempt try out. we're not there yet, but I feel like it wont be long before major tases of video games will all be made obsolete by ai
that's not how it will happen. AI will not "take over" TAS because as soon as it becomes even equivalent, it will be used as the base runs of TAS, which will in turn be optimized further than the AI could drive
@@Skycrafter_ theres a reason high quality tases require years of new community experience to be improved, because you cannot simply "improve" times. it requires hard trial and error and a lot of time. Ai is perfect for that. Why assume that the AI hasnt already tried the millions other of possibilites that a human would suggest to try out. ai wouldnt have any reason to ever stop optimizing times further, where as a human would (and will) have stopped far before any ai will. this is especially true for games, where the sole way of finding a "best time" is through trial and error, for example many 2d and 3d momentum based games. If you could just take a tas run and simply optimize it further, we would be seeing new tas runs every week or so. theres a reason high quality tases only take longer and longer the better the runs get, because you cant simply optimize a previous tas run. so why do you imagine a human would then have any shot at optimizing a time that was previously optimized thousands of times by ai already
@@TimeattackGD First of all i'd like to point out that I really well know what trackmania TAS is about, having tons of experience with it. Then, I agree with you on the fact that AI may beat TAS in some games which either requires strategy or is simple enough to not have many possibilities, for example a 2d game with only left right up down inputs. Trackmania is very different from all of that. It's a deterministic game, sure, but it's so complex in fact that during every second of every run, there is a total of 524292^100 different inputs combinations. I can ensure you that no matter how good AI gets, there will always be a way to optimize its run further. The only no-return point I can think of, is when AI starts making TAS runs by combining its driving ability and different methods such as bruteforcing inputs, in which case it would be far more efficient than a human making the TAS, but that is far different from the kind of AI you are describing.
@@Skycrafter_ my idea is an ai that uses tas tools, as in more an ai for creating tases, rather than an ai for playing the game, if that makes sense.
Also, the current ai for games are nowhere near what im imagining. They lack any sort of smart way of approaching improvement to times. My feeling is that that problem will be solved fairly soon with how ai in general is developing at the moment. (like it could look at current speedrun footage and interpret it or try other similar strats) like if we dont see an ai tying the tas super mario bros record within the next decade, I would honestly be surprised.
2:08 excuse me, the fuck?
in the old trackmania, the green rumblestrip is like a curb so when you go over it, there's a chance the car bumps into the air.
Ramm bug your talking about? Yeah its one of the many stupid bugs in the game
lol i beat the old ai but new 1 does shortcut i guess?
2:08 ... um... did it do this on purpose?
No idea :)
But probably not : see how it is the only on in the 40 AIs run at the end that does it.
That isn't even difficult, I've seen Riolu get tighter lines for a start.
No you haven't, at least not on the map this AI was trained to play well in otherwise the AI would not be beating the world record holder.
Try to make an ai that understand the game and can easily find shortcuts after analyzing the track
And the new era of cheating has begun. You might not realize it yet, but you're just showing future cheaters the way to go and encouraging them to try the same. If they can't get their hands on your software someone will make their own. From a development or machine learning standpoint, this is interesting and all. I don't wanna judge you for your choice of hobbies and as we all know, cheaters have been around almost as long as the game itself. I just hate seeing one of my all time favourite games ruined.
i don't think ai will seriously impact trackmania play, as it's not very generally applicable (yet). i think you're overreacting
@@astrovation3281 I don't think people will use cheat engine. It's such an overused tool. Ubisoft will surely be able to detect it.
Who cares, trackmania might be a game with leaderboards you could falsify but you as a player are not being directly aimbotted or otherwise having your fun practicing and playing better disrupted unless you're some leaderboard sweat.