AI Plays Trackmania - Training Progression Side by Side

In this video, an AI is trained via reinforcement learning. In order from the top left corner, top right corner, bottom left corner and bottom right corner the AI has received progressively less training time.
The video compares the lines taken by the various AIs in different parts of the map.
Game: Trackmania Nations Forever (TMNF)
Map: tmnf.exchange/trackshow/10460245

[@quakeev334]

This was really cool, would be awesome to see some players against the top left AI, and also how it performs on different maps and different styles.

[@lumitulva]

Amazing work, this is really impressive. What is the algorithm used here? Also, have you tried running it on other tracks it hasn't seen during training? (Unless it's an algorithm with no inputs that optimizes the given track). Very cool nonetheless.

[@linesight-rl]

Same answer as for Baldert :)
We hope to transfer this AI to other map without retraining from scratch, but this is an area for future work.

[@baldert6108]

This ai in the top left is crazy! i wonder if you could use these in different tm versions too? or would they have to relearn the physics and map

[@linesight-rl]

We use TMInterface to programmatically pause the game, take a screenshot and input our actions.
Until a similar tool is available for other games, we are limited to playing on TMNF.
We think that the AI would not need to completely relearn the physics on a new map, but this is still a work in progress.

[@LightningbrotherG]

@@linesight-rl Oh so it's not playing in real time? Do you need to pause for every frame?

[@Shweetz]

Very interesting, it looks like the run is done in "segments" like TASers would bruteforce it. How many segments are there?
The goal is not just be to reach the end line of the segment as fast as possible, i guess? Speed must be a factor, and also not crashing the wall right behind after the segment, right?

[@linesight-rl]

No, a run is not done in segments.
In each run the AI plays the full map without interruption.
It is only for the purpose of the video that I cut and synchronize segments to visualize the different lines taken by the AI.
If I had not done so, the worst AI would have been 30s late when compared to the best AI at the end of the video, this would have made comparing the lines harder.