Reverse Engineering Age Of Empires
Vložit
- čas přidán 25. 06. 2024
- I reverse engineered the original Age Of Empires to see how its AI works
Become a member to get early access to videos - / @nathanbaggs
Want to build cool stuff from scratch? app.codecrafters.io/join?via=...
Tools:
🐉 - Ghidra - github.com/NationalSecurityAg...
🐛 - x64dbg - github.com/x64dbg/x64dbg
💭 All views are my own 💭
Want early access to new videos and some behind the scenes content? Consider becoming a channel member czcams.com/channels/QvW_89l7f-hCMP1pzGm4xw.htmljoin
Other videos you might enjoy:
czcams.com/video/DvJLIWAGno4/video.html
czcams.com/video/ILY7tYdIS2Y/video.html
czcams.com/video/_2GObcrrWq8/video.html
It felt like you only just got started! As others have said it would be appreciated if you make these videos longer, and maybe even a part 2 of this video where you go further would be amazing!
Responded in another comment but I basically don’t know how long a video will be till it’s done. Always looking to experiment with different video lengths though
@@nathanbaggs Yes I would have liked to see a deeper dive
@@nathanbaggs I'd vote for longer/more detail, or alternatively devote several episodes to a *much* deeper dive into a single game: off the top of my head, now you know how the AI gets its broader instructions what happens if you fiddle with them, how does it decide how to *implement* the instructions (what does it prioritise, how does it cope with resource shortages, etc.), what makes it decide when to go on the offensive, why did Geoff decide to go on an expedition, what strategy does it use to expand territory and explore the map, how does it select its targets in combat, is there a way of exploiting the AI, etc. Obviously a massive undertaking as it requires reverse engineering almost the entire game, but it would be fascinating to watch the process.
@@elbiggus Yeah like 30 mins.
You figured out how the file works, now you need to test your hypothesis by building new ai files with strange or fun behavior.
Yep, what this guy said. :)
The AI in AOE1 is controlled by 3 files. The AI file lists the build order, PER their general behavior towards what they've built and towards other players, the PLY file defines how the AI attacks other players. The AOE Heaven website have guides for each file, but they're not all-encompassing since the AI wasn't publicly documented, it's sourced from modders and what they could find.
Cool thanks for the insight. I try not to look too much up before I start as (for me) it ruins a lot of the fun in figuring it out
@@nathanbaggsreal!!! but it also hurts when you re something and find out someone did the same thing 6 years ago lol
@@king_james_official The story of life is people repeat a lot of what other's have already done, but the exciting part is giving it a try on one's own. ❤
try the community made open ai for aoe
Love it when people try to reverse engineer stuff in old games. There's a guy that spent years working on a 23-year-old WWII 2D real-time tactics game of which the source code was lost to time, and he managed to get modern screen resolutions working which nobody in the last two decades managed to do successfully (the game was hard coded with a crappy resolution with square aspect ratio). The difference it made was absolutely huge and it works flawlessly.
Do you happen to remember the name?
@@ErtugrulK Sudden Strike (1999-2001) and its expansion pack "Forever". The author has videos on it on CZcams, HD mod it was called I think. Was a popular game back in the year 2000 in Europe, tons of mods and thousands of custom maps, but until 2022 nobody had ever managed to get higher resolutions working. Even the company which bought the rights to the IP years ago said they were unable to find the source code for the original games.
I came for the Wololo, stayed for the dissection of one of the games I played as a child.
Wololo
This popped up in my feed and I audibly gasped. Eagerly and excitingly watching now. Keep this great content coming!
Hope you enjoy!
@@nathanbaggs I most certainly did. I'm needing to brush up on using Ghidra so why not also enjoy a video at the same time. Thanks!
I really appreciated this video Nathan. When I was about 10 years old and first getting into computers and I had access to this game. Curiosity drove me to try and unpick installation files in notepad but obviously didn't get anywhere. 20 years later I'm an IT professional and write software and for a living. The desire to learn how things work has not gone away. You've made the process of reverse engineering software as accessible as I think it can be, which is an real achievement, well done!
One interesting AI quirk is in the original Dungeon Keeper. There's a map where you're pitted against an AI keeper, rather than heroes. If you spend too long building your dungeon, the AI keeper essentially just runs their own dungeon into the ground and all their monsters leave. You spend you're time building and training the perfect army to take them down, and when you're finally ready to go steamroll in and take them on all you find is a husk of a dungeon with a load of empty rooms and maybe two or three flies.
Wow I remember that, I always took a long time to train most things to level 10.
Really interesting reverse engineering! Especially the reveal of “this can’t happen before that”.
It’s been 25 years since I did this kind of things and I’ve forgotten a lot of my tricks, so it’s really nice to follow along.
Yeeesss! This is the type of content I love from you Nathan. Take a game, and do a walkthrough analysis of exploiting it. Amazing, hope you can do more, and in way more detail!
This was a great watch, hope you can do more AOE debugging in the near future.
It was great fun to do - I think I will probably look at something different next, but return to AoE in the future
I would love to see a in depth analysis on the grid path finding system it uses
Oh man the music throughout the video takes me back..
Really enjoyed this!
Thanks! It was a real blast from the past for me as well
longer videos please!
I am experimenting with video lengths. To give an insight into this particular video (and the challenges around it), I spent about two weeks doing all the reverse engineering at which point I looked at my notes and could see the story I wanted to tell. It felt like a lot of content but I don’t really know the video length till I spend another two weeks scripting, filming and editing. Basically I don’t know how long a video will be till it’s done
@@nathanbaggs This would have been better if it was about 50% longer (assuming you found the content), but every viewer is different. I know this felt a little short for me.
Will certainly be playing around with different video lengths in future videos. Like I said I looked at my pages of notes and thought I had loads of content, but it all edited down to this
@@nathanbaggsLonger videos also pay significantly more🤑
Would be amazing an analysis of Worms 2/Armageddon AI
12 minutes that was almost sure no less than 12 hours. No matter what, you have all my respect. That’s no easy feat.
Yeah about two weeks of RE work (and another two to script, film and edit)
This is great content. The format of walking through your thinking process and working through reverse engineering something is a great way to teach and get people excited about this subject. Thank you, keep it up!
Incredible video, as always. Keep it up!
Thanks, will do!
This has the same vibe as watching National geographic on those investigation and historical shows.
Had fun watching this video and hope you do a series of this. The end results does not matter, it was the vibe and the journey that was entertaining
Thanks! That's kind of what I'm going for, just solving interesting problems using games people love
mate, you're my hero. I loved doing this sort of thing as a kid but never mastered it like you. I want to get back into it. Thanks for this!
This is way above the level of programming I know how to do but it's very interesting to watch someone try to figure it out. I've played a lot of RTS games and used editors for those that had them and one of the things that really stood out for me about AOE was the fact that they had all of these descriptive names for the different AI scripts that specified civ and bulid order in the file name. I was confused when I started working with other games and was only some kind of "default" AI. Makes me wish more games would include some kind of more detailed AI script editor with their map editor.
The level of programming isn't actually that high, when reverse engineering something like this. You do not have to come up with a solution to a problem you need to solve via coding and that usually is the hard part in programming. What he does here is just looking at someones code and try to make sense of what it does. The debugger is the main tool here, because it enables to stop code execution at any point, to look at the state of the variables used. The difficult aspect of this work is just renaming variables and functions to names that are more meaningful then the decompiler decided to use. Imagine this as some task you get from your boss to look at some code from another one of you coworkers and them being awful at naming their functions and variables and not using comments in the source code to describe anything.
It’s fun seeing your channel grow! Keep at it :)
You have also got to think about things like player interaction. Like if you went over and destroyed their barracks, they would need to replace it to continue making troops. That 4th column definitely looks like it's where the action needs to be done. -1 just seems to be like universal, placing buildings etc which you wouldn't need to click on something to do. Whereas Man is 109, which looks to be the TC, and archer is 87 which looks to be the range.. Pretty cool tbf! Would be very easy to tailor the instruction set to difficulty by just making the build order absolutely perfect or making it terrible haha.
Saw the title about shit my pants lol
I’m pleased it had such an impact
@@nathanbaggs oh absolutely good sir
You make it look easy. I tried to reverse engineer a 1mb command line program using Ghidra and it was too hard (1 yr later the company open sourced it anyway).
I cut out a lot of the misery and me banging my head against the wall
@@nathanbaggs as a programmer very unfamiliar with reverse engineering, it would be cool to have a "behind the scenes" as a separate video to see how you tackle problems (all of the ways in which you've been banging your head against the wall lol)
This just became my favorite channel
Thanks!
Thank you for these videos! ❤
I would literally pay to learn those skills
This is the coolest thing I have ever seen! Immediately subscribed I need to learn how to do reverse engineering
Welcome!
turns out I most needed to hear the AoE menu music today then learn something cool!
I only reverse engineered the cracked version not being properly cracked back in the day. Fun-fact was that the RZR version was like 30 megs and contained the full game, while the full with CD was much bigger (also when installed I think) so I preferred that. However they only seem to have cracked for single player and multi did not work so I did it with softice, HIEW and some other similar tools. How great days they were - good to see people still do these kind of stuff 🙂
Thanks for the story - it's great to know I'm just a small part of large community that loves playing (and breaking) these games
@@nathanbaggs I did reverse GT97 Racing though - because all version online are buggy and the time for checkpoints go too fast - but its a smaller game and dos game - likely my latests achievement in similar vein but smaller than looking around the AI of a game...
Interesting finding there was that one can actually use dosbox as a debugger for smaller games and it eases a lot 🙂
I know there are people who reverse still - but always nice to see. There was this russian guy who created awful lot of DLLs for making games run over opengl instead of DX and often helped them work better on linux for example.... many great people - just never saw ai reversing before ;-)
It is easier to debug a game or app that uses DirectDraw to run in fullscreen mode on a 2-monitor setup. You use one monitor to run the game and the other one to debug. Alternatively, if you have 2 computers, you can use the remote debug feature (run the game on one computer, and debug on the other one).
This was great and all, although very short
If you would like a challenge you could attempt to get the players colours from game memory in Company of Heroes 1. I've been trying to do this for a while and its hard because the colours appear to be assigned by slot order when start positions are fixed but some kind of random algorithm when the player positions are set to random.
The Spiffing Brit has shown that people do watch long-form videos. Something like this needs a good chunk of time devoted to it.
This is a fair comment (and one others have also said). The issue I face is that there's no clear correlation between time spent doing reverse engineering and length of video. This was about two weeks of RE effort, which I thought was enough to tell a story. But by the time I scripted, filmed and edited this was the length of video that fell out. Just a behind the scenes insight - I'm always experimenting with content length and type
@@nathanbaggs Yeah, that's fair enough. I guess you have just hit the right formula to get our interest and we're demanding more because it's easy for a viewer to assume you should be making content every waking hour!
Also reverse engineering source code isn't actually entertaining for anyone else at all. There are probably hours and hours of just looking at values in variables. Good luck on commenting in an interesting way while doing this and keeping up on the mental concentration to not loose track of what you where about to track in the code snippet you are watching in the debugger.
Wololo.
Great vid once again!
Wololo
are u familiar or heard of userpatch, capture age, openage, freeage, aegis aoe engine?
Can you do a video on lego racers, love to see how everything works.
Please make more of exactly this :D
That’s my plan!
@@nathanbaggs aoe3?
Great tip about dumping the sources, thanks. Also it seems that you cut it way too short, I could watch many more of these videos.
I've been a programmer for over 30 years and I'm still not even near the debug-magician Nathan Baggs seem to be. I feel like it would really up my game, but there's so many new things to learn too. =)
Always be learning
Please do more age of empires reverse engineering
Kids, this is why you pay attention in Computer Science class
can you check if the remake uses the same system? maybe you can't debug it because of the drm (unless you get it from a questionable source)
Mr. Froddo Baggens of RE, why not you use your Ring on Starship Troopers, the 1st old RTS one, you command like 10 soldier with different abilities, in 3rd person; the thing is, the game will just bail, sometimes not even a error message; there's a specific mission I remember that will trigger this probably: is a mission where the Engineer, in a 'Mech Suit has the unique opportunity of all the missions, to deploy as many land-mines as you can set, you'd be alone, so be careful, and when deploying like 500 of them, the game crashes, other missions also have unknown limitations like that. Also DID F-22 ADF when you go at the right edge of the map, it'll exit without error, you don't actually get to the edge, but very close to trigger the exit. If you could fix these and increase the capacity of the buffers, or whatever gets overrun, and teach ud how to fix it, would be great and have our eternal gratitude
I would love for you to check out the code for Full Spectrum Warrior, Idk just for fun, I like these videos so much for some reason. Oh and Ghost Recon as it's main menu resolution is "hard coded" which is so low it makes it unplayable on some monitors.
When you want to patch out an instruction, what is the next step to run the patches version of the game? Does it need to be recompiled or…
That’s a good point, sorry I glossed over it. Ghidra has an “export program” option that allows you to save a new copy of the binary with the modifications
@@nathanbaggs ahhh awesome, thanks for clarifying. In your defense, you’re the only one who answered this question which I’ve posted on several rev eng vids
the person who wrote userpatch for aoe must have done a fair bit of reverse engineering, because he managed to enable modern resolutions, increased population, update and improve ai capabilities, and fix many bugs in the map editor. interesting to see the tools used. can they generate c as well as asembler?.
mostly cheat engine to get pointers. ik well the creator of up
have you ever done a video on how to use Ghidra for absolute noobs? like explaining what everything is, useful hotkeys and configs, etc. I think it would be very cool to learn from you
Nice work.
i agree, we need this guy
I've always wondered why Microsoft chose to name mangle basic functions by adding an underscore. I suspect it had something to do with not wanting name conflicts with kernel functions, most of which don't use an underscore, but I guess I'd have to ask someone from Microsoft that question. However, it is odd to me that Ghidra wouldn't automatically correct that name mangling.
Nice one, indeed RE takes a lot of time ey. AOE1 was also one of the first games I hacked/modded as a kid
I want these skillz!!!
Need to watch more videos!
Nice format. In a short video a lot information.
I see "Age of empires" in the title. I upvote immediately!
Wololo
@@nathanbaggs rogan?
really Nice video
Hi, wondrous job. Just i think you partially reinvent the wheel. Check out "How the AoE2 AI Thinks (ft. Promi)" by Spirit of the Law. It breaks down AI concepts of AoE2, sure it will be different but some parts might be similar enough so you will not start from 0.
are u ai scripter as well?
It would be awesome if someone could show how to do this sort of thing (windows exe reversing) alongside wine on linux 😉(though I really should probably just look it up myself at some point). As a bonus you wouldn't have to patch the app to stop it from forcing focus or a resolution globally for all other apps.
u use wine too?
I'd be hyped to see more reverse engineering of this game. The pathfinding in particular always seemed a bit broken.
There's certainly more I can go into in the future
You are insane man❤
Does chatGPT can help reverse engineering? If you paste the assembly
Haha no
@@rosen8757 he would need to create a topic and feed it lots of data, then ai will know what to do
@@AEGISAOE but the "ai" would always decompile some instructions incorrectly so the decompiled code would not compile down to the same machine code. Thus making it useless.
And what is the need for it when the available tools already do it correctly?
killer thumbnail dude
Thanks! Although there's actually two using CZcams's new A/B testing, which one did you see?
@@nathanbaggsthe helmet/armor one
Where's the next video >:( 5 hours passed already
nice to see
So after decades of waiting, our suspicions are finally confirmed.
How does the AI work so well? It cheats🤣
4:58 an asterix? Like the comic character? :P
Better than Obelix
It's "asterisk". Asterix is the comic book character. Otherwise nice video!
I hate that thing with debugging a fullscreen application in the same computer because of focus.
I always use remote debugging for this reason, usually I run the debugee in a virtual machine.
I don't think x64dbg has remove debugging (maybe as a plugin?)
I wonder how badly the AI ‘cheats’
I liked this and checked the worms vid out, subscribed
Thanks!
But can you find out why sometimes Computer Player tries to convert gaia elephant with their priests? lol
Maybe you can fix the pathing for us
now get KSP2 source code and let's make it open source :D
Amazing.
Thanks!
I once tried to extract the sprites of this game for fun, all these sprites are store in SLP files, these binary files are not compressed, it's not difficult to read and extract the data if you know the structure.
Sounds like a future video...
@@nathanbaggs yes, I can help you some documents.
we modded that game over 100 times, we even made it a 3d shooter
I have one AoM/AoE dream: to make an ability to play with 2 persons per each "player", I mean 2 computers or at least 2 screens and 2 mice. As a reverser, how far the state of this game from my dream?
This exists and is called Archon mode.
Low level shenanigans had.
Hope you enjoyed it
Wololo Wololo like this video
Dude I would really like to get your help or the help of someone from your community (discord maybe?). I have been trying for about a decade now (on and off) to understand the savefile format of PatricianII / PatricianIII. It's such an amazing game and there is huge potential for automation if only I could read and write savefiles (they are used for ship navigation). But they seem to use some sort of ancient archiver or something on top of some datastructure and eventhough the archiver.dll is tiny and mostly calling default windows functions I could never understand anything that is happening. I would mean the world to mean if someone could have a quick look at that.
❤❤❤❤❤ 👌 wowowowow
You remember Tormentium? He knew everything about this game. He tore it apart and built an online hack for it back in the 90's. Spoiler Alert!!! I beat him 1v1 while he was using hacks once. Up a hill in ArcherWar. If you know you know. Among Koren 9 year olds, my stiff old man hands are legendary. There's nobody on this planet who's played AoE online competitively that I haven't beaten in a 1v1. I was always too SLoW for em.
The way the script works is there's a base behavior for the starting villagers. They always build houses, a storage pit, and then prioritize between fishing, berries, or hunting, based based on the map seed. The script changes based on local resources and map placement. THEN it goes to the Civ specific AI. The map seed determines most of the behavior prior to the development of civ specific buildings and units. Little shortcut from back when there were no clear paths to begin with.
I sacrificed the print on many a keyboard and put a few thousand miles on some mice to this game. Good stuff.
ozone
wololo
Decent. Fun poking around aint it.
Always
bigdaddy
I’ve been called worse
Are you aware of the custom AI diffficulties that people have made for AoE2?
barbarian and immortal? the ai scripting is different
Is it just me, or is this video a little quiet?
Used the same audio presets I normally do. Maybe watch some other videos to compare ;)
So they hard coded the AI and didn't even use a deep neural network? Weak.
Amateur hour
this was 1995 (1997 at release) and was the peak of rts
empire earth(2002)>aoe(1+2+3+4....)
bye...
Do AoE 2 or we're not watching. Thumbs down.
aoe4 why not?
I'm almost done making a game that is better than this
wololo
Wololo
wololo
Wololo