Unity Multiplayer/MMO Game - Game Devlog #3
Vložit
- čas přidán 5. 08. 2024
- My technical thoughts on building a MMO Game. I have no professional experience in this area, and try to come up with an architecture that is simple to implement but also scalable enough for my needs.
00:00 - Intro
01:00 - 1. Using a Networking Provider?
02:19 - 2. Language for Game Server
03:29 - 3. Server Architecture
08:01 - 4. UDP vs. TCP
09:50 - 7. The Network Protocol
10:25 - 8. Position Packet
11:48 - 8. Login Packet
14:11 - 10. UDP Reflection Attack
16:12 - 11. UDP Client in C#
20:59- 12. UDP Server in Python
23:48 - 14. Distributing Player Position
25:58 - 15. HTTP Server Component
26:49 - Conclusion
27:00 - Outro
Part 1 - How To Learn Something New?: • How To Learn Something...
Part 2 - Igniting Creativity: • Igniting Creativity fo...
Part 3 - Unity Multiplayer/MMO Game: this video
Heatmap Animation: • out
=[ 🔴 Stuff I use ]=
→ Microphone:* geni.us/ntg3b
→ Graphics tablet:* geni.us/wacom-intuos
→ Camera#1 for streaming:* geni.us/sony-camera
→ Lens for streaming:* geni.us/sony-lense
→ Connect Camera#1 to PC:* geni.us/cam-link
→ Keyboard:* geni.us/mech-keyboard
→ Old Microphone:* geni.us/mic-at2020usb
US Store Front:* www.amazon.com/shop/liveoverflow
=[ ❤️ Support ]=
→ per Video: / liveoverflow
→ per Month: / @liveoverflow
=[ 🐕 Social ]=
→ Twitter: / liveoverflow
→ Website: liveoverflow.com/
→ Subreddit: / liveoverflow
→ Facebook: / liveoverflow
=[ 📄 P.S. ]=
All links with "*" are affiliate links.
LiveOverflow / Security Flag GmbH is part of the Amazon Affiliate Partner Programm.
Hello, Game Server-side dev here :)
I noticed some flaws in your designs, and I wonder why you didn't learned by using resources available online (for example, Quake Network Protocol, which is open-source, Valve protocol, etc.), it could make you gain a massive time on some of the problems you faced :)
First of all, lets discuss about your choices:
- 1 Thread per client (also called "Fork design", design used by Apache Web Server for example) vs Worker Threads (design used by Nginx):
In most cases, when you want to handle a massive amount of clients on the same server, a design where 1 thread is designated per client is a bad idea. Most async models will use a pool of threads for write and reading operations ! The most common usage is to use a Boss Thread Group (which is responsible for reading and writing to the sockets) and some Worker Group (Which are used to handle the heavy load operations).
The most common design in gaming is to use the Select (or epoll if possible) to have either one thread only (the order would be: Read packets -> Game Loop -> Send game state packets) or to use, as stated before, a Thread-worker-queue model (Using the Boss Thread Group to read the data and deserialize it then add it to a queue that will be processed by your game loop at the start of each loop).
The model you choose is probably, as you guessed it, why some packets where not sent to your clients !
Fork design was heavily used before the existence of "async" and "await" models, because they can provide a reliable way to order data. It's not used anymore because of the limitations it has (Kernel can limit Forks/Threads, switching context costs massive CPU-Time, etc.)
- UDP vs TCP:
As you stated, it highly depends on your needs. UDP Has the advantage to be very modular and controlable, but it's very complicated to handle properly: Security problems, ACK management in some cases, etc.
I will talk after how to tackle this kind of points, but here, let's just talk about the advantages and disadvantages of both transports.
The biggest advantage of UDP is, of course, its Speed (3 to 8x faster than TCP in general), which is proven to be very effective for real-time shooters for example. The less you will have delay between your client and server answers, less you will have unfair ping advantages for your players.
TCP in another hand is very practical: Because everything is checked for you, you don't have to manage anything very important regarding packet ACK and hackers stealing your IP. The only real disadvantage of TCP is to have an open connection (which prevents for reflection attacks)+ and a very low speed. (Open connection, for simply pinging a server for example, is overkill. That's way most "heartbeat" protocols (from master servers to game servers, master servers is what allows you to find servers on CS for example) will not use a connection-based protocol. It's not very useful at this state to open a complete connection.
Let's now talk about the protocol choices:
- Login packet:
The idea behind sending the "identification key" in each packet is interesting, but that means that an attacker can steal your key ! A common practice is to have the key be generated by using the user's credentials (as you've done in your game) and by signing important packets using this key.
For example, a login handshake can be done like this:
Client sends a login request => Server responds with a special key generated using the current timestamp => Client uses this key to generate its own private key, which be used to sign packets using its own credentials => Client sends to the server its login (NOT THE PASSWORD, ONLY AN IDENTIFIER) and signs it => The server can now check the packet by checking if the identifier challenge corresponds to the signature generated by the client. Tadaaaaa.
Using this method, you can sign your important packets (be careful, because signing has a cost in CPU-TIME !) and provide a way to transmit packets WITHOUT assigning them a key that can be stolen !
This method is also used in TCP too, to avoid having to encrypt all the data sent and avoid having credentials thrown on the network.
- Position packet:
Finally, one of the most complicated packets to handle in games in general.
The methods you described (Send position, send delta or send keys) are respectively named: Client-Authorative and Server Authorative (in general, in this model, we will send the keys AND the delta at the same time).
Client-Authorative ways are simple to implement: Simply accept the position provided by your clients, and interpolate on other clients. BUT, it's highly hackable or it will demand a massive amount of work on the server-side and massive checks to avoid hacks (which will cost dev time and CPU-Time).
The most commonly used approach is the Server-Authorative approach with client-side prediction. In this case, the client will send its inputs, the last snapshot ID received from the server (I will talk about this after, but to be simple, its the "current view" of the client) and the delta between the moment where you sent your input and the last snapshot received. This will allow the server to accurately execute your position and will not be affected by the client part. In this way, the client cannot cheat, because the position is handled by the server, and has the exact same way to calculate the position than the client-side. The client will predict its position (that means that, when it will send the keys packet, it will also start moving the local character) and apply what we call "Server Reconciliation" where the client will check what the server is sending him back as a position, and correct itself depending on this data.
And, another very important thing: Your way of doing things can induce rubberbanding !
Do not send ONE PACKET per client position update ! If there are too many clients, you will overuse your clients CPU power and your bandwidth very quickly.
The best way to handle this is to send what we call a "snapshot". Instead of sending one position packet per client, you will send a big packet containing all the positions of your clients. This has also the advantage to provide a reliable way to manage the snapshot ID which can be used for client prediction and entity interpolation !
I will provide various documentations about this bellow !
- One last point was your way to manage multiple instances of your game, because you have no interaction between your players, the model was not very relevant, but be careful with it ! I've added some design patterns commonly used in distributed game servers bellow :), the problem in your model is that, even if you are sharing your players between multiple instances, you are still sending them the positions of others players (which means that your server are still managing too many data that can be used more cleverly by using spacial hashing techniques etc.), I think that your model will handle more players than a single server model, but they will very fastly DDoS themselves with the player positions updates :)
I will not talk about your way to manage game events (using an image is a good way to do it if you have a very limited time, but it's not a very good long-term solution), I'm more interested about the network-part of things :)
Here are some technical documentation if you are interested about this a little more:
- developer.valvesoftware.com/wiki/Source_Multiplayer_Networking
- www.gabrielgambetta.com/client-server-game-architecture.html
- gameserverarchitecture.com/2016/03/pattern-seamless-world-game-server/
- And a list of useful real-world resources that I made when I was doing my thesis on multiplayer games models: docs.google.com/document/d/1TKqoxa4XEnuxDI8qfP1dEEFiorEg94JA4FU6YNylyPQ/edit?usp=sharing
Hope my comment will help others to have some learning resources on Networking in games !
Happy Learning :D
Amazing man! Highly appreciated.
"Instead of sending one position packet per client, you will send a big packet containing all the positions of your clients."
* checking my notes *
That's actually how he did it. There's a big packet containing everyone's positions/angles/status.
This is a great breakdown! I knew there was a better way to do it but wasn't sure how. Thank you for the links, I love learning stuff like this and I think it's time I finally read some articles from the experts. :)
holy shit, love the feedback! lot's to read. have to do that tomorrow. But just a quick comment about the first point. I did not implement per-client thread (it was just an idea that I didn't use in the end). I'm using multiple processes listening on different ports, and each process uses async coding to just handle packets. So the async event-loop is like a worker thread ;)
@@LiveOverflow Oh,, it's a misunderstanding on my part, mybad !
Wow, this channel is such a blessing. I can’t wait to rewatch these videos all over again later on when I have a much better understanding of practical networking and security concepts. Such a great inspiration, thank you for documenting your learning process
12:36 "it's shitty, but good enough"
probably the most programmer thing one could say
Whaaaat no dev would say that, never xD
man stfu
//TODO: fix this shitty code
@@thezipcreator ( ͡° ͜ʖ ͡°)
Is no one going to appreciate him standing up for every single video
This series is gold m8 so much juicy info
Thank you for making this very informative and interesting video! It was very fun to watch and I hope you continue making cool videos like these
So much effort poured in this game. Lovely content
As a networking major who loves games, this video is gettin me a bit excited
I think a UDP/TCP hybrid is a good option. You can give "session identifiers" to the UDP packets using the TCP protocol and just have a sequence number for the UDP protocol. This way you have the benefits of both worlds.
Id tell you a joke about UDP, but you probably wouldn't get it.
That joke is so old, yet still funny
YOU ARE A COMEDY GENIUS.
Hi, I would like to get a joke about TCP, do you have any?
yes, please, tell that joke. ;)
@@omri9325 You would have to understand it before the joke is even told.
lol!
I'm also making an MMO in UNITY :) Its cool to see more dev doing it !
cant they modify the client to send logins that do buffer-overflow, since its not handled serverside?
the production value of this is mind blowing .. amazing work .. even adding chapter metadata .. just beautiful craftsmanship all 'round
The blocks are added by CZcams if the creator does the timestamps correctly in the video description - No Metadata editing required :p
Isn’t that literally the definition of editing metadata? Metadata in the description?
Love the work bro keep it up😁😁😁
Distinguished job You have inspired me to work on my channel, thanks. 🐼
really really really good and interesting videos
Where you are checking if the player moves to fast (within the serverscript) you could square 100000 instead of square rooting distance, which would allow you to save expensive square rooting operations. You could do the same for the speed check. Lovely video
True 😅
I also used Go to build a small Webserver. Was my first project with it. It’s awesome for networking. The standard library is just awesome.
Also the built in concurrency would be perfect for the game server. Can only recommend to learn Go for the future! I’m in love 😍
Nice to see you 😍
24:30 - NATs. When a machine sends a datagram from a local ip:local port to server ip:server port, NAT translates the local address to external one and forwards the datagram as coming from external ip:external port. Doing that, NAT then also remembers (local ip, local port, external ip, external port, server ip, server port) tuple. When NAT receives a datagram from server ip:server port directed at external ip:external port, NAT looks up a tuple corresponding to those two addresses and thus can identify local ip:local port to forward the datagram to. However, if it receives a datagram from server ip:some other port, it simply does not know where the datagram is destined.
‘Spoofing’ source port on the server should work except that such datagrams might have been blocked by the kernel, i.e. never leave the server, because normal user cannot spoof source addresses.
What book do you recommend to learn reverse engineering?
Im just about to buy a server, this has some really good info to know.
You could make sure the UDP login packet is always bigger than the response by just requiring the same data to be repeated X times till you have a decent enough size (would also require more processing on the server though).
Cool video
So that's why the server was preventing me from walking over the walls - it had the whole maze as a 2D picture!
Although eventually I did find away to teleport "through" the walls, but mostly used it while standing on top of them, for better visibility.
Even got outside of the maze once. Not much to do there, except that walking on the _outer_ edge of the outermost wall is the easiest way to get from one side of the maze to another.
You could also look at QUIC, which is basically TCP but faster and modern. It supports multiplexing and TLS so it's secure and fast. I know this wasn't your goal because you wanted to make your game hackable but for other projects it's an interesting idea.
Something interesting from the draft:
"Streams in QUIC provide a lightweight, ordered byte-stream abstraction to an application. Streams can be unidirectional or bidirectional. An alternative view of QUIC unidirectional streams is a "message" abstraction of practically unlimited length." (Section 2)
And
"Streams can be created by sending data. Other processes associated with stream management - ending, cancelling, and managing flow control - are all designed to impose minimal overheads. For instance, a single STREAM frame (Section 19.8) can open, carry data for, and close a stream. Streams can also be long-lived and can last the entire duration of a connection." (Section 2)
why you didn't make event server with kafka or socketIO ?
Can you give more details how you converted the Unity scene to your image map? I have never seen this method before.
Zoom out. Screenshot. Photoshop
great!!
@LiveOverflow : You could have had a password-like field like in password managers. A password manager shows a little eye beside the secrets, so someone looking on your screen does not get it directly.
On one application I open multiple TCP streams and send the data on them all. This increases traffic, but gives redundancy if one packet is lost and one TCP stream stalls. I tried a UDP system with FEC(forward error correction), but I found that sending multiple UDP packets at the same time caused them all to be lost. UDP packets had to be sent at a precise time and that was hard from even a user mode C++ program that had boosted thread priority.
I have that same shirt on right now!!!!
If you want to make your own client-server setup in C#, I can recommend Tom Weilands tutorials on CZcams!
the bunnies are cute. btw chapter 7 is around your application protocol. the network protocol
is IP
very cool - i'd replace long chains of unsightly if() {} else if () {} instead with the switch statement.
Python doesn't have a switch :(
Although you can kind of hack together your own version of a switch-case.
Switch wasn't added to python for a reason. It's not enough pythonic I guess
@@KbIPbIL0 🐍
wow true lol, I used the dictionary-hacky-way in JS for comfortability b4 (although js has switch).
very interesting though!
@@KbIPbIL0 python sometimes seems kinda gay to me imo
21:45 oh I see a flag here👀
Could you send a link to your games? They look amazing.
If you check the source address then you a WAN-loadbalancer could be killing the game if the IP switches, or did I miss something?
The source address will be the external IP (public, usually assigned by your ISP), not the local IP (set by you or a local DHCP server).
i always wanted to use redis to store positions of players stored with quadtree since itd be fast and big
hey L.O. what about things like ZeroMQ?
Noice!
11:39 up until here we thought of the same things more or less, which is pretty cool since i wasnt wrong this time lol
Can you describe briefly why did you decide to go for UDP? You mentioned all the benefits of TCP and then went with UDP? Thanks :)
Cool video btw
I said that in the video. PwnAdventure 3 already used TCP. So I wanted to do something differently. And I never did anything with UDP, so wanted to learn
nice
Python3 tip: f"Are you using the quicker notation for formating?: { "Yes!" if isUsingFormating() else "No!"}"
Use f in the beginning of quotations and then use formations much like javascript `Test ${test()}` notation.
The chapter timestamps are broken
@LiveOverflow the times are wrong configured
Thumbs up for the chapter numbering UDP joke.
You should really check out GafferOnGames, it has a very detailed explanation why you would want to use UDP for real-time networking. And how to work around the shortcomings of UDP. I understand the lack of time but i should still use UDP definitely not TCP for a real-time game.
Go is probably the easiest language to get started with though it can be a little awkward to debug concurrent code for the usual reasons.
"username is limited to 32 bytes" looks like you can inject a longer username to me. Add a split(0,31) for username.length>32 before it's put into the packet.It's still possible to overflow by debugging and skipping that check in assembly, but still much harder. anti-hacking is mostly just about making effort required to hack the target process more than the perceived reward.
19:15 klassisches RubberDuck debugging :D
I was actually wondering if WebSockets would be a viable solution for the client-server communication. It would be fairly easy to implement on the server side with frameworks such as Starlette or FastAPI (Python), and afaik, Unity can handle WS, too...
It is already used in iogames like agar.io :)
It could be used, but wouldn't provide any benefit other than supporting browser clients. (WebSocket is just using TCP under the hood)
It could make it easier to work with, but so would other prebuilt UDP/TCP frameworks
@@AlexMog100
do you know if in mobile it is using TCP ( ws ) also or its UDP ?
@@umen2424 probably still ws, but cant confirm it
In 15:40 you talked about restricting packet IP source to the one that was used during login. Unfortunately in real world situations, there are legitimate users that are under a Multi-Wan network, such as being in some corporate office or educational institute or even savvy home users that are running dual-wan routers. In a UDP situation, these Multi-Wan networks may auto load balance the packets onto different WAN networks leading to a different source IP, causing them to not be able to play your game.
thx good point
I dont think using source IP to prevent UDP reflection is a good idea. A client could be using a tunnel with multiple NAT egress points and they wouldn't know whats happening. Ofcourse this is some edge case, but it does happen, especially in enterprise environments.
1:10 I told you about that in our call, there is also Mirror.(Comunity UNET replacement)
do you have discord ?
exchange data with a chat message structure.
All clients join a channel which broadcasts all movements.
Server checks if action is permitted, drops message from chat if denied.
yes.
God damn Id love to just make a game but its so much to take in
I would have loved seeing Go used instead of Python!
Valorant, CS:Go, ...: "F***ing hackers, cheaters!"
LiveOverflow: "You HAVE to hack my game...!"
ever heard of differential sync(used by google docs for collab)
LiveMMO PogU
I see you avoided chapter 13. Bad luck averted!
I recommend mirror, it's open source and very easy to use. The transport method can be chosen to suite your needs.
assetstore.unity.com/packages/tools/network/mirror-129321
I solved the offline part without hacking 🙃
Damn i had the same idea. A game where the point is to hack it. Except I was thinking of making it an fps with counterstrike inspired movement.
he looks like the guy in the hollywood movie - Zombieland maybe.
Just imagine how much time this guy saved by simply naming the variable "pkt" instead of "packet"... wow
Michael Cera is that you?
its been a few days since the second video, and you still wear the same shirt!?
Don't own a "go-to shirt?". Most adults do; but maybe if your mom is still picking your clothes for you this is not the case. Also, some people wash their clothes everyday :)
@@pictureus guys...chill :D
most likely he recorded them all at the same time...
5th
where do I find files edited by tools such as cheat engine or gameguardian on Android, and how do I see package data transfers in online games? tutorial please
Just curcious: have you tryied Godot? Not a fanboy of it but i think it`s a great option too.
You can use shared memory between processes
Maybe using websockets would have been better
5:40 funny enough Python 3.8 has multiprocess.shared_memory now
RecieveDataThread? Your code hás already a bug ;) Should be ReceiveDataThread!
Hey! I see a lot of people giving their experiences in the comments. I would like to give a different piece of evidence : orders of magnitudes. You had a lot of questions about bottlenecks and I think that a lot of your questions could be solved by a simple order of magnitude math which is really well explained here (look at the talk linked there) : github.com/graydon/napkin-math
Huh. I thought about your speed code and for some reason I thought about "yahoo yahoo yahoo yahoo yahoo yahoo".
One bit of small feedback as well is about message encoding. I've noticed in the client that there's a lot of code to parse each packet byte by byte etc. This wastes a lot of time, error prone and most importantly hard to get backward comptability righ if you ever want to change anything in the protocol. Instead of you implementing your own binary protocol, you could use something like protobuf (developers.google.com/protocol-buffers) or thrift (thrift.apache.org/). In protobuf/thrift you define your structures in a language agnostic DSL and then they take care of generating serializers/deserializers for your messages in multiple different languages. Adding/removing fields is then handled by those libraries as long as you follow the rule for modifying your definitions. I think this could make your life much much easier. Hope this was useful! Great series btw, I love it!
every "mmo" style gameserver I'ce ever made (going all the way back to VB.6 days) I've had {clients} [x]MMs As where x is the server id, MMs is middle-man-server and As = actual server. Essentially, instanced server, one database. Pretty much replicating runescape's server structure. your client[x]serverProcess[x]ServerProcess structure is interesting though
So, what do we learn from this?
One does not simply make a singleplayer game into a multiplayer game!
(Especailly if that singleplayer game is running on a already "overloaded" engine ...)
You all know what assortment of bugs, aehm I mean game I am talking about ...
Your description has a bug it seems...
Unless I'm mistaken, your "UDP Server in Python" segment begins at 20:59 or 21:00, not 22:08
#PenTestingWithLiveOverflow
Liveoverflow, have you seen tryhackme? Maybe you can work with them and see if they will host you. They are doing kingofthehill hacking challenges.
24:57 too much reddit
/r/mazemessagebroker ;D
under 301 club
3rd reeeeeeeeeeeeeeeeeeeeeeeeeee
You look like and act like Jameskii
Bruh
Most of These Days Using C++ to hack Online Games
Lazy Optimization technology 🤪🤣
haha fugly code yesyes
I prefer it when you are not standing up. It feels more gamer to be sited. I guess.
Go sucks, don't use it. If you need more than what Python can give you just bite the bullet and use Rust. You'll thank yourself later.
why are there always these dorks that don't actually use Rust but love to trash everything else in every comment section
Explain? I'm not familiar with either.
@@xhivo97 it's nothing just ignore it. Rust is a great language, but there's a bunch of people online who never actually use it but somehow still become hooligans that trash everything else.
18:33 eheem... switch-case. I know you were in rush but this huge else if block makes me sick.
doesn't matter
(We don’t need another yander dev)
@@walksanator true
Not sure what you mean. python doesn’t have actual switch case. Though concept is the same and imo calling it switch case is a better description for the logic than if-else.
@@LiveOverflow I was talking about the c# code where you check for the packet type. Overall switch is better than else-if.
But after I rewatched this fragment (18:33) I realized that it would be hard to change it to switch-case.
Sorry my bad.
bah, it's eazy. Just hack the server using admin admin and html :)
i want hack diamonds in free fire it's possible?!?
My brain had an aneurysm trying to read this word salad
Please tell me this is a joke