Is there a Limit to Number of Connections a Backend can handle?
Vložit
- čas přidán 9. 07. 2024
- Someone asked me a question and I felt its interesting to make a video about, is there a limit to the maximum number of TCP connections a Client can make to the server? If there is what is it? and how does that make sense in all the configurations?
0:00 Intro
1:00 Is there a Max Connection Limit?
4:30 64K Connection Limit Explained
7:20 Max Connections on Reverse Proxies and Max Connections
14:30 How does Router get around Max Connections?
🏭 Backend Engineering Videos
• Backend Engineering (B...
💾 Database Engineering Videos
• Database Engineering
🛰 Network Engineering Videos
• Network Engineering
🏰 Load Balancing and Proxies Videos
• Proxies
🐘 Postgres Videos
• PostgresSQL
🚢Docker
• Docker
🧮 Programming Pattern Videos
• Programming Patterns
🛡 Web Security Videos
• Web Security
🦠 HTTP Videos
• HTTP
🐍 Python Videos
• Python by Example
🔆 Javascript Videos
• Javascript by Example
👾Discord Server / discord
Become a Member
/ @hnasr
Support me on PayPal
bit.ly/33ENps4
Become a Patreon
/ hnasr
Stay Awesome,
Hussein - Věda a technologie
Man, subscribing to your channel is one of the best decisions I have made in 2020. You have touched on the topics that a few people discussed in detail on CZcams. Thanks, keep doing these great work.
I appreciate you Tien! So glad the content helps, let me know how I can do better
Man I am just hooked to your channel! Can,t just have enought of it. Thanks for producing such quality content!
Sir you just leveling up my confidence for life with your videos .. huge thanks ! Will surely applaud !
I listen a lot of your explanations, and they help A LOT ! and I really really love how you use explain software engineering, and I really trust it! Some are new and some are like putting the missing puzzles at to place 🎉👏🏼👏🏼👏🏼👏🏼
Btw you look like a brother of mine, he’s civil engineer and very lever , like you 😊❤️
likes your all videos. A real backend engineer
One of the best channel CZcams ever recommended to me. Thank you for all your efforts🙏
Thank you dear!!
Can you tell me other CZcams channels that was recommended to you?...It may be helpful. Thanks
Saw this video and listened to you, and immediately a like and a subscription 👍
You are awesome mate 🙂
This video made life a bit simpler. Thanks!
If my parents let me choose between university or watching your CZcams videos, I'm too dumb if not choosing your videos!
I super love your contents, really.
I love the way you share and teach me what are the problems, what are the fundamentals, how to solve them until you gave the solution.
Thank you, man!
We definitely like your videos. I have one suggestion: for the knowledges that can be better presented using diagrams or you need draw animations while expressing, get a ipad with apple pencil, draw on a sketch pad app, and record the screen. That way it's much more clear than waving your hands in the air.
Thanks for the tips! Appreciate you dear
Really good content, I appreciate your effort. Have a nice Day. Thanks
Thanks for the dedicated video on this topic..
Sudha Rajamanickam Thank you for watching!
Another great one!
Thank you Ahmed!
Thanks!
Nice clarification, Thanks. Actually, the real problem comes from the maximum file descriptor (FD) limit, that must be configured for servers.
Mahdi Shojaei that is something I missed thanks for sharing!
@@hnasr Thanks Hussien, Is it possible for you to explain file descriptor (FD).. I have done little bit of research.. not able to understand the context here.
You are awesome !!! Very interesting and you try to explain it in lame man terms - watching this gives the interest to learn more...
Great video.
I am a beginner so didn't quite understand the last part. But rest was all good 😁. Keep making these videos great source of learning.
another great video.. thanx
Thanks Ruhan!
Hi. Idea for the topic. Build systems. Why do we need one. History. Shell, Ant, Maven, Buck, Bazel. When do you don't need any. When you could look into Bazel or Buck or any other. Personally i currently digging into Bazel. Seems quite powerful and mono-repo centric. Probably will try to adopt for my workflow as i am working on 7+ related projects simultaneously.
Thanks for vids dude, a correction though. At 13:50, neither adding more backends nor listening on multiple ports on the backend solve the issue that you are addressing (i.e the number of TCP connections that the reverse proxy can open with your backend)
If the ports on the reverse proxy exhausted, if you add one more backed at this point of time, reverse proxy can't open an extra connection just because you added a backed, reason being ports are already exhausted.
Even If the backend is configured to listen on multiple ports, it doesn't help reverse proxy to open one more connections once the ports are exhausted.🙂
Hey sorry @saimanojch, I am two years too late to answer this and since no one answered here, I am. @hnasr can correct me, if necessary. Each TCP/IP packet has basically four fields for addressing. These are:
source_ip source_port destination_ip destination_port
Now, when the number of backend servers increase, along with source_port, destination_ip changes too. So the TCP connection can be registered uniquely.
This content is fire 🔥
Hi Hussein, on the backend machine aren't we limited with file descriptor limits on the OS?
Question: you said that once the reverse proxy hit the max limit it can't open more connection and to solve this we need more backend. Question is that if the reverse proxy is exhausted, then on connecting to the reverse proxy we will be waiting till a socket is open and if we are still waiting here on the front end of the backend that is the reverse proxy and haven't even reached to the backend, then how our problem is resolved by adding more backend?
Is there a limit to in-memory caching? Because I have a websocket connection where I store names of the active users in an array and emit that array to whoever opens the website, So that way they can see who all arw online. But is that a good idea? Or should I use something like redis?
Debug Media that is a good question, if you exceed the memory the OS will take over and page excess memory to disk which can slow things down.. i would implement an LRU cache where you dumb all pages from the cache
Thanks for the explanation, I would like to know if there is a limit in a mysql database connections, I really appreciate your answer
Hundred things run in his mind when he says "when you make a request"
If we add multiple backend servers with only one Load balancer then also the same issue of 64K connections running out would arise right ? It is only with multiple load balancers we can increase the number of connections(more than 64k) right ?
Pulkit Kedia correct the load balancer can only open 64k connections in total from its side.you can avoid this limit by listening on multiple ip addresses on the load balancer. Its equivalent to installing multiple network cards on the load balancer machine so you get multiple nics and as a result multiple ip network addresses and you can listen to all of them and use them accordingly
ah... I thought 64k connection is limited to server, so basically if I run websocket server, even 200k connections from client connected to server then it's no problem right ? the problem might be if the client send more than 64k request at same time to server, is that correct ?
With regards to limits at the backend, can you shed more info on what is the effect of /proc/sys/net/ipv4/ip_local_port_range? Thank you.
8:38 shouldn't it be layer 7 proxy/load balancer , bcoz as u have explained in your layer 4 7 lb video that layer 7 proxy initiates a new tcp connection with the backend server ?????pls explain ??
p k layer 4 and layer 7 proxy usually has two tcp connection. There is one special case of layer 4 proxy where it acts like a gateway (router) and in that case it is a single TCP connection
@@hnasr I watched both your Layer 4 and Layer 7 videos to understand this point, Got it.. Thank you !!
"One stiinki'n server"
Not just "a" server, a *big* server. A *VERY* big server.
Whatsapp is a bit of a utopian example because there are only a handfull of websites as big as whatsapp, the vast majority that us mere mortals have to deal with struggle to get 1Mln users, let alone 1M concurrent connections.
About the proxies; a reverse proxy can ofcourse use as many internal IP's as it needs to, and each backend server can listen on as many internal IPs as it needs to, so there that 64k limit doesn't really exist there either. So a simple HAProxy setup would remove the TCP connections limit alltogether.
vinny142 that is a great point Vinny!
So how would that work ? different network interfaces on different subnets?
I haven’t played with this at all any resources? Cheers and thanks for those valuable info
@@hnasr No real resources off the top of my head but this support question does the same thing:
discourse.haproxy.org/t/set-frontend-ip-as-source-ip-for-backend/320
What it boils down to is that you can instruct HAPRoxy to use a particular IP for outgoing traffic, so as long as that IP is defined on the NIC you can comunicate fro there to many backends and use all of the loadbalancer's IP's for all backends.Thre IP's = 60k connections,etc. Given that most hosted servers will have a private secondary network, and that IPv6 is getting more mainstream, this should solve the problem nicely.
13:59 Actually Hussein could another solution be to add more IPs to the single backend host? Like for example having a NIC with multiple IP addresses and backend is listening to both? While Proxy would have to have some load balancing functionality but the backend host will still be a single machine?
Flankymanga that is correct! The proxy can listen on multiple ip addresses .. i missed to talk about this , Someone else mentioned this in the comments
How does client 2 open another set of 64k connections on the same server port ?
Sharthak Ghosh because client 2 has a different IP address and a whole set of a different local ports
This might help
How Network Address Translation is used on Layer 4 Load Balancing and Port forwarding
czcams.com/video/RG97rvw1eUo/video.html
@@hnasr Makes sense
Nice hair style 😊😊
Great explanation. One question, how does a server establish a connection with a client? A server can handle millions of connections concurrently? After reading it seems like this optimisation is done at the kernel level?
thanks! the client is usually who initiate the connection not the server. I explain this here
czcams.com/video/bW_BILl7n0Y/video.html
@@hnasr I watched this, yes I understand the 3 way handshake and the client sends packets to the server in order to achieve this. My question was more around how the server can handle 3 million concurrent requests like WhatsApp. From what it seems, the server assembles packets together to form ONE TCP Connection and the amount of memory each connection takes is the bottleneck. Is the TCP connection ephemeral or reused in this case?
So at a moment a client can make 64k open connection with reverse proxy. What should be done to handle more than that with reverse proxy like nginx?. Clarify please
It is pretty bad idea to open 64k connections from the same client so I would want to know the use case (same IP address). I don't know anyone including big companies that hit that limit. But if you absolutely have to, nginx or any can listen to different IP addresses and put those behind a DNS as I explained in the video
@@hnasr
Now I understand a bit more after your reply. From a single client , a reverse proxy can handle 64k connection which is more than required for any client server communication.
Say , client A is engaged with reverse proxy with 64k connection (not a use case but curiosity)
Now client B tries to connect with reverse proxy..
Question is :
1. What will happen if client A adds one more request to reverse proxy? Will it be in Q?
2.Say, multiple clients are having max 64k connection with reverse proxy..
can reverse proxy process a new client request without much performance issue?
6:08 it does make sense if you are doing DOS attack like slow lorris.
That is true, slow loris connections are long lived.
@@hnasr As per wikipedia, Nginx and lighttpd server doesn't succumb to this attack. Not sure, how.. en.wikipedia.org/wiki/Slowloris_(computer_security)
I don’t understand who are those 5 guys who disliked your video. Seems some of your coworkers
Hey man you mentioned that whatsapp uses some 3 million ip connections. Do you mind sharing where you get this information? It would be interestingbto see how some other companies are handling on thier backend.
Oh yeah sure I discuss this in length in the WhatsApp video here czcams.com/video/vQ5o4wPvUXg/video.html you can find the resources in the description box
@@hnasr cool man thanks so much! Your efforts are muuch appreciated. :)
❤️❤️
So, is there?
Limit is 2^bits-used-to-represent-int -1
sir layer 4 in your video czcams.com/video/qYnA2DFEELw/video.html you talked about it wont terminate the connection , it will be the original tcp connection throughout but here I got confused as you mentioned layer 4 would terminate the connection and make a fresh connection to backend server?
You are right. I wanted to bring that up as well. With Layer 4 this problem won't occur it is like a pipe. With layer 7 - this problem will occur.
Wowwwwww
We are doing it[64k], We are doing file processing 64k is not enough for us, we are tyring to do same as WhatsApp
I just muted your video for sometime, it felt like you were doing some harry potter stuff with your hands.
hoax