System Design: How to design CDN? Interview question at Facebook, Google, Amazon
Vložit
- čas přidán 27. 07. 2024
- Content delivery network : Instead of getting the content from origin server, we get the content from closest available servers which is edge server.
This is a common interview question related to system design and I tried to explain in simple terms.
In this video, I have talked about below questions
1) Why do we need CDN or Content delivery network?
2) How CDN fits in your system design?
3) How to explain content delivery network to a layman?
4) Production issues across CDN
5) CDN system design.
Other interesting System design links
AirBnB and Booking.com System design: • System Design : Airbnb...
Online Payments (QR Code) Paytm System design: • System design intervie...
Spotify and AppleMusic System design: • System Design: Spotify...
Top 10 Songs in youtube/spotify: • System Design : Top 10...
Amazon (Bestselling Book) System design: • System design: Amazon....
WhatsApp System design (part 1): • Whatsapp System Design...
WhatsApp System design (part 2): • System Design Intervie...
Indian Railway Booking System design: • IRCTC System Design | ...
Distributed logging System design: • System Design : How to...
Redbus System design (Part 1): • System Design: Redbus ...
Redbus System design (Part 2): • System Design: Redbus ...
Twitter System design: • System Design: How to ...
Tiny URL System design: • System Design : Design...
Web Monitoring tool System design: • System design: Web mon...
#systemdesign
----------------------------------------------------------------------------------------
Who am I ?
You can find my experience and skills at linkedin
/ keerthikanth
--------------------------------------------------------------------------------------------------------------------
Disclaimer :
All the views, thoughts, and opinions expressed in the Reachgoals channel are my personal opinion and not necessarily to the author’s current or previous employer, organization, committee or other group or individual.
Excellent video & explanation is very clear !
Glad you think so!
Loved the analogy between buying fruits from the market and a CDN network.
Thanks for liking the content delivery system design.
This video is very basic and covers only top layer of CDN when designing a system like Content Delivery Networks.
Yes, it is basic and want to cover at high level. If you have questions, you can ask and I am happy to answer.
thanks for the great content! easy to understand.
Glad to help!
amazing job! thanks buddy
Glad to help!
AWESOME
Thank you! Cheers!
What kinds of file storage system should be used for the CDN edge web server? How many requests and how much data can they handle? Thanks!
Hi Jack,
Glad you asked this question.
The storage is usually a key value pair, you can use Redis kind of distributed cache with edge server.
To understand the request and data capacity you need to understand the capacity planning. Normally, they are configured to have horizontal scale.
You can understand the capacity by running the testing for machine you have deployed.
Please go through the capacity planning video in this channel and there are multiple way to plan the capacity and I have talked about few methods. czcams.com/video/RH915oGKuyg/video.html
@@ReachGoals Thanks for the reply! Great share!
I was thinking about using local file system as cache. If redis is used, can that handle large concurrency for a accessing the same file?
My initial idea was to use Nginx with local file system as cache. If the required file does not exist in the local cache, it will fetch it from the network file system (NFS) in the edge data center such as (S3, HDFS, etc). In case the NFS does not have it, it will try to get it from the source server.
Since NFS can handle 3-10K request for the same file per second (based on specs of AWS S3), each of the Nginx node fetched the same file and can also handle 3-10K requests for the same file, then the overall concurrency can reach C10M. For a data center close to a big city, that seems enough for most cases.
However, NFS may be very a bit slower for CDN purpose than redis.
Thanks for the easy explanation. I have few doubts wrt DNS resolution of CDN servers. Please help me understand.
1) All edge servers have same domain name?
2) If yes, how does DNS resolver know that the domain name belongs to a CDN server ?
3) How does DNS resolver finds he nearest geo location? Does the DNS server know all the geo locations of a given CDN ?
4) Is there a load balancer in front of the CDN edge servers?
5) Can CDN web server and application server have same domain name?
@Kanika Gupta
1) Edge servers will have domain url like below. This is an example only.
edge1 : abc.com/edje1
edge2 : abc.com/edje2
UI -> CDN path ( CDN will have mapping to multiple edge servers.)
CDN finds the closet edge server. It has algorithm to find.
3) There are multiple algorithms most are property. FYI ,Dijkstra's algorithm is one among many algorithms. CDN server will have the info all the edge servers.
4)Yes LB will be there if there is a need to balance the load between the edge servers.
5) Domain name is a readable name for IP address. It's like a name in telephone directory to find the phone number. Webserver is a front layer to application server. There are application server which has both inbuilt webserver and application server. Apache is a webserver where as weblogic is application server.
Hope this helps.
@@ReachGoals Thank you so much for addressing my questions. It helps me a lot.
You have mentioned in "1" that CDN will have info of all the edge servers. So, UI -> DNS -> CDN -> Edge server becomes the general flow. Could you please briefly touch upon what kind of component "CDN" is and where does it reside? Is it an authoritative server and managed at DNS resolver side?
CDN is a software component which is the bread and butter of the CDN tech. From UI, you point to CDN URL. CND will have connected to origin server. I discusses this is video. There are multiple features in CDN. Please check.
Why do we need to set up a DNS server ourselves? How would clients know that they need to connect to our DNS server instead of their default ones?
>I din't not ask ask to setup on our own. I used Rout 53 which is AWS provided DNS.
It's in the screen presentation too.
>CDN URL used in the web application points to the IP address and the mapping is in DNS.
@@ReachGoals thanks!
Content is good
thanks
Hi can you please provide the list of projects on CDN and AWS Cloudfront ?
@Susmitha,
Are you asking the projects listed in this video? Any online high traffic web applications will have CDN. Ex Kohls.com
What is the necessity to use webserver when we have app server
In the 3 tier architecture will use both app server and web server right
App server can do what ever web server can so what is the necessity
If really both necessary And how the communication will happen b/w app sever and web server
Will use RMI or rest or something else
Please clarify me on this sir
Webservers are for static files and Application servers can handle business logic too. Now a days, you see a combination.
I would define web vs application server base on the purpose of usage.
WebLogic is sold as application server and you can also use only for web serving.
Similarly, webserver with added components, you can make as application server too.
For your original questions
> There will be load-balancer or reverse proxy in front of webserver or application server. LB can connect to the server based on the content type or LB can connect to webserver and webserver traffic is routed to application server. This is purely based on use case and architectural
decision.
>You can also apply rules at webserver to block the traffic, route the traffic based on conditions.
> Also, if your request is only to server JavaScript file or image, webserver can handle. If you need to create dynamic content, you may need to reach db. In this case, webserver can’t handle. It needs to reach another server. Right? That is application server. Web server to application server are mostly http messaging.
Reach Goals thank you sir clear now 🙏