Twitter System Design on AWS | Microsoft, Google, Facebook Whiteboarding Interview Question
Vložit
- čas přidán 17. 07. 2024
- Designing Twitter is a common system design interview question. In this video, we will cover features like tweeting, retweeting, timeline, trending. This design needs to be scalable, highly available, and secure.
In addition to the design, this whiteboard architecture is also implemented using AWS services.
VP of Engineering at Twitter Design Talk: www.infoq.com/presentations/T...
Please Subscribe
▶ / @cloudwithraj
*Connect with me*
🤳 Instagram: / cloudwithraj
🏢 LinkedIN: / rajdeep-sa-at-aws
🐦Twitter: / cloudwithraj
🖼 Facebook: / cloudwithraj
👩🎓**My Courses - Comment in video for discount coupons**
Discounted Link for my Kubernetes on EKS course: www.udemy.com/course/rocking-...
Discounted Link for my Serverless Course: www.udemy.com/course/aws-serv...
Discounted Link for my CloudFormation, CDK with Devops, Interview Guide Course: www.udemy.com/course/rocking-...
Timestamps:
0:00 Requirements and features
2:06 Table design
5:36 Twitter timeline design
8:53 Tweet, retweet design
12:34 Database selection
17:21 Edge case and design challenge
19:24 Trending feature design
21:34 Security
#SystemDesignInterview #WhiteboardInterview #AWSArchitecture - Věda a technologie
My Rocking Systems Design course is out now in Udemy! Limited time discounted link to get the course for $9.99 - www.udemy.com/course/rocking-system-design/?couponCode=LEARNSYSTEMDESIGN
I cant watch it in corporate side of the udemy. is it not available to companies?
Raj, your videos are incredibly helpful. You are masterful at breaking down complex technical requirements into readily understood concepts. I’ve learned a ton from just a few videos. Thank you for all of your help!
Thanks ! Excellent understanding .
Amazing episode! Thank you.
Thanks Raj!! Let it flow....it's amazing how you turn this complex topics in to simple & understandable
Thanks Rony for the kind words 🙏
Explanation is precise and simple. Please continue with such videos. Thanks for your effor
Thank you Kashmora
Raj, this is super simple. Have also bought your course.
Excellent presentation!
youre like the only channel that makes information so digestible i really appreciate it
I appreciate that!
Great videos from you ! I am learning a lot of AWS !! Thanks a lot
Good One ..thanks
Your biggest fan..keep sharing the knowledge 👍 ...More system design please 😁
More to come!
Excellent thankyou
AMAZING
Perfect as usual!
Thank you! Cheers!
Nice video
Great video!
When explaining when to generate timelines: "It will only create the timelines for the users who have been active in the last 10 hours". I'm assuming that this reduces the compute strain? But what happens when everyone on the US East Coast wakes up, everyone starts logging and looking at their Twitter feed at roughly the same time? The system is busy creating timelines for millions of users at the same time. How does this solution cope with such a situation?
Some content suggestion/request for future videos - Compare/contrasts different way to implement a problems using different AWS technology. So for example so how you would implement something using EC2 vs. using Fargate vs. Lambda and contrast the trade-offs. Same for using Redis vs. Elasticache vs DAX, or using SNS vs. SQS, etc...
Appreciate the great knowledge sharing course.
Glad it was helpful!
brilliant as usual .. Thank You!
Thanks again!
Brilliant Video. Many Thanks Raj
Glad you liked it!
Nicely explained Raj , Thank you so much
Thanks Shyam for your support as always 🙏
Thank you. This is very informative. Please do create more such videos. I am great fan of your videos
Thank you, I will Mahesh
Great video! TY!
Glad you enjoyed it!
Hey Raj Thanks for the awesome video... Smashed the like button as you said😃😃👍
Thanks Sharath for your support as always 🙏
Pretty funny you chose Elon Musk as the famous person way prior to him buying the company. Great video btw!
Thanks Moccar!
Hello Raj, thank you for the video. It was very informative. I am curious though. Is Twitter really using a kind of table as you described for the relationships; where a user's name appears in multiple rows to link that user to his or her followers? I had always assumed some kind of graph database based on what I read on data structures and algorithms.
You are amazing
Ty for the kind words Mesanga
Hi Raj! excellent video! I have a question about the EC2 and ASG part, what if instead of EC2 we use Step Functions with Lambda and EventBridge? could be an option? I mean having a Step Functions workflow that inserts the tweet on Redis and then to Kinesis. And also, could Redis be replaced with Dynamodb?
Hi Luis, Raj mentions in the video that Redis can be replaced by DynamoDB and if the system needs even better performance than plain DDB, you can use DynamoDB Accelerator (DAX)
Like ur shirt bru👊
Ty bro!
Thank you , excellent video , very well explained , in simple language . really appreciated your video can we have some video on AWS Glue ETL tool ?
Thanks Mangesh, yes planning on doing an AWS Analytics video in near future!
Raj thanks a lot for this video. As far as group messaging is concerned, we will use notification service with one queue. As for grouping message is same for every member so our message will pass by SQS and after SQS we will use a notification to every single group member. Is it right please suggest? Rest of the design would be the same.
Hi Raj, great videos, please continue , I like your content, but question: is iy sql or nosql DB?
Hey Mahmoud, Redis is considered a NoSQL database, however it has limited capabilities to create keys and query using the keys. The primary reason to use Redis is to have really fast read/write ability.
How would the schema look like if you were to use DynamoDB? The keys need to be unique, so you could not have three rows of Tina unless you had some other partition key. Would you have some random-ish hashkey as the partition key and then have the Tina column be a GSI?
Another great comment megabenman. Yes, the table needs to have both partition and sort key to make each record unique.
For mobile app deployment will need separate two builds ? One for apple other android app store right?
We can use any graph db to store followers relationship if not why?
Can you please make a video on Bandwidth pricing of AWS and other cloud providers?
Would API Gateway scale enough to handle twitter-esque scale, or would you need ALB/NLB? The AWS Quotas docs state that AWS Gateway is limited by default to 10,000 RPS, but it can be increased upon request. Still, even at a 10x increase, I'm not sure 100k RPS would be enough.
Great question megabenman! The limits are based on account/region. Beyond raising the limits, you also segregate the APIs on different region/API based on user location. Also CloudFront comes into play to cache certain contents without hitting the backend API. Generally it's a mix of ALB and API Gateway for different APIs. ALBs you can prewarm to handle high spokes for certain events (New Year), but it does cost you more.
Question: kms gets overloaded with high number of api calls, they cached it with S3 but how does it happen with other services. What is the throughput and how kms cops up with api calls
Why do we do not use EKS or Kubernetes in EC2 before Database rather than use Autoscaling and EC2? It will reduce Cost of Autoscaling and invocation of EC2?
Where can I get that shirt? Employees only? Ive been looking for some more AWS gear because the stuff in the certification store is just...meh
This is from a Serverless hackathon I participated in Imonous, don't think you can buy this one :(
Thank you , assume full stack code ready, deployment of web twitter.com will be on container s but not server less ? As there's database?
Like ur shirt bru👊
Ty Resham 👊