Group messages: create a database instance just for group messages given the number of users. Each group chat has a unique identifier with an array of all the IDs of the member of the group. Similar to the explanation here, a message from one of the members hits the SQS, then is processed by the backend, which then sends the message to the other members of the group. The only real difference is the database instance dedicated to group messages and that the message itself is sent to various recipients.
@@cloudwithraj btw I have passed my aws pro architect, thanks to your videos too!!! I am continue now to learn and practice, so such course would be helpful for me
Great video, Raj. I had to pause it to read up on how websocket servers can scale/failover while still maintaining connection to the clients. Once I had that straight in my head, the rest of the video made a lot more sense. More System Design videos, please :-)
Just a try ...For Group chat, most of the functions same. But the DB will hold a separate table for the groups which has all the users who are added part of the group. The primary key will be the group name...The app should read from that table in case of group messages.. Still, it can use most of the same infra.
Thank you very much for taking so much time to compile this video. For a message to be unique , the table should hold source, destination and message. So for a group message, the destination could be a set of destination ids( all members in group) instead of one destination id.
I really like the way you explain keeping it very simple and starting from very scratch . Thank You so much . Now am a big fan of yours !! Hope you create some designs which also focuses on Networking , Edge networking , Usage of AWS backbone .
We can design table schema like this sender_id receiver_id( for group message received I'd type is group ,for individual type is single..groupid has 1 to many table having user details part of group... In LLD we can implement composiit pattern to read receipts, message deliveries ( or just fanout to send to groups but for individual buser only one user would exists)
Hi Raj, thanks for sharing your idea on design. I have a question: Why did client app put messages directly to SQS behind the API Gateway. I know API Gateway can directly connect to SQS. In my opinion, the flow should look like this: Mobile App -> API Gateway -> ELB -> EC2 in ASG -> Save to Database, then send the message to SQS (Queue for 1:1) or SNS (Topic for 1:M), then Another EC2 instance as a consumer to get messages from SQS or SNS, and deliver the message through WebSocket to client app, then update the database status, and ACK to message delivery. Isn't it better flow? Thanks,
I know this comment is super old, just replying with my thoughts since I'm trying to improve my systems design process. I'd do API -> SQS without involving EC2 for a few reasons: 1. You remove a point of failure sending items directly to the queue instead of waiting on and relying on the EC2 resources being available. 2. You can use the queue length to scale your compute - you'd need to scale based on resource utilization otherwise. 3. You mentioned having multiple sets of EC2 instances - seems like overkill, in my opinion. Less is more - having the SQS queue directly behind the API lets you offload more of your responsibility onto Amazon's platform. Since you're already paying for the queues, you might as well take advantage of their reliability. :)
Wonderful Raj. The system design series helped me a lot in understranding the use cases of different AWS services. I wish if you could keep them on coming.
Was looking for such an explanation since a long time . Very well explained. You earned a subscriber. Kindly make a video on system design/architecture of applications like Dunzo, swiggy or flipkart. Want to understand how DNS routing works specially when application is accessed globally.
You have thorough knowledge of the subject and you showcase with real time examples. Thats what i liked about your videos. Thanks for all these videos. Can we have more videos to describe projects using STAR method and also to explain challenges faced Vs solution provided for different domains like telecom, banking , e-com etc, please? ..
Hi Raj, all your videos so many useful yesterday I taken your udemy EKS course, please do video on VPC and Route53 how it's working in application level
A queue is not necessary. A user sends the message it goes to the backend and gets stored in the DB then it gets published on websocket to go to user. For unseen messages when the user goes to app it will do a request from server and get all messages that hasnt been seen yet
Again great content. A general comment, your videos are great but you seem to gloss over the DB schemas/data models; you just point an arrow to a DB and don't really go into the details. Data models and schemas are something almost all system design interviewers will dig into. Some questions: 1. You are using DynamoDB but have not defined the data model/table structures. Could you expand on this as I'm sure this is one of the first things the interviewer will ask :) 2. How is the web socket connection between the Sender and the backend established? The Sender connects to the APIG, and the SQS is between the Backend and the APIG. In my mind the Backend has not direct link back to the Sender. 3. How would you implement the feature that allows users to list all the conversations they have? I'm curious to see how this type of relationship is implemented in a noSQL database like DDB 4. You hand-waved away a lot of the actual logic, putting it in EC2. I would have expected a micro services approach. Using EC2 makes it feel like a monolithic application.
For #2 - I believe the client is connected to the API Gateway via websockets. But internally between the APIGW and SQS, and between SQS and the backend compute - these are just standard (believe http) calls Agree for #1 - some simple data modeling for dynamodb would be good to see
This is great Video, It really helped, Hats off. I have Two Question & will wait for your answer please... 1- Assume per second 1 billion messages are being transferred. Will it create 1 billion queues? And incase of one queue & say its in FIFO order, once Alice, bob record is processed , then only the next record will process. This will make the last person/message(last in 1 billion) other party will receive it after 1 hour or so. How this will be taken care? 2- If 2 user and A is served by server1 & server2 then how the communication/Co-ordination happens between the servers?
Thanks Raj, great explanation. I previously faced with a scenario like this and could not explain it very well. This would go along way for me. I'm currently working on your Serverless course on Udemy.
I have enrolled with your boto3 serverless course on udemy, It is quite thorough, I really liked it. Can you make a course on "System Design using AWS" on udemy? I like your way of explaining things. I hope this message gets out of the QUEUE and get an Acknowledgement...:D
Thanks brother! I'll think about it. making a course takes lots of effort, since you have to design the chapters and lectures with gradual progression. I'll keep gathering feedback and if more folks want me to do this, will do it for sure. Thanks for watching and subscribing to my course!
@@cloudwithraj I want that to sir jee. I have learnt a lot of AWS and hae done 4 certifications from practioner to pro level.. I have also cleared CKAD and CKA and Terraform associate.. I have also got formal training in design a microservice... All I want now is more videos and case studies like these so that I really know how things can be brought together
@@cloudwithraj Yes sir.. I have been in the industry for over 15 years now and sadly like most of us in India I just went to Project Management which is not a skill. So I have worked hard to upskills myself but just waiting for trainings that can help me put together all these learnings. You are the best trainer for me on CZcams and udemy for these topics. God bless you
Just one thing wanted to mention, For text message it's fine, but in what’s the app we send images and videos for that storage we need to consider AWS S3 to store those files.
You talked about bringing code to the edge for last online. However Tina needs to go all the way to BE server to retrieve last online status of Bob. Can Tina subscribe to a SNS Topic to retrieve Bob Last online status. Can there be some other Architecture design ?
Good question Anirban! SNS is not ideal in this case coz it is push based notification. So for SNS to work, only when Tina opened Bob's messages in her app, Tina has to let Bob know that Bob needs to send the last online status, however Bob might not be online that time. Also SNS requires explicit acceptance from Tina (same way when you accept every time you set up SNS/email sub). And with growing users, phone to phone SNS connection becomes harder (who manages the topics, subs etc.). That's why Tina fetching the status from the backend is implemented. Hope this helps.
Hi, thanks for this it's really helpful...for the old messages that have already turned green or old messages/history is there a requirement for cache to be maintained in the phone or on the server side that need to be synced with devices
Hello, thanks for the detailed analysis, can you please also include a follow up video for group chats and how authentication using cognito etc can be integrated with this . would be super helpful
@Cloud With Raj: Thanks for the video: I'm assuming for Group Messaging we have to use SQS Fanout method, and if we want specific order to deliver the message then SQS FIFO queue.
I read that lambda has a limit of 1000 concurrent invocations. How would it then be possible to use Lambda as a backend instead of EC2 especially with the amount of requests and users?
Great Video..Quick question if we aren't going for Lambda for the compute layer, why did we chose Fleet of Ec2 with autoscaling and not ECS or Fargate? Keep making more system design videos 😀
Thanks Ojaswiankur for the kind words. I did this one on EC2 coz EC2 is still the most common compute layer. If you are comfortable with Lambda/Fargate, you can use them in the interview instead. I always suggest going with the compute layer that you are strongest in.
Great question Anh. For each websocket connection, the connection id is stored with the end user's cellphone number. When the sender's message comes to the server along with recipient number, the backend looks up the websocket connection id and send the message appropriately.
Hi Raj! You talked about use of microservices but you have only one database, microservices should have their own databases right? Like in this example, if you talk about 'Status' microservice then it should not talk to the main database directly right?
Great question Smita. For DynamoDB there is no concept of database holding tables. It's just multiple tables and each table can have separate scaling criteria. For microservices each API should "try" to interact with a non overlapping tables, but sometimes that's not feasible. Hope this helps.
Hi Raj, Many thanks for taking pains and teaching us. I have query, you are talking about microservices here.. However, in the diagram you are using EC2 has the back end where the application code is and using dynamo DB as DB. I am confused as to where does the microservices fit in here?
Great question Ankush. In this case you run one microservice in one EC2 and put it under it's own auto scaling group. This way each microservice can scale independently since they are under separate Auto Scaling Groups. Sometimes enterprises will implement more than one similar microservice into one EC2. But idea is to enable code implementation and scaling irrespective of each other. Hope this helps.
@@cloudwithraj Ok thanks Bhai... Actually.. Since Microservice is always linked to EKS/K8s in general and therefore, I have just forgotten that its still just an application that can be deployed on EC2 and doesnot have to be a container all the time :)
@@jetank2016 haha yeah, it is natural. Most enterprises move to EC2 first, then modernize to K8s. I purposely avoided K8s as backend to focus on the general design and not bogged by K8s stuff. If you are confident on K8s, you can draw the design with K8s as backend as well.
Like I always say, in interview go with the compute option you are most comfortable with, all have pros cons. Choose amongst Lambda, EC2, and EKS based on your comfort level.
For Group messaging we can create room and add individual users to the specific room just like whatsapp. So if one message is in one room so that message will be part of that room and visible to the user in that room only. This high level idea, let me know how we can it scaleable. Look forward to the next video.
Good Day Raj! Thank you very much for walking us through this awesome design! I'm just starting out in cloud, specifically AWS. I'm studying for the CCP exam currently and I was excited to be able to understand the services you outlined to make this architecture work! It all made perfect sense! I would love to see the group chat architecture solution for sure!! Again thank you so much, be well, be safe and cheers!
Group messages: create a database instance just for group messages given the number of users. Each group chat has a unique identifier with an array of all the IDs of the member of the group. Similar to the explanation here, a message from one of the members hits the SQS, then is processed by the backend, which then sends the message to the other members of the group. The only real difference is the database instance dedicated to group messages and that the message itself is sent to various recipients.
would be great to have system design udemy course, guys like the comment if you agree on that
Since you were the first student on one of my Udemy courses, I will give this some thought Denys! Will keep you posted.
@@cloudwithraj btw I have passed my aws pro architect, thanks to your videos too!!! I am continue now to learn and practice, so such course would be helpful for me
@@cloud_architector Nice Denys, congrats! AWS pro is the hardest cloud cert exam right now. Keep rocking my friend!
Great video, Raj. I had to pause it to read up on how websocket servers can scale/failover while still maintaining connection to the clients. Once I had that straight in my head, the rest of the video made a lot more sense. More System Design videos, please :-)
Just a try ...For Group chat, most of the functions same. But the DB will hold a separate table for the groups which has all the users who are added part of the group. The primary key will be the group name...The app should read from that table in case of group messages.. Still, it can use most of the same infra.
Lets say a group id. Since the name can be duplicate and it would be hard to query the groups.
Thanks for the video Raj. I would like to see more of these system designs with cost effectiveness as well.
More to come!
Hi Raj, Please also make a system design video on e-commerce sites such as amazon, Flipkart etc.
Thank you very much for taking so much time to compile this video. For a message to be unique , the table should hold source, destination and message. So for a group message, the destination could be a set of destination ids( all members in group) instead of one destination id.
Hey Raj... did you ever a do a WhatsApp Group chat system design video? If not - I love to see one!
Would be great to follow up with a database design video for chat applications. Great video, thanks.
Thanks, raj, this series is really helping to understand different services uses as well.
Thanks Rahul brother 🙏
really enjoyed it ... and subscribed for more
Awesome, thank you!
I really like the way you explain keeping it very simple and starting from very scratch . Thank You so much . Now am a big fan of yours !! Hope you create some designs which also focuses on Networking , Edge networking , Usage of AWS backbone .
Thanks and welcome. I do plan to do some networking videos
We can design table schema like this
sender_id
receiver_id( for group message received I'd type is group ,for individual type is single..groupid has 1 to many table having user details part of group...
In LLD we can implement composiit pattern to read receipts, message deliveries ( or just fanout to send to groups but for individual buser only one user would exists)
Thank you again, I really like this kind of videos.
More to come Nicolas!
Thanks. Its give good view to start thought process while designing.
Hi Raj, thanks for sharing your idea on design. I have a question: Why did client app put messages directly to SQS behind the API Gateway. I know API Gateway can directly connect to SQS. In my opinion, the flow should look like this: Mobile App -> API Gateway -> ELB -> EC2 in ASG -> Save to Database, then send the message to SQS (Queue for 1:1) or SNS (Topic for 1:M), then Another EC2 instance as a consumer to get messages from SQS or SNS, and deliver the message through WebSocket to client app, then update the database status, and ACK to message delivery. Isn't it better flow? Thanks,
I know this comment is super old, just replying with my thoughts since I'm trying to improve my systems design process.
I'd do API -> SQS without involving EC2 for a few reasons:
1. You remove a point of failure sending items directly to the queue instead of waiting on and relying on the EC2 resources being available.
2. You can use the queue length to scale your compute - you'd need to scale based on resource utilization otherwise.
3. You mentioned having multiple sets of EC2 instances - seems like overkill, in my opinion.
Less is more - having the SQS queue directly behind the API lets you offload more of your responsibility onto Amazon's platform. Since you're already paying for the queues, you might as well take advantage of their reliability. :)
Wonderful Raj. The system design series helped me a lot in understranding the use cases of different AWS services. I wish if you could keep them on coming.
My Rocking Systems Design course is out now in Udemy! Limited time discounted link to get my courses for $9.99 - bit.ly/3Eku9RH
Thanks for sharing your knowledge and experience. The coupon code "LEARNSYSTEMDESIGN" is expired. Can we get new coupon-code Raj?
@@laxmisuresh Yeah, try LEARNCLOUDGETLOVE. All my course discounts are here - bit.ly/3Eku9RH
OMG, how could I miss this awesome video from my cloud guru Raj..
will explore your Udemy course soon guru..
thank you! respect my authoritah :)))
Great video
Glad you enjoyed it
very clear and useful info. Thanks for making this video. What is your amazon ID BTW? I am in Prime video.
It is very very understandable... Raj Keep rocking..
Thanks Selvan for the kind words 🙏
Very Good technical videos.....Thank you..
Thanks for watching
Was looking for such an explanation since a long time . Very well explained. You earned a subscriber. Kindly make a video on system design/architecture of applications like Dunzo, swiggy or flipkart. Want to understand how DNS routing works specially when application is accessed globally.
Thanks Jayateerth! Will upload a retail online shopping design video in next couple weeks for sure, thanks for the suggestion.
Thanks Raj for this wonderful video. Your channel is near to 10k subscriber :) advance congratulations :)
Thank you so much Vinod 😀
You have thorough knowledge of the subject and you showcase with real time examples. Thats what i liked about your videos. Thanks for all these videos.
Can we have more videos to describe projects using STAR method and also to explain challenges faced Vs solution provided for different domains like telecom, banking , e-com etc, please? ..
Will definitely keep this in mind for future videos Sneha
Hi Raj, all your videos so many useful yesterday I taken your udemy EKS course, please do video on VPC and Route53 how it's working in application level
A queue is not necessary. A user sends the message it goes to the backend and gets stored in the DB then it gets published on websocket to go to user. For unseen messages when the user goes to app it will do a request from server and get all messages that hasnt been seen yet
Nice video, very well explained. Yes, pls do follow-up video on group messages.
Again great content. A general comment, your videos are great but you seem to gloss over the DB schemas/data models; you just point an arrow to a DB and don't really go into the details. Data models and schemas are something almost all system design interviewers will dig into.
Some questions:
1. You are using DynamoDB but have not defined the data model/table structures. Could you expand on this as I'm sure this is one of the first things the interviewer will ask :)
2. How is the web socket connection between the Sender and the backend established? The Sender connects to the APIG, and the SQS is between the Backend and the APIG. In my mind the Backend has not direct link back to the Sender.
3. How would you implement the feature that allows users to list all the conversations they have? I'm curious to see how this type of relationship is implemented in a noSQL database like DDB
4. You hand-waved away a lot of the actual logic, putting it in EC2. I would have expected a micro services approach. Using EC2 makes it feel like a monolithic application.
Great feedback, will keep this in mind for future
For #2 - I believe the client is connected to the API Gateway via websockets. But internally between the APIGW and SQS, and between SQS and the backend compute - these are just standard (believe http) calls
Agree for #1 - some simple data modeling for dynamodb would be good to see
Great video Raj. Much appreciated.
Glad it was helpful!
Awesome video again! I would like to see a Google Authenticator app system design too :)
Well explained
Thanks Siva!
This is great Video, It really helped, Hats off. I have Two Question & will wait for your answer please...
1- Assume per second 1 billion messages are being transferred.
Will it create 1 billion queues? And incase of one queue & say its in FIFO order,
once Alice, bob record is processed , then only the next record will process.
This will make the last person/message(last in 1 billion) other party will receive it after 1 hour or so. How this will be taken care?
2- If 2 user and A is served by server1 & server2 then how the communication/Co-ordination happens between the servers?
Cool brother! Would like to see more architecture whiteboard session from you :D
More to come!
Thanks Raj, great explanation. I previously faced with a scenario like this and could not explain it very well. This would go along way for me. I'm currently working on your Serverless course on Udemy.
Thanks Oladele for supporting my Udemy course and watching this video. I am glad you found this video helpful.
I have enrolled with your boto3 serverless course on udemy, It is quite thorough, I really liked it. Can you make a course on "System Design using AWS" on udemy? I like your way of explaining things. I hope this message gets out of the QUEUE and get an Acknowledgement...:D
Thanks brother! I'll think about it. making a course takes lots of effort, since you have to design the chapters and lectures with gradual progression. I'll keep gathering feedback and if more folks want me to do this, will do it for sure. Thanks for watching and subscribing to my course!
@@cloudwithraj I want that to sir jee. I have learnt a lot of AWS and hae done 4 certifications from practioner to pro level.. I have also cleared CKAD and CKA and Terraform associate.. I have also got formal training in design a microservice... All I want now is more videos and case studies like these so that I really know how things can be brought together
@@jetank2016 Thanks for the feedback bhai :). Amazing job on the certs - CKAA/D is super valuable right now.
@@cloudwithraj Yes sir.. I have been in the industry for over 15 years now and sadly like most of us in India I just went to Project Management which is not a skill. So I have worked hard to upskills myself but just waiting for trainings that can help me put together all these learnings. You are the best trainer for me on CZcams and udemy for these topics. God bless you
@@cloudwithraj I would be glad to buy it
Thanks for such an amazing vedio raj, i am definitely going to enroll some of your udemy courses!!
you are amazing
Thanks Ravikumar, glad you found my videos helpful
Great video Raj!
Glad you enjoyed it
Excellent explanation Dada ..!!
Thanks Dhiraj bhai. Glad you found it useful.
in that case kinese service can be used for large client environment ?
try 1.25x play speed, thank me later
Nicely Explained
Thanks for watching Sachin
Great
Just one thing wanted to mention, For text message it's fine, but in what’s the app we send images and videos for that storage we need to consider AWS S3 to store those files.
You are correct. For images you need to resize if needed and save in S3.
Can you design us microservice architecture sir
Hi can consider design, iot service having api with rate limit and process them in sequence using AWS services.
Can you please share a video on how to make search engines such as trivago on AWS
Will keep in mind for future video Vinay
hey raj, can you give some tips for whiteboarding during amazon aws interviews? like how to start and process?
You talked about bringing code to the edge for last online. However Tina needs to go all the way to BE server to retrieve last online status of Bob. Can Tina subscribe to a SNS Topic to retrieve Bob Last online status. Can there be some other Architecture design ?
Good question Anirban! SNS is not ideal in this case coz it is push based notification. So for SNS to work, only when Tina opened Bob's messages in her app, Tina has to let Bob know that Bob needs to send the last online status, however Bob might not be online that time. Also SNS requires explicit acceptance from Tina (same way when you accept every time you set up SNS/email sub). And with growing users, phone to phone SNS connection becomes harder (who manages the topics, subs etc.). That's why Tina fetching the status from the backend is implemented. Hope this helps.
Hi, thanks for this it's really helpful...for the old messages that have already turned green or old messages/history is there a requirement for cache to be maintained in the phone or on the server side that need to be synced with devices
Hello, thanks for the detailed analysis, can you please also include a follow up video for group chats and how authentication using cognito etc can be integrated with this . would be super helpful
Will do in a future video Anshika, thanks for the suggestion
Don't mind me. Just commenting to help the CZcams algorithm
haha thanks brother! Much appreciated!
@Cloud With Raj: Thanks for the video: I'm assuming for Group Messaging we have to use SQS Fanout method, and if we want specific order to deliver the message then SQS FIFO queue.
Do we have video created for system design for Hotstar?
I read that lambda has a limit of 1000 concurrent invocations. How would it then be possible to use Lambda as a backend instead of EC2 especially with the amount of requests and users?
Great Video..Quick question if we aren't going for Lambda for the compute layer, why did we chose Fleet of Ec2 with autoscaling and not ECS or Fargate? Keep making more system design videos 😀
Thanks Ojaswiankur for the kind words. I did this one on EC2 coz EC2 is still the most common compute layer. If you are comfortable with Lambda/Fargate, you can use them in the interview instead. I always suggest going with the compute layer that you are strongest in.
@@cloudwithraj makes sense..We want people to be comfortable with this and then can take to next level as they feel comfortable with..great stuff..
What do you mean by backend here?
I completed aws course i interested in job
why we can go with Kafka instead of sqs?
if there are a million users, how does the backend know which websocker connection is for Tina?
Great question Anh. For each websocket connection, the connection id is stored with the end user's cellphone number. When the sender's message comes to the server along with recipient number, the backend looks up the websocket connection id and send the message appropriately.
Hi Raj! You talked about use of microservices but you have only one database, microservices should have their own databases right? Like in this example, if you talk about 'Status' microservice then it should not talk to the main database directly right?
Great question Smita. For DynamoDB there is no concept of database holding tables. It's just multiple tables and each table can have separate scaling criteria. For microservices each API should "try" to interact with a non overlapping tables, but sometimes that's not feasible. Hope this helps.
bro wtf , this video is from 2021 but his hair style is from 2000s haha🤣🤣
Hi Raj, Many thanks for taking pains and teaching us. I have query, you are talking about microservices here.. However, in the diagram you are using EC2 has the back end where the application code is and using dynamo DB as DB. I am confused as to where does the microservices fit in here?
Great question Ankush. In this case you run one microservice in one EC2 and put it under it's own auto scaling group. This way each microservice can scale independently since they are under separate Auto Scaling Groups. Sometimes enterprises will implement more than one similar microservice into one EC2. But idea is to enable code implementation and scaling irrespective of each other. Hope this helps.
@@cloudwithraj Ok thanks Bhai... Actually.. Since Microservice is always linked to EKS/K8s in general and therefore, I have just forgotten that its still just an application that can be deployed on EC2 and doesnot have to be a container all the time :)
@@jetank2016 haha yeah, it is natural. Most enterprises move to EC2 first, then modernize to K8s. I purposely avoided K8s as backend to focus on the general design and not bogged by K8s stuff. If you are confident on K8s, you can draw the design with K8s as backend as well.
@@cloudwithraj Agreed. Super thanks..
Wouldn’t it be better to use Lambda over EC2 ?
Like I always say, in interview go with the compute option you are most comfortable with, all have pros cons. Choose amongst Lambda, EC2, and EKS based on your comfort level.
@@cloudwithraj Yeah, I agree. loving your videos btw. Recently designed a chat service for this project im working on and your design inspired me!
For Group messaging we can create room and add individual users to the specific room just like whatsapp. So if one message is in one room so that message will be part of that room and visible to the user in that room only. This high level idea, let me know how we can it scaleable. Look forward to the next video.
You are at the right path Rahul. I'd push you a bit and tell me what ideas you have to make it scalable, then I'll comment
Good Day Raj! Thank you very much for walking us through this awesome design! I'm just starting out in cloud, specifically AWS. I'm studying for the CCP exam currently and I was excited to be able to understand the services you outlined to make this architecture work! It all made perfect sense! I would love to see the group chat architecture solution for sure!! Again thank you so much, be well, be safe and cheers!
What is DLQ ?