System Design: Messenger service like Whatsapp or WeChat - Interview Question
Vložit
- čas přidán 27. 07. 2024
- This is a system design interview question asked at companies like Amazon, Facebook, Google, Microsoft and many startups: How to design a messenger service like Whatsapp, WeChat or Facebook Messenger?
I will guide you through this interview question, give you talking points and point out the right questions to ask. A quick and easy explanation even if this is your first system design interview question.
=== The resources I mentioned in the video ===
High Scalability Blog: highscalability.com
Hired in Tech: www.hiredintech.com/courses/s...
Music: www.bensound.com - Věda a technologie
I'm doing a little experiment over on IGTV, the SiT VLOG. Check it out! instagram.com/tv/BkYf4GphfQz/
Success in Tech great i will follow you. As. Miami.accent
Pls send me your no
Talking about the DB side could also be interesting. For example, to send push notifications, you probably need to know the device ID of the recipient. So you need some sort of database to keep a user id / device id pair. That means first time a user is connected or when they change the device, they will need to register their device ID to the server. Spices things up a little bit :)
+Muhip Tezcan really good point to think about, I like it!
Hey first I would like to thank you for making systems design interview videos as there are not many available out there. It would be better if you had discussed following issues which are very much expected to be asked in interviews regarding this question.
- What happens if a server dies before it delivered message to receiver or before pushing notification to sender?
- What approach should be taken to solve message ordering?
- How would we ensure message security? (Hint about ssl/tls)
- How sessions will be stored?
A little broad discussion rather than only high level overview is what I missed. It would be great to see more extensive discussion in the following videos. All the best.
Thank you very much for this valuable feedback. I will definitely try to incorporate as many of those points as possible in the next system design video. It's hard to strike the balance between too much and too few details (also given that the videos shouldn't be tooo long). Stay tuned =)
Standard system design interview is about 45 to 60 minutes long. Your existing videoes are about 25 minutes long. You can add another 20 to 25 minutes to take a deep dive on high level overview in next videos.
Success in Tech First of all thank you for uploading these contents. A suggestions from my side to the problem is by making 2 parts where 1st part deals with general overview like this video. 2nd part can focus about details like in this comment or handling corner cases or asking questions from interviewers point of view.
+Pinak Roy Chowdhury I like the idea! I‘ll come up with „deep dive“ videos for some of those topics, thanks for sharing!
Hi Farsan Rashid, nice questions you asked. Can you please let me know any answer on your first question "what happens if a server dies before it delivered message to receiver"? How are such scenarios handled?
thanks for making this video! In real interviews, regarding this topic, interviewers would like to hear more about
1. what database should be used, SQL or NOSQL, how to design database table
2. how to support group chat(it could totally change the design you're thinking for one to one chat)
3. how to scale up database.
anyway, thank you for your contribution!
Hi, is there a chance you to give answers to those questions? I would also like to understand these kind of applications from the db perspective. Thank you in advance..
Do you know the answers?
Great video but extremely high-level. Specially the part you explained how msg sent/delivered/read, you could have touched more details around the logic and algorithm.
yeah no way this is a good answer for an interview, even for a non-top tech company. I would expect a junior or intern to answer like that
Ah. Bob and Alice. Exchanging messages since the dawn of time.
lol, this made me laugh out loud! :D
Very well explained Ramon. Thanks for sharing and I am your latest subscriber as of this moment. Please continue creating these.
+C S Hood Of course =) New video coming tomorrow!
Nice introduction for basic discussions, but need to get more into details,
Message structure, which fields we have in a message, message types, messaging brokers cluster, Sync/Async architecture,
Persistency and cache, queue per user or one queue for all users (partitioning of queue if needed).
User management, and when we talked about Whatsapp , the main use case is WhatsApp groups which add more interesting complexity to the design.
What run in client side and what run in server side.
After basic introduction I think it is better to take one use case and give a design which fully cover all the pitfalls.
The ordering of messages it is interesting problem that should have handled here.
Thank you man, for this video.
Thanks Ramon. I'm enjoying these videos
+shelton paulsen Thanks for watching! 😀
Hello there, great video appreciate the effort you put into this! One suggestion / comment ; If I understand correctly, you utilize one queue per sender, where messages are stored. Wouldn't it be more efficient to utilize one queue per receiver? That way, when Bob is connected, only one queue has to be examined for his messages, while the other way around all sender queues have to be polled for messages directed to Bob. This also alleviates the fact of out of order message delivery, since you can always store messages queued by a time stamp.
Further, you said that connecting to different servers can be done efficiently. However, I do not understand how, when Bob connects to a random server, all other servers containing messages for Bob will realize that and start delivering him his queued messages.
I know that design is highly objective and tradeoff driven just wanted your opinion on this! Thanks!
Hello re Andrea, exeis thavmastes!!!!
i agree, if bob never gets online, this should not block all other messages.
Cool, could you cover the video about the technology used in this system, like wether using xmpp or websocket, use sql database or no-sql...
Thank you for publishing this video, very well explained.
Thanks!!
Doesn’t system design require specifying the hardware required, for storage, processing, communication and the like? How is scaling done? Just add more servers? How do the servers communicate with each other?
Awesome video. Can you make a system design video on voice-activated system like Alexa or Siri and Airbnb.
Great video, Ramon. Well explained and informative. Thank you.
njb2226 ramon ))
Thanks for the video. I have a question regarding delivering messages from different servers to Bob. When Bob goes online, how servers get notified that Bob is online, because he gets connected to one of the servers. Should a server notify other servers? Thank you
Great explanation!
Thank you for this video! You mentioned that the connection between the users and the server will be TCP. Are you thinking XMPP over TCP? If we were thinking about Whatsapp in the browser (instead of native apps), would we consider XMPP over WebSockets? I don't know much about either XMPP or WebSockets, but I've seen these used in other system designs for messaging problems, so I'm curious about them. Thank you
Francesca Guiducci I don’t know the answer to your question but for clarity xmpp is really built on top of tcp
You touch everything very briefly and the explanation is very shallow for an actual interview.
Hi thank you for all the explanation.
Regarding fire and forget pattern, I guess that’s about synchronous transaction as sending request and stop without waiting for any return, while being called back after the request is finished.
Great video. Thank you.
You mention using WebSockets as the preferred route between the client/user and the N1 app services. What would Native Apps do?
My question is, how does Bob know on which servers there are informations for him?
1. Does he connect to all servers to see if there are messages? (Highly inefficient)
2. Are all servers informed that when he goes online so they can send him messages if they have? (Also highly inefficient)
Somehow all servers that have information for him need to be informed that he is available, without bothering the other servers.
Good questions. I think for the first one, a kind of sharding based on the user ID could be a solution.
@@titoluzuriaga8217 I think the problem with that approach is that we want each server to be capable of serving everyone their messages
When Alice sent message to bob but bob is offline, the message will be stored in the queue in server. And I think there are also a server to save some metadata info: A sent to B some message and stored in server 1. When bob login in, his client will sent notification to the metadata server, the metadata server will notify server 1 to release the message A sent to B.
Amazing & Amazing...!!!
Thanks man!
Great quality content, thanks man and keep it up!
Thank you so much! Please share the channel on your social media if you want to support it =)
Great video. Thanks a lot for taking effort and uploading it. Can you help me understand does A and B needs to be connected to same server, if not then how the message which is received from A by say server1 reaches to B say connected to server3? And what can be done to make server1 receiving message fault tolerant such that message is delivered no matter server1 remains part of cluster or not? Thanks!
Great video..just one question ...how does the server know when any of the client comes online...coz you mention that a lot but what is the mechanism to identify an online user..polling would be too heavy I guess
i imagine the client would heartbeat to server every few seconds
Hi ,
You mention about the race condition where the order of the message being received in the wrong order , so it's a good thing to mention such sort of thing in the interview ?
You are good, man
thanks for effort and time
Very grounded presentation. Congratulations and thank you for share with us!
Thank you, Juan! Appreciate the kind words
@@SuccessinTech hey, do you know any IM designed over blockchain technologies? I'm curious about an IM serverless platform.
Hmm no I don‘t. Makes me curious though!
@@SuccessinTech send me an email to jp.the4ducks/gmail to keep this thread
Hi Ramon,
Thank you for this video. I have one question though.
Since you are putting messages sent by Alice in his queue, then what if Alice wants to send another message to some 3rd person lets say John who is online. But message for John would appear in 2nd position in Alice's queue as 1st message is for Bob (who is offline, now). In that case, 2nd message can't be delivered until 1st message is delivered since you are using a queue (FIFO), which shouldn't happen. So, I believe, we should have a persistence layer for un-delivered messages with receiver's ID and as soon as receiver comes online and connects to the server, then the server can read pending messages for that receiver from that persistence layer deliver it to him on FIFO basis. And in this case, messages to Bob and John would simply be there in the persistence layer with no blocking unlike the above solution.
Please let me know your thoughts on the same.
Superb . Yes definitely enjoyed the video.
Hello Ramon. . . I loved this video . . . Can you please tell the server which is used for messaging services?
1. One: one - This video is awesome
2. Thanks
Commentcava you inerid van hmmmgn
🥰commentcava inerid vanemmen
you need more subs you're very helpful
Tell your friends! 🤩😁
Woah what a great explanation. Thanks!
I would scope each chatroom to 1 server, and queue up all messages using 1 server so that all messages are ordered (important). we can do this using the hashed chatroom ID and a load balancer (in case the chosen server is down). i would also utilize websockets and broadcast events to certain topics when a message is sent, as well as saving to the database. when a user opens up the chatroom i would update the "last_read_at" and mark all "set message.status to "READ" where sent_at < chat_users.last_read_at "
there's a much simpler and elegant solution - simply add a timestamp, or better a counter to each message and sort on that upon receiving.
What if we want to keep the ordering of the messages? Should the client reorder them by timestamp?
Thank you so much for such a nice explanation. Just one small suggestion if you guy can explain about class level design after high level explanation. Hope to see more such videos.
+Adarsh singh Yeah, for problems which are aimed at class diagrams I'll go into more detail. Thanks for the feedback!
hi, can you please upload a video for designing instagram ? also explain how would we go on developing this as a standalone application first and then how would we go on scaling it ?
Nice video! How can a server recognize if a peer is online? How do they manage the opening of a new connection to deliver a read/delivered message since the receiver can be behind a nat? Do they use persistent connections with websockets opened client side?
in a scenario where there are millions are users how many queues will be used. Also how the search of messages will work within queues?
I didn't get the part where you want search within queues. I don't think that is efficient without use of another data structure. Anyways, I couldn't get the use case.
"Even after Facebook bought it, they are still advertisement free." LMAOOOOOO
What guarantees that push will be sent before the message is delivered? Or both will be sent at the same time. Because according to the design, a different server is handling push notifications. What if push is sent after the message is delivered?
Possible. Those systems are optimized for eventual consistency.
Short and sweet
Thanks for the video. One thing that wasn't clear is how does Bob get the message after it gets sent by Alice and queued on the server. Does Bob poll the server periodically or does the server somehow maintain a connection with Bob? Please clarify, thanks!
Thanks a lot
You‘re welcome! If you want to support the channel and future content please share my videos and spread the word on your social media =)
Really cool videos man
Hey! Thanks for explaining this type of architecture. can you describe
1. why you prefer TCP over HTTP Rest API for sending messages ? we can get response in terms of acknowledgment and other plus point is while sending file to server we can easily can progress.
2. About push notification, you described general FCM/GCM but the main thing is that when bob sent message to Alice and Alice was offline at that time now when Alice comes online he receives notification. so in this process how server decides to send push notification for that message or deliver the message on other way because its online?
Thanks
If the servers have queues holding the messages that couldn't be delivered, how do you immediately deliver a message from the queue as soon as the user comes online? Basically, when a user connects to the server, how do you find all his messages from the queue and deliver them to him?
how you'll manage if the message is read and your push notification arrives ...as both are not in sync ...(though it can be handled at app end)
I have a query - How do all the parallel servers maintain consistency? Maybe there is a load balancer between the servers and the client that facilitates that.
what happens if Bob and Alice are not connected to the same server? That's crucial here.
+Sazzad Hossain No it‘s not, that‘s the nice part of this architecture. They can use totally different servers or even switch between servers during their conversation. This is what enables us to scale horizontally.
Success in Tech if a message from alice to bob sits in alice's queue in server 1,with bob connected to server 2, would messages need to be copied between servers or would you expect bob to switch servers. How do you deliver a message from a server to which the recipient is not connected?
aneessw servers talk to eachother, there are many ways to do it, ex gRPC or messaging systems like AMPQ, or managed DBs like firebase
Exactly. That's a good question. Topic which IMO is not tackled in this video is deliberately not tackled is, how the server is sending the messages to the recipient. In order to this, there has to be an open connection between the clients and the servers. Then, because of that reason it actually does matter which servers are the users connected to and therefore there has to be some intermediate layer that enables the interchange of data between the servers (gRPC, AMlPQ, Firebase etc.). The other solution for that is to use a third party to send data - e.g. google messaging system mentioned in the push section. Anyways, then the architecture problem is still there, but now "sold" to the google system, which actually, is not so much different from the "sending" part of the simple messaging system being designed here.
What I'm lacking from this video is how to tackle either the push part from whichever server received the message or the communication between the servers that both parties are connected to...
In a distributed system , message processing should not be restricted to a sticky session or a single server. The queue messages should be flexible among the servers , any of the servers can pull from the queue and push it to the users
Hi, If 2 clients A & B and A is served by server1 & A by server2 then how the communication/Co-ordination happens between the servers?
How would bob’s device know on which server alice’s queue is present?
In your diagram you had Alice send the message to the Alice queue. Shouldn't that be Bob's queue?
During such interviews do they ask for what technologies would you use ? i.e. what solution would you use for the queue ? or the server ?
Yeah possibly. I‘d recommend talking about technologies you actually have experience with instead of name dropping 10 different tools you heard about.
Thanks for the video lesson!
Had a small question...why did the guys at WhatsApp leave the race conditions of different servers unmanaged ? Does it cause significant delays or performance issues?
+Hari Mohan what race condition do you mean? Afaik Whatsapp tries to optimize for throughput (concurrent open connections etc.), something to keep in mind when thinking about their architecture.
Very good approach. Could detail partitioning and fault tolerance a little more. Also, it would be good if as engineers we remove the words master/slave from our vocabularies and instead say primary/secondary (w.r.t. the redis conversation).
Queue solution will be pretty bulky one. Assume per second 1 billion messages are being transferred. Will you create 1 billion queues? Wont it explode server? And since queues are supposed to be in FIFO order, once Alice, bob record is processed , then only John , doe record will process. This will make the last person(last in 1 billion) who sent message, other part will receive it after 1 hour or so :)
Would it be reasonable to use Kafka for the message queues?
[12:10] Maybe my question is naive, but can we not use web-socket instead of opening new connection between server and client to send the delivered and read notifications?
Is there any side-effect of using web-socket?
What if one of the servers went down holding messages from A awaiting B. Would they be lost? Also, how can a queue be specific between A and B? Are you saying the queue is like a KV store or something and the mesage for B is queued as "B". Otherwise, how does the server discern which Q'd messages go to B vs C vs D? The implementation of the queue is not very clear to me.
Also - when Bob "comes online" how do both Servers 1 and 2 know this and communicate with bob? Again - wouldn't a persistent store be better here since the messages are in one place and we wouldn't need to check all N Servers for their ephemeral queues.
PS thanks for vid!
Ramon, great job!
Do you have a video lesson dedicated to Load Balancing latest techniques?
This is a great video. I have one question: how does the server know whether WhatsApp is in the foreground or not? Basically how is WhatsApp or any messaging platform able to show whether user is online or offline independent of login/logout? For a website (Facebook, LinkedIn, WhatsApp web, etc.), does the server ping each user every few seconds for knowing online/offline?
Follow up: WhatsApp shows whenever a user is typing. Does an event get triggered on every key press of user A which passes the conversation ID (group X ID or user B ID) to the server, and server then informs the group X or user B that user A is typing?
Thanks for the great video bro, stright to the point (= Keep up sharing the great work with us. Looking forward to new videos. All the best from Sweden.
This is fresh grad or junior engineer level explanation
One more requirements is how chat messaging whether user is online or not ? where messaging server still the data when user is offline . i think we can have more on which data structure should we use like nosql or redis for undelivered message.Just thought
Thanks for the video. But isn't the TCP connection between server and sending/receiving parties a Web Socket connection instead?
en.wikipedia.org/wiki/WebSocket
Hi @sucess in Tech. Could you please do the series video on this whatsapp design to explain the database-table and its mapping. Because i am very much interested about to know how the Alice sends messages to his contacts? and how the message history get tracked
Thanks for the Video. I have one doubt what happens to the messages when one of the servers dies and queues are also lost?
Nice video, you mentioned a possible race condition on message delivery given difference serve loads or circumstances but not really a way to address this. Any suggestions?
Probably using an incremental is for each message inside the conversation, the receiver will be able to handle this.. delaying the first arrived message or showing a message to the receiver explaining the situation...
I get push notifications again and again (Even if I clear them without reading them) on my WhatsApp. I'm using a Mi mobile set. My friends using other company mobile phones does not receive it once they clear it. What could be the reason for it?
Can you explain how to fix the race issue between servers and deliver messages in order, you were discussing at 10:00 ?
+Rakesh Ranjan An In-Order message guarantee without compromising throughput is hard (maybe even impossible) to achieve because it requires some sort of coordination. Messaging systems like Kafka or Googles PubSub favor throughput over consistent order. What you could do to fix the problem on a very rough level is reordering messages on the recipients device based on a timestamp which is attached to the message (this could lead to confusing UX). Keeping your queues healthy and therefore fast is also a good way to deliver as ordered as possible. I'm sure there is a better solution out there, if you find a good resource, let me know!
Could be send second message to server once received first message delivery success acknowledgment.Till then cache all the messages in client side. When internet reconnected or server not busy. send messages from cache.
+Suresh Babu Can you elaborate on that? In my understanding you would be taking away the possibility to scale out horizontally.
I mean. Send first message to server.
If (first message delivered success)
then send second message to server.
else
{ cache second and further messages.
Retry cache messages later when we receiver ack for first message.
}
That still doesn't guarantee that the servers which handle your messages deliver to the recipient in order. Unless you wait until the server confirms every delivery in order which adds overhead for your servers and slows down messaging.
Hello! In case you would need to use a DB to store messages which one would you use? SQL or NoSQL? Cassandra maybe?
Whatever fits your specific use case, existing knowledge in the team, personal preference =)
Thank you for your response!! I am asking because it is not always clear which type fire best... for example in this scenario we could use RDBMS for small number of users an in case we have a big system we could use Cassandra or perform sharding (best in the location maybe) in the initial RDBMS. Maybe you could make a video based on DB servers! Either way thank you for the amazing work you are doing!
So does this mean Whatapps stores old messages in the device? strange, because it lazy-loads messages when you scroll up, which means they are not readily available on the device.
Hi.. For the read receipt, we send a special kind of message with the source messageid to a different server. Where do we store this messageid? In the client? and how does the app know the it has got a read receipt for a particular message.
+TheoPaul Antony Read receipts could behave like any other message (follow the same logic). Once they are delivered to the app the vanish from the queues like any other message. Thats the approach I described in the video.
Hi Ramon, I am still stuck with the idea on how even if both bob and alice are not on the same server, still receive the mesaages in real time.. Could you please guide me..?
+Shuvam Bosana The servers deliver as fast as possible, they don’t care if other servers have messages for the same recipient. So all the servers are interested in is knowing how to reach the recipient and empty their queue. Look into web sockets for that matter.
Could have touched upon Audio / Video calling in Whatsapp. That is beyond the simple message sending mechanism.
If Alice send message m1 which is sent to server 1 . And he again send a message m2 which this time goes to server 2. Then how would we ensure that the message delivers in the correct order, that is, how we maintain the order of the messages if there are multiple servers??
+Vishesh khandelwal We don’t. This is the trade of with this architecture as pointed out in the video. There are some mitigation tactics you could do (like delaying delivery to wait for incoming messages with an earlier timestamp) but those can only do so much. If you care about guaranteed ordering you would need something like a master/slave setup where a central component dictates ordering or maybe something like Consul does where they use a highly sophisticated coordination protocol based on Paxos and a form of gossip. But all of that is not trivial and comes with the downside of increased coordination overhead.
Or will it be possible to order the messages based on the timestamp at the client side once it is delivered?
+Shuvam Bosana Yes see a couple of comments below where we discussed this. You can rely on timestamps but there is always the possibility of showing messages in the wrong order, namely during the time not all messages have arrived from the different servers.
How does BOB and Alice knows which box to connect?Since there is no load balancer between servers?
we can simply send with the massage to bob an ip address of alice and if bob opens that massage which contains a hidden alice ip and alice is online print read in alice chat's box
I have a doubt what if, Bob message for Alice was stored in Server 1 and Alice got connected to Server 2?
10:20 how can we fix this wrong oder problem then ? :) should we also record the timestamp ?
timestamp attribute for each msg is a "must"
The message queue should be a receiving message queue, i.e., every user has a queue of received message, instead of every user having a queue of sending message. e.g, if Alice sends a message to Bob, essentially the server will write the message to Bob's message queue, once that done, nothing is related to Alice any more. Also these message queues of all users can be maintained in a centralised db system, sharded by the user id.
This is better and clearer and also avoid the problem of messed up order of messages when receiving
Do you have an example of what kind of message queues you are referring? Also, the use case has millions of users, so are you expecting millions of queues to be maintained?
How any server can send chat messages to users? you need to mention socket based connection or polling mechanism from client app; if its socket connection, then client will be connected to some server through websocket; it can't just connect to any of the 100's of servers. Or the client can use Pub/Sub mechanism from systems like Kafka
Why does servers have to be involved with the "Push notification" feature? I thought that the server's job is to send the message to the app currently running on the phone. Once it's there, it's the job of the app to push the notification !
Content and notifications are usually decoupled on modern mobile platforms. The message could be large (or a file) which you can‘t transfer to the phone in the background (OS‘ prevent that for many reasons like battery drain, data usage etc).
understood. @@SuccessinTech
I feel like most interviewers would drill further into the incorrect message order issue. Would be nice to have kept going down that path.
Well, the client would connect to one of the message servers at first time. So you could have one of following situations:
1. All message servers have access to the delivery queue of all users' messages. In this case there would have no problems with delivery synchronization.
2. Each message server keeps their own delivery queue. So each time a user connects to a message server, a synchronization process would begin. The message server could ask the others if they have messages destined to the connected user, if they do, then the current message server could download the messages, sort them and send to the client application, or the server could send the localization of the messages to the client, which would download the messages of each server and sort them.
Your videos are really informative!!! Thank you for putting up such great and helpful content!
If whatsapp deletes messages when they are delivered, then how is it that many times police ask whatsapp to provide chat details during crime investigations?
I am a bit confused here. We get push notifications when we receive messages. Why do we need to think it as a separate topic? When the user receives a message, simply show a notification. Am I missing something?
Think of sending images or files. You want to be notified via push before your device starts downloading them. Thats just one reason why notifications should happen through a different channel. Also on mobile OS‘s push notifications have to follow certain restrictions.
Which things need to create messenger like WhatsApp and WeChat .
Hey man, were you in Amazon.. u were talking of dive deep
Does anybody see the huge difference author talking about between messages and push notifications? Why no word about the way of selecting the right server for sending and delivering?
thanks. i have a Q. i thought the message gets delivered whether or not someone's is online.
No data model, no data volume estimation, no real scalability discussion, no real failure recovery discussion, I'm not sure what other people are praising this video for
A lot of major components are missing? Caching / Load balancing (load balancing mechanism ? ) / Hashing / Indexing . Also App Server and Queue mechanisms totally seems mixed. I think this video could be confusing and misleading for many people.
Great video. Would be easier to hear if you used a lapel mic. Camera mic is not great.
If a system design approach of a chat application doesn't design the solution for the messages to be in order, then it will never be accepted. Just saying, we can mention it to the interviewer that messages can be out of order will never work. The solution needs to cover how we ensure (or at least try to ensure) how messages are linearizable. This is THE most basic feature of a chat application.
The other features were also very off-handishly discussed. Moreover cases like
1. If the other user doesn't come online for several days, what do we do with the messages in the queue?
2. What is the scale of the service we are going to design?
3. What will be the DAU? What will be the average message size? How much would be the queue size then?
.
.
.
etc.
I really liked your video on Twitter system design but unfortunately this wasn't up to that standard.