Basic System Design for Uber or Lyft | System Design Interview Prep
Vložit
- čas přidán 7. 06. 2024
- Visit Our Website: interviewpen.com/?...
Join Our Discord (24/7 help): / discord
Join Our Newsletter - The Blueprint: theblueprint.dev/subscribe
Like & Subscribe: / @interviewpen
This is an example of a full video available on interviewpen.com. Check out our website to find more premium content like this!
Problem Statement:
Design a high-level system for a ride-sharing service. There will be drivers & riders requesting rides:
- Rides select their pickup point on a map & can view ETA + price data
- Riders must be able to pay for their rides
- Drivers & riders are matched, drivers accept rides
- Riders receive information about the driver, driver location updates in real-time
Table of Contents:
0:00 - Requirements
0:30 - High-Level Overview
1:10 - Tight Coupling!
1:42 - Visit interviewpen.com
2:01 - Event Bus
2:34 - Rider & Driver APIs
4:14 - Database Data
6:01 - Database Structure
7:57- Map Infra
8:39 - Payments
9:46 - Dynamic Pricing
11:40 - Rides Service
12:19 - Uber's H3 (Global Position Indexing)
13:14 - Matching
14:22 - Next Steps
15:43 - Go to interviewpen.com
Socials:
Twitter: / interviewpen
Twitter (The Blueprint): / theblueprintdev
LinkedIn: / interviewpen
Website: interviewpen.com/?...
Thanks for watching! Visit interviewpen.com/? for more great Data Structures & Algorithms + System Design content 🧎
Need basic design system for PayPal and Braintree 😮
THIS IS PURE GOLD!
It just somehow magically landed in my recommendations and damn, I'm so grateful it did. Just 2 minutes into the video and was convinced to subscribe!
So thankful to Bobby, he has a great future ahead!
Thanks for the kind words! Have a great day
same for me too
Same thing happened to me, I'm grateful for that!
I can't believe what I just saw. This man is literally what I want to become in the long term. For me, this video has set up my end goal as a senior. What an amaizing video. Thank you so much. New subscriber here.
Thanks for the kinds words - and thanks for watchinf!
This is not "senior" by any stretch of the imagination. Any junior / medior engineer can understand and explain these concepts.
@@DavidBcc Welp, I guess I'm way more junior than I thought.
@@DavidBccno they can’t lol
I swear there’s always gotta be someone like you
@@gewdvibes100% 😂
These system design videos are awesome! Really interesting topic with good visual explaining, keep up!
Thanks! We have a lot more coming
Great content. Please make more videos. Content is king. Even though there are tons of videos on Internet about System Design but the kind of clarity that one gets from these videos is unparalleled. It's to the point.
Worth watching again and again.
Really glad you enjoyed it, we've got more content on the way. Thanks!
This channel just came at the right time, when I am doing system architecture, thank you and keep them coming you are indeed contributing to my career growth
ok! we'll post more and more! building a team rn
This is so good. I have searched for this type if videos but this is very very good. Clean fast forward explanation with much detail. Thank you!
Appreciate it, thanks for watching.
This channel is easily gonna get a lot of subscribers, great content.
Thanks! We'll be posting more!
I'm not a software engineer or developer, but you just had my eyes glued to the screen for 16 minutes. While I might not understand the concepts in this system design, the way the information was presented was extremely well done and approachable for a wide range of viewers.
Thanks for the kind words!
Totally agree. I love how he put things together and it all makes sense to me and makes me want to learn more.
no offense, but the reason you like this is exactly because you are not an engineer or developer.
Almost everything seen in this video is obsolte and just a nice visualization of what the process of creating a real app looks like (which is done without all the painting)
Loved it. Well structured and comprehensive!
thanks for watching!
Mate im just speechless, you earned my subscription and this is one of the best videos ive ever watched
Thank you!
CZcams reccomended this video to me and 2 minutes into the video, i subscribed to the channel, i can't let this kind of content pass me by, please do more.
Thanks! Glad you liked it.
I cannot believe you've placed many valuable topics in a 16-minute video! This was worth more than a 12-hour course! Thank you!
Glad it was helpful!
I've seen lots of system design prep videos on youtube especially cause I'm being interviewed right now for Senior BE position and this guy really shows valid ideas on required abstract level. For everyone who has doubts, "yes" this is proper level for senior position and his conceptions are valid for every-day usage with this level of abstraction.
Thank you!
This is one of the most educative videos I have seen in a long time
thanks for watching!
This guy is really good at teaching, very informative and concise, thanks.
I appreciate that! Thanks for watching!
One of the best systems design videos I've seen! Great job!
Thanks!
The way he explains everything is just pure gold
Thank you!
pretty decent take - I've seen some comments on specific improvements, but narrowing the scope of a given problem on SD interviews is A MUST, otherwise you won't make it inside the given time frame. functional/business prioritization on this case was very well done!
Super good point. This was very broad; usually a system design interview will drill into a specific part of this much larger system to see how deep your understanding goes. Thanks for watching!
You're very good at explaining! Thanks for this
Glad you enjoyed it!
WxCtly what I needed. I don’t have time to learn syntax, so I generate all my snippets but I need form of macro assembly training to manage larger models thanks a million
awesome effort guys please keep up this momentum!
Thanks!
This is absolutely perfect! Thank you so much for sharing!
Appreciate it, thanks!
I wish I can lecture the interviewer like this. Very well articulated.
Thanks!
I did not know that spark could be used to solve problems in this kind of project, very informative and detailed explanation thank you.
Thanks for watching!
instant subscribe over here. very clear information, amazing content. Keep it up! Thanks
Thanks!
These videos are so good
Hope you post more
We will! Stay tuned - we will be posting weekly (try our best to).
This is super cool man, thanks so much. Just a small feedback, I clicked on the video because I saw a really nice diagram. I'm a highly visual person, probably like a million others and when I saw your drawings I was a bit dissapointed, not because they are bad but because studying with other people hand writting definetly slows you down like "what does it say here?" Don't take this the wrong way, you are a genius, keep up the good work.
Glad you liked the video! Yeah, I know my handwriting isn't great, and I'm striving to improve that always. Thanks for the feedback.
@@interviewpen keep up the good work, you gained a new subscriber
Mind blown... this was amazing to watch...
Thanks!
These types of videos imo are the best as these can get one out of “tutorial hell”. Although challenging - I think it forces me out of my comfort zone and actually makes me think on how to program and shows me gaps in my knowledge. New subscriber - I would like to see more of your videos!
Thanks for the comment and thanks for watching 👍
Haven't started watching the video yet but I know I have to subscribe!
Thank you!
Def gonna buy interview pen!! Awesome stuff!!
Thanks for supporting
This is a really good explanation. Thank you for sharing your knowledge.
Sure - thanks for watching!
Good analytical skills man. Congrats!
Thanks!
After seeing this I am definitely buying a subscription in your webpage
Thanks!
thank u for making these videos. love to see how u would implement these in code
sure! we can experiment with that - will be releasing more content
Thank you for the this content... What a pleasure!
Thanks for watching!
Very well done. Cool.
This video is amazing and was recommended at just the right time for me. Subscribed within 2 minutes of watching!
Thanks for watching 👍
Insane! Thank you
Sure - thanks for watching!
Excellent architecture work there bud.
Thanks!
Love this!
thanks for watching - more videos coming soon!
Got me subscribed! Nice.
👏
Great content here. Subscribed
Thank you! Glad you liked it.
Amazing content. Instant sub.
Thanks!
I feel like you can make a whole series on this one problem man, would love to see an extended version of this where you go a little slower and into more detail. Either way, this video is super useful!
Glad you found it useful!
I agree! Typically during interviews, we'd take an hour to cover one of the many key concepts he addressed here: efficient data storage and retrieval at scale; distributed systems and comms; buy vs build tradeoffs; algorithms to for efficient search at scale; etc. I was very impressed at how good he was at packing all that info in under 20mins, and conveying it in a way that's easily digestible!
Fantastic video.
Thanks!
tbh I am very impressed about a content, that autor produced despite of his young age he has a lot of knowledge in projects and it's greate! I found a lot of interesting things. Thanks a lot!
Thanks, glad you liked it!
Thank you, it was interesting and informatively
Thanks for watching
thanks for the information.
sure
This is a fantastic video - lots of content in a short space of time delivered clearly - thanks!
One question I had was regarding storing of driver locations and loading them into H3 - how would that be accomplished using the DB design here? Or would the H3 index constantly be updated separately? Thanks.
We showed the two databases separately to show that they can be decoupled, but they certainly could be done in the same one. However, the index would still need to be update separately either way. Thanks!
Visual explaination was comprehensive
Thanks for watching!
You got 1 more subscriber. Please keep posting with such a great explanation
ok, thanks!
im so blown away
Thanks for watchinf!
Thank you. This is amazing
Thanks!
Cool stuff! Anyways, I think it is very important to get the requirements as clear as possible before jumping into the design. Ask questions, make sure you are not assuming too much. This solution could be way over-engineered for some cases and it might rise some red-flags among the interviewers.
Yes, you're absolutely right. This is a super important first step (and our more recent videos try to be better about this). Thanks for watching!
Awesome!
Thank you! Cheers!
I was asked to design Über in an interview about a month ago, if only I came across this before
This is amazing! As im in a junior developer, this video is inspiring me with verious concepts of the highly efficient architecture in real time situation and suggesting ultimate goal! Thanks for sharing this great insight!
Thanks, glad you liked it!
This is Gold
Thanks for watching!
Love this video
Thanks!
You know ball dude, keep it up
Thanks!
Good stuff :)
Thanks!
Great video
Thanks!
I've heard Uber uses a structure called a Quadtree to split their map area into grids for easier rider - driver matching.
Yep, quadtree is another data structure that can be used instead of the H3-based approach discussed here. Both have pros and cons. Thanks for watching!
Uber uses H3
great video
Thanks for watching
This is awesome! Could you kindly do a video on how to work around the same, like making this a complete project maybe. Thank you
It's tough to cover every part of systems like this in detail in a video (Uber has spent years building their systems), but we try to give you the core foundations you need to get started and to approach similar systems. Thanks for watching!
@@interviewpen Alright, it is still nice, let me try from here, thank you
one organization improvement I'd make on the diagram, not only for readability, but also for a better notion of resources management, would be delimiting some domain/namespace boundaries, organizing the services appropriately:
- Client APIs
- External APIs / Services
- Events Bus
- Internal APIs / Services
- Data / Storage
etc
Good point, thanks!
Excellent vid! Thanks!
Question: What is that drawing tool being used..?
GoodNotes - thanks for watching!
excellent
Thanks!
good information. im subscibe you.
Thx 👍
It was just amazing, I really wish there was some course.
Thanks for watching! We have a full system design course on interviewpen.com :)
You inspire me mehn
Thanks
Beware that the second you mention a pub/sub system in an interview you may have to touch on the topic of message ordering. The immediate scalability of kafka event streams seems great in paper but if you have to process things like chats in order you may have to make kafka store all the relevant events in one partition which isn't scalable.
Great point. In this system, message ordering isn't a huge concern, but there are certainly situations where this is an important consideration. Thanks for watching!
Mantap bang makasih
sure
Hey, first of all great video, great content, exceptional delivery - seriously wow.
Around minute 6:20 of the video you correctly say that the DB will have to be able to scale horizontally as it is expected to be very large with high traffic coming in. You said, the easiest way to do it is by sharding, which left me wondering, did you consider a noSQL option? Clearly it is easier for horizontal scaling. If so, why did you decide to stay with the relational approach?
I don’t think sharding implies a relational database, in fact you’re absolutely right that NoSQL databases are far easier to shard. Good thoughts and thanks for watching!
like it these architecture use case solutions
Thanks
very good
Thank you! Cheers!
Thank God I found u
Thanks!
Hey @interviewpen, great video, thanks! One question I have is why did you go for the server side API app with load balancing rather than an API gateway to access Rides (and maybe other services) directly? I know load balancers and gateways are not mutually exclusive, just interested why you went with one app routing all the client requests?
Another question is about payments, will the user get payment confirmation on the client side as well? You show in the little diagram that user will receive the confirmation after the webhoot will send the message to kafka, so the server side of the app will be subsribed to the topic of payments filtering by user_id (for example), but how would the client side receive the confirmation afterwards?
Thanks for watching! Usually when people talk about API gateways, it's a nebulous term that probably means a load balancer. If we want to scale our API, we need some way to route our clients to one of several nodes, so a load balancer is critical. About payment confirmations, this is something we could set up by sending a message to the client over WebSockets after the server receives the message. Hope that helps!
It is very important to me levels to development the system like uder
Cool cool, thanks for watching
Amazing except I needed this 5-6 years ago 😂
Thanks for watching!
great video. What software did you use to make the diagrams and presentation?
GoodNotes 🙂 thanks for watching
Thanks for the video, it's very educative.
I didn't fully get how global indexes are working. Would appreciate it if you can elaborate on it.
Sure--a global index is essentially a copy of a database table, but organized differently. This allows us to query the data in different ways efficiently. Hope that helps, thanks for watching!
It looks like a rouiting component is missing. Pricing clearly depends on the route length (the mentioned surge just scales this price up) and, sometimes, on traffic jams. You can't calculate ETA without a route. And most client UIs draw the route on the map.
Routing algorithms are hard, and we abstracted a lot of this logic away. The ETA service will of course calculate a route to get the ETA, the pricing algorithm must take a route as an input, and the route must be sent to the client to display it on a map. Thanks for watching!
I’ve watched quite a few of your videos and I see that you mentioned websocket, but you don’t elaborate about the impact of that in terms of how to scale it. Thank you 🙏
Thanks, we'll consider doing a video on that!
Fam, speechless.
Thanks for watching
Please upload more video of system design
ok! Thanks for watching
Really great video! How is the calculated price communicated from the Pricing service to Stripe?
There's a few ways to do this; the simplest would be to push the price into the trips database before processing payments. Glad you enjoyed the video!
lisan al gaib, he is the chosen one
This is great, now how do I explain requirements to someone who has no idea about uber/lyft or applications in general? You have good intuition here used to synthesize requirements and identify possible issues immediately. That intuition comes from knowing what you're trying to build, for new product development where the final product is not clear, how could we handle that?
You're right, we should've gone through the requirements first before diving into the solution. Our newer videos are much better about this! Thanks for watching.
excalidraw my dude :)
One critical part that is missing in this SD interview is, discussion about different trade offs. e.g. in the matching service, why use Uber's H3, what about quadtree, geohash and Google's S2, what's the pros and cons of using these different methods.
Good point, there’s tons of options for geospatial indexing. Thanks for watching!
great video, why a monolithic db? should the microservices not own their own data?
Most of the microservices in this design do in fact own their own data! The core database is only being used by the driver and rider APIs, which of course need to see the same data. Thanks for watching!
FYI (for those interested in project and program management) systems design and project management works hand-in-hand. Technical Project Management is breaking these down one by one, little by little then coming up with a solution - that's technical project management. More often than not, technical project managers help the engineers by solving what they cant: meaning issue X is more of a devops issue than an engineering issue, the tpm looks at the problem from an engineering standpoint, asks the devops team how to solve it, creates a plan, budget etc. If you're into firefighting all day, get into TPM, we need lots of people that wants to fight chaos.
I think this is kind of a intern/junior level software design, which is good, but certainly lacking in most details that these systems require. For example, when you talk about sharding the database, it's not so easy to say just shard it. Some of the problems with sharding a db using such a key in this way would be referencing and constraint enforcement as well as update request stream consumers and their ordering.
Also, your "rider and driver APIs" discussion is...flawed. Most of the system design for those APIs should be about actual data modeling and api design. You also can't just say to use "websockets" because it has some very bad tradeoff. For example, sockets are not stateless and requires consistent connection, which you will definitely not have in a driver's mobile phone. Another problem is data consistency. If your data producer sends a message to the socket but the receiver was in a temporary disconnected state; what happens to that request? Also, since it's websockets, who is doing the broadcast? And since cars can "come online" at any point; does the requester keep resending the request? If so, then isn't polling necessary anyways? If the concern is concurrent request scale, it's much easier to shard the api using the same H3 algorithm, isn't it?
There are a lot more problems with this design, but for a student and assuming an ideal world, this is a good first draft. I would say though that it would be much better to assume those ideal conditions (perfect and constant connectivity, some scale target, some location target, etc), and then to better simplify your solution. I feel like a big problem with this presentation is the use of buzzwords without knowing how they work or what they mean. I'd rather you use fewer techy sounding words and be more accurate with your solution because your interviewer will certainly see right through these buzzwords and you will get drowned with real world problems that you can't explain (like the Kafka network limitations that makes this just not possible as described)
Also, for this channel in general: this is not a good interview question for system design. Usually, system design interviews are more for mid level engineers so doing it at this scope is not good enough. When I interview engineers for system design, it's okay to start with a broad system question, but it's much more revealing to dive into one specific area because the complexity of a problem is when you have to design in the details. I think even just the payment processing is sufficiently complex to deep dive let alone trying to design every system in Uber.
edit: I'm a senior software engineer at a FAANG and have conducted 100+ mid level engineer interviews.
We call these Activity Diagram in UML/SysML
Cool
I am not even a software engineer, but this was interesting to watch. The seemingly simple applications that we use in day to day have complicated backends. Hats off to the engineers.
Thanks!
Would you go for SQL or NoSQL based DB here? Great video btw!
Personally, I like NoSQL databases for these types of systems since they're inherently designed for sharded data models where related data is stored together. However, it is also possible to use a relational database for this and some developers prefer that approach. Thanks!
DB sharding by the Trip ID means you're going to use a separate database for each single trip, isn't it a huge resource consumption?
Good question. With sharding, we can use the shard key to algorithmically determine which DB node the record should be stored on--but there's no requirement to have a distinct node for every key. We could have many many trip IDs that all map to the same node, and we can create as many or as few nodes as needed to meet the load. Thanks!