Scaling Distributed Systems - Software Architecture Introduction (part 2)
Vložit
- čas přidán 11. 10. 2020
- Software Architecture Introduction Course covering scalability basics like horizontal scaling vs vertical scaling, CAP theorem and more.
In this second video of the Software Architecture Introduction series, I cover scalability basics and important considerations you need to take into account when scaling distributed systems, like Horizontal Scaling, Vertical Scaling and the CAP theorem. You will learn how to split the load in stateless and stateful services, load balancing, sharding strategies and replication. I also introduce some topics like consistent hashing and other relevant topics. For more topics regarding Software Architecture and Design checkout the playlist: • Software Architecture ...
----------------------
🔖 Some resources used as reference for this and the other videos in the series:
Incentivized:
- Building Evolutionary Architectures: amzn.to/337bPwj
- Cracking the Coding Interview: amzn.to/3cEvabk
- Building Microservices: Designing Fine-Grained Systems: amzn.to/3ibsQtg
- Risk Driven Architecture: amzn.to/3a8wdQC
- Microservices Patterns: amzn.to/3e5C7Ds
Others:
- Software Architecture Patterns: www.oreilly.com/programming/f...
- Martin Fowler's Architecture Post: martinfowler.com/architecture/
- 10 Common Software Architectural Patterns in a Nutshell: towardsdatascience.com/10-com...
- CAP Twelve Years Later How the Rules Have Changed: www.infoq.com/articles/cap-tw...
- Scalable System Design Patterns: horicky.blogspot.com/2010/10/s... Design Scalable Systems: hexadix.com/design-scalable-s...
- CAP Theorem (IBM article): www.ibm.com/cloud/learn/cap-t...
-----------------------
🤓 If you are preparing for a technical interview, don't miss my playlist: devstory.page.link/technical-...
📚 I have also prepared FREE study guides available here (no subscription needed): devstory.page.link/study-guides
-----------------------
Remember you can reach me here:
- #️⃣ Twitter: / cv13lm4
-⌨️ Github: github.com/cvielma
-----------------------
In A Dev' Story I explain Software Development related topics from my point of view. I'll be explaining different topics in "Chapters" according to the stage of the story I think they belong. Hope is as entertaining for you as it's fun for me to create these videos
Thanks again for watching! Feel free to like, comment, share and subscribe, it means a lot to me and helps the channel grow.
#SoftwareArchitecture #SoftwareEngineering #Programming - Věda a technologie
Hey folks!👋 Want to have hands-on practice on these concepts? 🤓
Try out CodeCrafters.io we have partnered and you can get 40% off 💸with this link: app.codecrafters.io/join?via=DevStory
They have great hands-on tutorials and easy to get up to speed with complex systems. Check them out!
Your handwriting is really impeccable.
Thank you very much. Not my proudest skill xD
Very underrated channel!
Thank you! I guess it's mostly that's newer. I need to get the word spread xD
Thanks for the series, love the explanations followed by practical examples!
Thank you !
I knew these concepts, but putting them together like that was great! Thanks!
Glad it was useful!
Thank you; incredibly helpful
Thank you for the rarely discussed topics about at the essence of software engineering on CZcams, these information can be found in books (expensive ones), and universities in theory only, but you put such a great effort to explain in simple basic way practical use-case scenario of such concepts, great work keep it going.
Oh really nice comment! Thank you very much and I'm glad you enjoyed it!
wow, your videos are technically accurate, and to the point! just subscribed thank you!
Thank you! Hopefully more content coming soon!
Every concept is concise, clear and gradual. Best video I have watched so far. Thanks a lot 😊
You are welcome! Glad you enjoyed the video :)
Thank you so much for these videos!! So helpful and so well-explained
Glad it was useful to you! If you would like me to explain another topic feel free to suggest 😁
This kind of video should have more likes.
Thanks! :)
very helpful thanks
Thanks for sharing, great video!
Thank you!
Great video
thanks for sharing! very helpful!
Glad it helped!
Very cool series, keep going man!
Thanks! Glad you liked it!
Thanks!
thank you
Muy buen video. Saluos desde México
¡Gracias!
great series!
Thanks! Glad you liked it!
Game changer! 👍
Thanks!
Thank you :)
You're welcome!
Great content hope you will add more videos to this software architecture series... and if you dont mind can you also make a video on what books, courses, etc you used to learn about software architecture?
Oh yeah. A good idea! Just in case, I've put some recommended books/links in the description, but definitely a video would be cool too.
Thank you!
You're welcome!
ty
perfect video. keep them coming...
Thank you! Any specific topic you would like to see?
@@ADevStory resiliency, performance, database types (cassandra, HDFS, ...), datalake vs datawarehouse, microservices, principles of Kubernetes & Docker
Very nice 👍 thanks
Thank you!
great channe!
Thank you! Glad you liked it!
Good learning
Glad you think so!
Amazing video brother!!
could you plz make more videos like that, mm, for ex How to make X or Y app, using Cloud \ DevSecops services and all tools needed.
Many Thanks in advance!
Glad you liked it! Will try to make more videos, but trying to focus on general concepts more than specific technology. For sure can try to prepare something with cloud. Thanks for your feedback!
❤
❤
❤
Hi, your channel is very good! Please make more videos on all aspects of system design and architecture.
You mentioned horizontal scaling is not cheap to do? Isn't the reverse true, as we can get a lot of commodity hardware when we want, unlike in case of vertical scaling where we need a expensive machine.
Hello! thanks! I'm glad you are enjoying the content!
So in terms of hardware, commodity hardware is cheaper but you need many more than just one server. But I was referring mostly to the architecture complexity of building software that scales horizontally vs vertically. You need to take way more considerations like load balancing, network, consistency level, transactions, etc
Hope is clear now :)
Hermano! Buen contenido. Gracias.
Sería épico si agregas una card al final del vídeo que te lleve al siguiente :)
¡Pensé que estaba! Vale lo reviso. ¡Muchas gracias!
Hi Chiristian, Great Content, Thanks for this wonderful video. I have one question, what is that BE represents in your layered architecture diagram?
Thank you! Glad you liked it.
BE represents an instance of the Back-End service
your drawings really help
Thanks!
Noice'n'eezy!
Dear Friends,
When I design a software system, I divide it into 3 categories:
1/ Conceptual: I will state the names of the software that will respond to the requirements of the business process from users, describe the names of the software to be processed and the behaviors.
2/ Logical: I can define software architecture by applying architectural pattern in this step and I will include constraints on availability, scalability, maintainability. I also describe in detail the function of each component. This step
3/ Physical: Define programming language, platform to realize the description in Logical step.
Could you recommend?. Thank you.
Great information!! what do you mean with "state" for the CAP Theorem?? :)
Glad you liked it!
What I mean by state is basically data that resides on a specific server . If all your servers were equal there's no problem if the network partitions, but that's only possible if they are _stateless_.
For example: if the servers just do sum of 2 number that you send on your request, in case of failure of some servers there's no problem regarding CAP.
On the other hand if you are handling data, say you register new products in a catalog in the servers, if they all save data to a common database, the risk is in the database going down. If you have multiple instances of the database where you write to, they have to sync between them. And that's when the CAP theorem applies.
Since you are partitioning DB in to multiple DB's , BE1,BE2,BE3 are not connected to DB1,DB2,DB3 right? any BE server can send requests to any DB server based on hashvalue ? got bit confused by diagram
Great question! It's actually left intentionally vague. It can be either way depending if you are going for replication or sharing. In sharding you could make a backend talk to different DBs based on a hash value for example. But in replication you could make it only read from a specific instance. If you need write, and want to scale the data layer (DB) then the backend s should either be able to pick a db to write to or there would another piece in the middle redirecting the traffic to the specific db instance
Hope is clearer!
Thank you for the video. One question: if the databases are separate then how will they be synched ? and what did you actually mean by "keeping state" ?
So if you want to have the database separated there are different ways. One of the most common and easiest ones is by doing sharding (explained in the video), which means dividing the data in different sets and storing them separately. If you want to keep replicas of the same information, other complexities arise, like certain coordination (aka consensus) between the different database instances.
Keeping state means any type of data that you want to store or modify. From a user session to any specific data related to your business.
Hope is clearer :)
@@ADevStory got it now, thanks for clear explanation ❤️
Anytime!
1:15 not what I expect :)
Can you please help me what "BE1" and "BE2" mean around 5:23? Does BE mean balance equalizer? Please help. Thanks.
BE means Back-end. Just a representation of a service
Thank you so much for your quick response! Very helpful!
Happy to help!
Hi Dev, i have one silly question....what is actually state and stateless means here?
Hi!
So basically state means if you store any type of data.
For example: if you have a service that when it receives 2 numbers it returns the sum of them it would be stateless. If you have a service that serves as a contact list and whenever you ask for a user it returns the phone number if it was registered, the phone number would be the state.
Hope it's clearer
@@ADevStory Thanks for your prompt respond Dev. It's really helps.
Thanks again for the explanation.
Bro what do you mean by BE1, BE2, BE3 on the figure? 5:23
I just meant different backend instances (BackEnd instance 1, BackEnd instance 2 and BackEnd instance 3)
What's the meaning of the term "state"?
State means data. It's the information you operate on and/or want to store. Is it clearer ?
What does "BE" stand for in the diagram?
It stands for Back End
@@ADevStory thank you!
@@ADevStory so you’re saying the nodes are back end components that get distributed to the database, right?
Kind of. I mean that you can distribute the processing and the data in different nodes.
For example: if you need to do more processing, you can have multiple backend nodes to do the processing. And for the database you can do the same splitting the load between different databases.
Not sure if it's clearer 😅
Anytime!
A un español hablando inglés le entiendo mejor que a un inglés hablando inglés, yea!
Jajaja qué bueno :)
You can use doodly for making hand writing animation video
Yeah, but actually prefer pen and paper 🙂
What is the definition of HASH?
Will link to Wikipedia: en.m.wikipedia.org/wiki/Hash_function
But paraphrasing, basically a hash is a value that represents another value uniquely in a hash function. It could be anything but it has to be unique for the same input.
For example, you can have a "simple hash function" that for a given string returns the number of letters in it (the hash). In this case: the hash of "house" would be 5, and the hash of "cat" would be 3.
Hope it's clearer!
Is it possible to add english subtitles?
It already has right? If you press on the CC icon in CZcams.
@@ADevStorysorry! Thanks! Somehow I maybe missed it
No problem!
Bad quality. All parts need to be in one video. Made into un necessary partitions for monetization.
Hey man sad to hear you didn't enjoyed it. On each one I try to improve quality but it has been a learning curve for me. I appreciate any feedback on things that can be improved .
Besides of that just wanted to correct you on the monetization part since that's not the reason. My goal was to make them short and more digestable.
Videos over 8 minutes can actually include more ads inside so not a big difference monetization wise vs splitting. I also try to make them shorter than that and in the new ones I'm also removing the longer intro/closure to reduce the unnecessary filling.
Not saying I made them without interest in monetizing but not the reason for the split :)