Awesome video! I have a quick question on the replication pipeline though. You mentioned that if write from A to B is failed, client could choose to ignore it and let the Name Node to do the replication. What if write to A is failed? For client he didn't get an ack neither way, but the second one definitely can't be ignored since Name node won't know how to do a replica without the original data. How the client knows the difference?
If you know the client is closest to a certain data center, can you manually or automatically have your two replicas assigned to that data center? In the case where you have multiple clients in different zones, would writes be slower on average because you can’t guarantee the secondary replica is in the same zone as the client +primary?
It's less so that they're in different zones, ideally you have two data centers on the same rack and a third on a different rack same datacebter or something like that
What is the difference between the replication pipeline in hadoop and the chain replication? Both look very similar. Chain replication guarantees strong consistency. So this must also offer strong consistency, right? Is the difference in the way the reads happen in both the cases? Like reads can happen from any datanode in case of hadoop but in case of chain replication it happens only from the tail node.
Great video!
There's so much to learn from your videos, thanks for all the hard work :)
Thanks Arjit!!
Finally a YT channel that goes in depth about system design topics.
Yooo these videos are goated
Ur goated
Awesome video! I have a quick question on the replication pipeline though. You mentioned that if write from A to B is failed, client could choose to ignore it and let the Name Node to do the replication. What if write to A is failed? For client he didn't get an ack neither way, but the second one definitely can't be ignored since Name node won't know how to do a replica without the original data. How the client knows the difference?
I guess generally as a client we should likely assume that no ack = no write and try to rewrite until we get one
If you know the client is closest to a certain data center, can you manually or automatically have your two replicas assigned to that data center? In the case where you have multiple clients in different zones, would writes be slower on average because you can’t guarantee the secondary replica is in the same zone as the client +primary?
It's less so that they're in different zones, ideally you have two data centers on the same rack and a third on a different rack same datacebter or something like that
What is the difference between the replication pipeline in hadoop and the chain replication? Both look very similar. Chain replication guarantees strong consistency. So this must also offer strong consistency, right? Is the difference in the way the reads happen in both the cases? Like reads can happen from any datanode in case of hadoop but in case of chain replication it happens only from the tail node.
I think you answered your own question ;)
I subscribed :)
Alright in exchange I'll offer you foot pics ♥️
nah u not ugly bro. U look like Miles Teller
That's a new one, ur too kind
A small suggestion, please avoid speaking when panning around or zooming in / out on iPad. its distracting