Google File System - Paper that inspired Hadoop
Vložit
- čas přidán 26. 07. 2024
- Distributed File Storage made by Google around 2003
Channel
----------------------------------
Complex concepts explained in short & simple manner. Topics include Java Concurrency, Spring Boot, Microservices, Distributed Systems etc. Feel free to ask any doubts in the comments. Also happy to take requests for new videos.
Subscribe or explore the channel - / defogtech
New video added every weekend.
Popular Videos
----------------------------------
What is an API Gateway - • What is an API Gateway?
Executor Service - • Java ExecutorService -...
Introduction to CompletableFuture - • Introduction to Comple...
Java Memory Model in 10 minutes - • Java Memory Model in 1...
Volatile vs Atomic - • Using volatile vs Atom...
What is Spring Webflux - • What is Spring Webflux...
Java Concurrency Interview question - • Java Concurrency Inter...
You have managed to take the written article published by google and explains Google File System and MapReduce in a very good, clear way so anyone can consume it. good job!
This is how one should teach. Loved it!
Best explanation of GFS on CZcams. This is how one should teach. Wish the channel had posted more such videos.
Thank you for getting into distributed systems realm. Would love to see some videos about the workings of cassandra.
This explanation and the voice clarity is too good, thanks a lot for the video
amazing video. the finest tutorial of GFS on youtube.
Excellent video clearing the basics of GFS.
Your videos are crisp and very clear. Thanks. Keep up the good work😊👍
Very well systematically explanation.! Great Work! Thank you. God bless you.
Now I realsed that my teacher must have watched this video before lecture .
Perfect explanation .Thank you for this.
Thanks for the great content. But I haven't seen any recent videos from you, it'd be great if you start making these videos again, they're just wonderful.
high quality video. your explanations are clear as crystal
Very clear explanation. Thank you!
Very nicely explained.Thanks a lot :)
Your explanation is GREAT.
Thank you! a very clear explanation.
Great explanation! Thanks.
Very well explained and articulated.
Very well explained. Thanks for the content and effort
Excellent Explanation sir. Loved it. Wanted to learn more on it. I am now big fan of yours. Watched most of your videos and trying to read as much possible from you sir. Please consider to share more videos on this topic.
Thank you so much for your kind words! Planning to do more videos soon
So a simple explanation. Amazing.
Amazing video, super clear.
Very nicely explained. Thank you.
Amazing video. Thank you so much
Nice! Very easy to follow
Well explained ,thank you.
Very well explained , thanks
FWIW the google paper left out some of the details you needed for full resilience. We had to learn those through things going wrong -e.g the infamous facebook cascade failure where, once a critical mass of servers went offline, the recovery work took the rest down
Great video. Thanks for making. There is a typo in the last slide about chunk size being 64kb instead of 64mb :)
Can't wait for the BigTable and DynamoDB episodes!! Also can you do one on pagerank?
Great job!
Such a good explanation.
Very good explanation!
Exceptional sir.
Great explanation.
Awesome explanation
great content , thank you
Great explanation
Well done,thank you
Awesome 👍
Really good video.
Very helpful.
Thank you for your video
Epic!
Good work
Thank you so much..
Awesome
Perfect.
Gold !
I really like your videos and regular follower of your channel. Please publish a video on Cassandra architecture. If possible.
Yes sir! Cassandra is similar to Google Bigtable. Will cover it soon.
🙏 You are the Best Teacher that I have ever seen.🥇, A classical method of approach = Superb indeed.👌.🎯Please Could you kindly teach us Flutter & Dart & Go lang & Scala too . Pleeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeese Sir..... 🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏
Thank you so much for the kind words. Definitely plan to continue creating videos, and also covering more topics like Dart and Go
Very nice explanation. Thank you so much for the same. I had a few questions - 1. Which type(s) pf write operation(s) can a client application perform on the files. In other words, what does GFS supports in terms of writing into files? 2. Where in the file the writing can be done? 3. How much data can be written there and by how many applications at the same time and at the same location? 4. In what type of memory is the metadata stored on the master
Bit late but hopefully others find it useful:
Regarding (1) & (2), GFS was primarily aimed at applications which involved appending data to a long-running file, with random location writes happening rarely (if at all).
For (3), the limits would depend on the actual implementation - this was just a theoretical paper on GFS. But yes, it is meant to support concurrent writes.
And regarding (4), the Master keeps a track of all the files on the GFS, a list of chunk handlers that make up each file, and the list of chunk servers that hold a particular chunk (there are multiple servers for redundancy)
Additionally the master keeps a log of all the changes that were made (along with checkpoints), so that it can recover from crashes
waw u explained so smilply
its architecture somewhat similar to napster ( I think napster was ahead of its time)
Hi , Can you make video related to transaction management ,propagations ,isolations,2-phase commit, deadlocks in transactions..
Gr8
💙
Good one buddy👏
Thank you buddy! Btw, Congrats on 40K, that was fast!
Defog Tech thanks dude🙏🏻
I m happy to see 2 good Java content creators at one place. Continue the legacy. :)
Sanjay Kantheti 🤘
Thanks, can you also explain MapReduce please?
Absolutely. That video is coming tomorrow!
👍🏿👍🏿👍🏿
Hey , request you to please make new video.
Eagerly waiting for your new videos.
If each chunk has 3 replicas, then will there be a total of 4 such chunks ?
Can you please do Dynamo paper
Hey , why have you stopped uploading videos....let me know if you need any funding
hey, thanks so much for reaching out.. just been busy with new job, and added laziness :(
I am researching for NoSQL series, but will take time.
Regarding funding, absolutely.. I love teaching and hope to someday do this fulltime. Might start CZcams subscriptions for earning. Let me know your thoughts, is ~130 INR a month something everyone would go for, or should it be course based fees on a separate website?
@@DefogTech , this channel is my first point of reference for any cs related topic , things that make your videos unique are the awesome slides , quality content & very crisp explanation. I myself being a software professional at Adobe frequently watch your videos, so yes youtube subscription seem to be a viable option. Hoping you to resume soon so that we can make best amidst the lockdown. :)
is the master something like a zookeeper?
No, it's a single instance backed by a secondary one unlike concensus based one like zookeeper
The chunk is **64KB** NOT MB
It is 64MB. GFS is used mainly for large files with append-only structure. In the paper there is good trade-off of why they chose 64MB.
MB
Explained clearly. Thanks.