CSE138 (Distributed Systems) L20: online systems vs. offline systems, raw data vs. derived data; ...

UC Santa Cruz CSE138 (Distributed Systems) Lecture 20: online systems vs. offline systems, raw data vs. derived data; intro to MapReduce; MapReduce example: forward index to inverted index
Recorded May 18, 2020
Professor Lindsey Kuper (users.soe.ucsc...)
Course website: decomposition.a...
Schedule of topics: decomposition.a...
Playlist of all lectures: • CSE138 (Distributed Sy...

[@fatihsenturk649]

Hi Lindsey,
one question would be, what is happening when a new reduce worker added to the system? how map-workers decide which piece of computational result will sent to new reduce-worker machine? they will communicate with each other to decide? and what would be communication pattern for that?

[@lindseykuperwithasharpie]

Well, let me turn the question around: do you think it makes sense to increase the number of reduce workers in the middle of a job? Why or why not?