Designing MapReduce Algorithms. Ch. 3 Lin and Dyer’s text http://lintool.github.io/MapReduceAlgorithms/MapReduce-book-final.pdf Pages 43-73 (39-69). Improvements. Word count: Local aggregation as opposed to external combiner that is NOT guaranteed by the Hadoop framework
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Designing MapReduce Algorithms
Ch. 3 Lin and Dyer’s text
Pages 43-73 (39-69)
Word co-occurrence (matrix)
4 different reducers
<(var34, left), value>
<(var34, right), value>
<(var34, middle), value> all delivered to the same reducer.. What can you do with this?
Reducer can “middle(left’s value, right’s value) “ <var34, computedValue>
<KEY complex object, VALUE complex object>
You can do anything you want for function… “KEY.operation” on “VALUE.data”
Therein lies the power of MR.