{"id":1456,"date":"2016-08-22T12:10:21","date_gmt":"2016-08-22T12:10:21","guid":{"rendered":"http:\/\/codethataint.com\/blog\/?p=1456"},"modified":"2016-08-22T12:15:42","modified_gmt":"2016-08-22T12:15:42","slug":"how-mapreduce-works","status":"publish","type":"post","link":"https:\/\/codethataint.com\/blog\/how-mapreduce-works\/","title":{"rendered":"How MapReduce Works"},"content":{"rendered":"

Mapper<\/strong>
\nThe map function in Mapper reads row by row of Input File <\/p>\n

Combiner<\/strong>
\nThe Combiner wont be called when the call to the Reducer class is not there in Driver class. <\/p>\n

Reducer<\/strong>
\nThe Reducer and Combiner need not do the same thing as in case of average of 0 to 100 Numbers.<\/p>\n

Input to Mapper<\/strong><\/p>\n

\r\n1\/1\/09 1:26,Product2,1200,Nikki,United States\r\n1\/1\/09 1:51,Product2,1200,Si,Denmark\r\n1\/1\/09 10:06,Product2,3600,Irene,Germany\r\n1\/1\/09 11:05,Product2,1200,Janis,Ireland\r\n1\/1\/09 12:19,Product2,1200,Marlene,United States\r\n1\/1\/09 12:20,Product2,3600,seemab,Malta\r\n1\/1\/09 12:25,Product2,3600,Anne-line,Switzerland\r\n1\/1\/09 12:42,Product1,1200,ashton,United Kingdom\r\n1\/1\/09 14:19,Product2,1200,Gabriel,Canada\r\n1\/1\/09 14:22,Product1,1200,Liban,Norway\r\n1\/1\/09 16:00,Product2,1200,Toni,United Kingdom\r\n1\/1\/09 16:44,Product2,1200,Julie,United States\r\n1\/1\/09 18:32,Product1,1200,Andrea,United States\r\n<\/pre>\n
Output of Mapper<\/strong><\/p>\n
\r\nKey = Product Date {2009-01} Product Name {Product2}\r\nValue = Product Price {1200} \t  Product No {1}\r\n----------------------------\r\nKey = Product Date {2009-01} Product Name {Product2}\r\nValue = Product Price {1200} \t  Product No {1}\r\n----------------------------\r\nKey = Product Date {2009-01} Product Name {Product2}\r\nValue = Product Price {3600} \t  Product No {1}\r\n----------------------------\r\nKey = Product Date {2009-01} Product Name {Product2}\r\nValue = Product Price {1200} \t  Product No {1}\r\n----------------------------\r\nKey = Product Date {2009-01} Product Name {Product2}\r\nValue = Product Price {1200} \t  Product No {1}\r\n----------------------------\r\nKey = Product Date {2009-01} Product Name {Product2}\r\nValue = Product Price {3600} \t  Product No {1}\r\n----------------------------\r\nKey = Product Date {2009-01} Product Name {Product2}\r\nValue = Product Price {3600} \t  Product No {1}\r\n----------------------------\r\nKey = Product Date {2009-01} Product Name {Product1}\r\nValue = Product Price {1200} \t  Product No {1}\r\n----------------------------\r\nKey = Product Date {2009-01} Product Name {Product2}\r\nValue = Product Price {1200} \t  Product No {1}\r\n----------------------------\r\nKey = Product Date {2009-01} Product Name {Product1}\r\nValue = Product Price {1200} \t  Product No {1}\r\n----------------------------\r\nKey = Product Date {2009-01} Product Name {Product2}\r\nValue = Product Price {1200} \t  Product No {1}\r\n----------------------------\r\nKey = Product Date {2009-01} Product Name {Product2}\r\nValue = Product Price {1200} \t  Product No {1}\r\n----------------------------\r\nKey = Product Date {2009-01} Product Name {Product1}\r\nValue = Product Price {1200} \t  Product No {1}\r\n----------------------------\r\n<\/pre>\n
Magic of Framework happens Here<\/strong>
\nInput of Combiner<\/strong><\/p>\n
\r\n----------------------------\r\nKey = Product Name {2009-01} \t Product No {Product1}\r\nValues\r\nProduct Price 1200\t\r\nProduct No 1\r\nProduct Price 1200\t\r\nProduct No 1\r\nProduct Price 1200\t\r\nProduct No 1\r\n----------------------------\r\nKey = Product Name {2009-01} \t Product No {Product2}\r\nValues\r\nProduct Price 1200\t\r\nProduct No 1\r\nProduct Price 1200\t\r\nProduct No 1\r\nProduct Price 3600\t\r\nProduct No 1\r\nProduct Price 1200\t\r\nProduct No 1\r\nProduct Price 1200\t\r\nProduct No 1\r\nProduct Price 3600\t\r\nProduct No 1\r\nProduct Price 1200\t\r\nProduct No 1\r\nProduct Price 1200\t\r\nProduct No 1\r\nProduct Price 1200\t\r\nProduct No 1\r\nProduct Price 3600\t\r\nProduct No 1\r\n----------------------------\r\n<\/pre>\n
Values added together in Combiner based on Key<\/strong><\/p>\n
\r\nkey 2009-01\tProduct1\r\nproductPrice 1200\r\nproductNos 1\r\n----------------------------\r\nproductPrice 2400\r\nproductNos 2\r\n----------------------------\r\nproductPrice 3600\r\nproductNos 3\r\n----------------------------\r\nkey 2009-01\tProduct2\r\nproductPrice 1200\r\nproductNos 1\r\n----------------------------\r\nproductPrice 2400\r\nproductNos 2\r\n----------------------------\r\nproductPrice 6000\r\nproductNos 3\r\n----------------------------\r\nproductPrice 7200\r\nproductNos 4\r\n----------------------------\r\nproductPrice 8400\r\nproductNos 5\r\n----------------------------\r\nproductPrice 12000\r\nproductNos 6\r\n----------------------------\r\nproductPrice 13200\r\nproductNos 7\r\n----------------------------\r\nproductPrice 14400\r\nproductNos 8\r\n----------------------------\r\nproductPrice 15600\r\nproductNos 9\r\n----------------------------\r\nproductPrice 19200\r\nproductNos 10\r\n----------------------------\r\n<\/pre>\n
Output of Combiner and Input to reducer<\/strong><\/p>\n
\r\nKey = Product Name {2009-01} \t Product No {Product1}\r\nvalue = Product Price {3600}\tProduct Nos {3}\r\n----------------------------\r\nkey = Key = Product Name {2009-01} \t Product No {Product2}\r\nValue = Product Price {19200} Product Nos {10}\r\n<\/pre>\n
 Output of Reducer<\/strong><\/p>\n
\r\nKey = Product Name {2009-01} \t Product No {Product1}\r\nValue = AvgVolume {1200}\tNoOfRecords {3}\r\n----------------------------\r\nKey = Product Name {2009-01} \t Product No {Product2}\r\nValue = AvgVolume {1920}\tNoOfRecords {10}\r\n----------------------------\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"
Mapper The map function in Mapper reads row by row of Input File Combiner The Combiner wont be called when the call to the Reducer class is not there in Driver class. Reducer The Reducer and Combiner need not do the same thing as in case of average of 0 to 100 Numbers. Input to… Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[182,150],"tags":[],"class_list":["post-1456","post","type-post","status-publish","format-standard","hentry","category-interview-questions","category-map-reduce"],"_links":{"self":[{"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/posts\/1456","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/comments?post=1456"}],"version-history":[{"count":3,"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/posts\/1456\/revisions"}],"predecessor-version":[{"id":1459,"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/posts\/1456\/revisions\/1459"}],"wp:attachment":[{"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/media?parent=1456"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/categories?post=1456"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/tags?post=1456"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}