{"id":1497,"date":"2016-08-30T13:57:34","date_gmt":"2016-08-30T13:57:34","guid":{"rendered":"http:\/\/codethataint.com\/blog\/?p=1497"},"modified":"2016-08-30T15:58:22","modified_gmt":"2016-08-30T15:58:22","slug":"custom-partitioner-mapreduce","status":"publish","type":"post","link":"https:\/\/codethataint.com\/blog\/custom-partitioner-mapreduce\/","title":{"rendered":"Custom Partitioner MapReduce"},"content":{"rendered":"<p>By using Partitioner we can group the output based on specific column.The Column based on which the output should be grouped is used for Partition.In below case I have used Second Value of TextPair key for grouping.<\/p>\n<p>The Output of reducer will be equal to Hash Modulo Denominator<\/p>\n<p>The Below Custom Partioner again makes use of HashCode and divides by the Total number of reducer. <\/p>\n<p>PartitionValue = (HashCode Value of String x Max Val of Integer)\/Total No of Reducers;<\/p>\n<pre class=\"brush: java; title: ; notranslate\" title=\"\">\r\npackage com.mugil.part;\r\n\r\nimport org.apache.hadoop.mapreduce.Partitioner;\r\n\r\nimport com.mugil.avg.LongPair;\r\nimport com.mugil.avg.TextPair;\r\n\r\npublic class FirstPartioner extends Partitioner&lt;TextPair, LongPair&gt;\r\n{\r\n\r\n   @Override\r\n   public int getPartition(TextPair arg0, LongPair arg1, int noOfReducers) \r\n   {\r\n\tint partitionValue = 0 ;\t\t\r\n\tpartitionValue = (arg0.getSecond().hashCode() &amp; Integer.MAX_VALUE)%noOfReducers;\t\t\r\n\treturn partitionValue;\r\n   } \r\n}\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>By using Partitioner we can group the output based on specific column.The Column based on which the output should be grouped is used for Partition.In below case I have used Second Value of TextPair key for grouping. The Output of reducer will be equal to Hash Modulo Denominator The Below Custom Partioner again makes use&hellip; <a href=\"https:\/\/codethataint.com\/blog\/custom-partitioner-mapreduce\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[150],"tags":[],"class_list":["post-1497","post","type-post","status-publish","format-standard","hentry","category-map-reduce"],"_links":{"self":[{"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/posts\/1497","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/comments?post=1497"}],"version-history":[{"count":3,"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/posts\/1497\/revisions"}],"predecessor-version":[{"id":1500,"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/posts\/1497\/revisions\/1500"}],"wp:attachment":[{"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/media?parent=1497"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/categories?post=1497"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/codethataint.com\/blog\/wp-json\/wp\/v2\/tags?post=1497"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}