1 / 26

MapReduce Example | MapReduce Programming | Hadoop MapReduce Tutorial | Edureka

This Hadoop tutorial on MapReduce Example ( Mapreduce Tutorial Blog Series: https://goo.gl/w0on2G ) will help you understand how to write a MapReduce program in Java. You will also get to see multiple mapreduce examples on Analytics and Testing. <br><br>Check our complete Hadoop playlist here: https://goo.gl/ExJdZs<br><br>Below are the topics covered in this tutorial:<br><br>1) MapReduce Way<br>2) Classes and Packages in MapReduce<br>3) Explanation of a Complete MapReduce Program<br>4) MapReduce Examples on Analytics<br>5) MapReduce Example on Testing - MRUnit

EdurekaIN
Download Presentation

MapReduce Example | MapReduce Programming | Hadoop MapReduce Tutorial | Edureka

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  2. Agenda for today’s Session  MapReduce Way  Classes and Packages in MapReduce  Explanation of a Complete MapReduce Program  MapReduce Examples on Analytics  MapReduce Example on Testing EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  3. MapReduce Example on Word Count Process EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  4. MapReduce Way EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  5. MapReduce Way – Word Count Process EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  6. Input/Output Classes in MapReduce EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  7. Input Format – Class Hierarchy EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  8. Output Format – Class Hierarchy EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  9. Packages and Classes in Word Count MapReduce Example EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  10. Packages to Import import java.io.IOException; import java.util.*; import org.apache.hadoop.fs.Path; import org.apache.hadoop.conf.*; import org.apache.hadoop.io.*; All these packages are present in hadoop-common.jar import org.apache.hadoop.mapreduce.*; import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; import org.apache.hadoop.mapreduce.lib.input.TextInputFormat; import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat; All these packages are present in hadoop-mapreduce- client-core.jar EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  11. Mapper Class Name of the Mapper Class which inherits Super Class Mapper public static class Map extends Mapper<LongWritable, Text, Text, IntWritable> { Mapper Class takes 4 Arguments i.e. Mapper<KEYIN, VALUEIN, KEYOUT, VALUEOUT> EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  12. Reducer Class Name of the Reducer Class which inherits Super Class Reducer public static class Reduce extends Reducer<Text, IntWritable, Text, IntWritable> { Reducer Class takes 4 Arguments i.e. Reducer <KEYIN, VALUEIN, KEYOUT, VALUEOUT> EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  13. Its Time to see some MapReduce Examples EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  14. MapReduce is useful in a wide range of applications in multiple domains. It is majorly used for 2 things:  Analytics: Process the data and give the desired results  Testing: Perform few test cases using MRUnit EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  15. Let us see few MapReduce Examples on Analytics EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  16. MapReduce Temperature Example EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  17. Temperature Example  Problem Statement: » Analysing weather data of Austin to determine Hot and Cold Days. We have weather data set of Austin by NCIE. NOAA's National Centres for Environmental Information (NCEI) (previously NCDC) is responsible for preserving, monitoring, assessing, and providing public access to the Nation's treasure of climate and historical weather data and information. Weather Forecasting Refer -> ftp://ftp.ncdc.noaa.gov/pub/data/uscrn/products/daily01 EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  18. Temperature Example - Weather Dataset 6thColumn Max Temp 6thColumn Min Temp EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  19. MapReduce Example EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  20. Last.fm Example is an online music website where users listen to various tracks, the data gets collected like shown below. Write a map reduce program to get the Number of unique listeners. The data is coming in log files and looks like as shown below: UserId TrackId Shared Radio Skip 100001 150 1 1 0 100005 103 0 0 1 100142 78 1 0 0 110005 289 1 0 1 EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  21. Let us see a MapReduce Example on Testing EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  22. MRUnit Testing Framework  Provides 4 drivers for separately testing MapReduce code » MapDriver » ReduceDriver » MapReduceDriver » PipelineMapReduceDriver *JUnit is a simple framework to write repeatable tests.  Helps in filling the gap between MapReduce programs and JUnit*  Better control on log messages with JUnit Integration EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  23. MapReduce MRUnit Example EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  24. Learning Resources  Hadoop Tutorial: www.edureka.co/blog/hadoop-tutorial  MapReduce Tutorial: www.edureka.co/blog/mapreduce-tutorial  MapReduce Interview Questions: www.edureka.co/blog/interview-questions/hadoop-interview-questions-mapreduce EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

  25. Thank You … Questions/Queries/Feedback EDUREKA HADOOP CERTIFICATION TRAINING www.edureka.co/big-data-and-hadoop

More Related