Download
1 / 8
80 likes | 90 Views
Learn how to develop a MapReduce application using HDFS for packet dissection. Explore the steps, central phases, and tools involved in the process for efficient cloud computing power.
E N D
How MapReduce Works? • MapReduce application is based on Hadoop Distributed FileSystem, HDFS. • Steps of the MapReduce process: • The client submits the job. • JobTracker coordinates the job and splits into tasks. • TaskTrackers run the tasks, here is the main map and reduce phases.
Shuffle and Sort • These two facilities are the heart of MapReduce which make Cloud Computing powerful. • Sort phase: guarantees the input to every reduce is sorted by key. • Shuffle phase: transfers the map output to the reducers as input.
A MapReduce Application – packet dissection • With Jpcap library, captures packets and writes to HDFS directly.
A MapReduce Application – packet dissection • Setup a job configuration and submit the job.
A MapReduce Application – packet dissection • The mapper filters packets with the port 1863, which is the MSN protocol.
A MapReduce Application – packet dissection • The reducer dissect the packet, and write message to output collector.
A MapReduce Application – packet dissection • See the result:
