MapReduce Training and Certification

Lesson 1: MapReduce Tutorial
Section 1.1: Purpose
This part of the lesson involves teaching beginners and professionals of the User Interactions of Hadoop MapReduce framework.

What are the Prerequisites to Learn MapReduce

Hadoop supported environment where it is duly installed and configured.

Section 1.2: Overview
Hadoop MapReduce

What is Hadoop MapReduce?

MapReduce is a framework. The purpose of MapReduce is to write applications easily. The application can process vast data. Hadoop framework is executed in Java. But it is not necessary that MapReduce be written in Java. Further knowledge about Hadoop Streaming and Hadoop Pipes are discussed.

Section 1.3: Inputs & Outputs
Input to the Job and Output to the job values are explained in this chapter.

Lesson 2: MapReduce Commands

Section 2.1:  Introduction
MapReduce commands are invoked by bin/mapred scripts.
Running mapred scripts, Option parsing framework of Hadoop and description regarding commands are explained in this lesson.

Section 2.2: User Commands
Various Commands involved in Hadoop Clusters and their functions are trained. Some of the Commands include-

Archive
Classpath
Distcp
Job
Pipes
Queue

Section 2.3: Administration Commands
Commands that are eminently used by Administrators and their functions are introduced in this session.
This session involves practical demonstration of commands and the working scenario.

Lesson 3: Compatibility between Hadoop 1.x and Hadoop 2.x

Section 3.1: Migrating from 1.x to 2.x
In this session migration techniques used for transformation from Hadoop 1.x to Hadoop 2.x is described.

Section 3.2: Binary Compatibility
The preliminary work is to ensure applications handling old mapred APIs are compatible. Applications built against MRv1 mapred APIs run straightaway by pointing them against Apache Hadoop 2.x Cluster.

Section 3.3: Source Compatibility
Complete Binary compatibility with the applications and APIs involved are explained in this session. Compiling applications with respect to map reduce APIs against MRv2 Jars is described in detail.   

Section 3.4: Tradeoffs between MRv1 users and Early MRv2 users
Managing compatibility between users of MRv1 and MRv2 is discussed in this session. One of the tricky aspects of the lesson as it forms involves discussion of breakdown of applications and the adaptability towards

Lesson 4: Hadoop Encrypted Shuffle

Section 4.1: Introduction
Introduction to Hadoop Encryption and MapReduce shuffle using HTTPS forms the training of this session.

Section 4.2: Configuration
To enable certain encrypted shuffle, it is essential to set the following properties in core-site.xml
The list of properties and the assigned default value combines the session.

Section 4.3: Keystore and Truststore Settings 
Keystore factory and implementation theory is explained in this session.

Section 4.4: Activating Encrypted Shuffle
After all changes have been made, Encrypted shuffle can be activated by restarting all Node Managers.
This session also involves learning
Client Certificates
Truststores
&
Debugging

Lesson 5: Pluggable Shuffle/Sort

Section 5.1: Introduction to Pluggable Shuffle
Introduction to Pluggable Shuffle and Pluggable Sort is learned in this session.
Experimental Pluggable Shuffle and Pluggable Sort capabilities are explained.

Section 5.2: Implementing Custom Shuffle and Custom Sort
Basic knowledge about Custom shuffle and Custom sort is explained. Understanding to Differentiate a Custom shuffle implementation and Custom sort implementation forms the base of this session.

Section 5.3: Configuration
Job Configuration and Node Manager Configuration properties are learned in this session.

Lesson 6: Hadoop Distributed Cache Deploy

Section 6.1: Introduction
This session involves Introduction to Distributed Cache Deploy and MapReduce framework via Distributed cache.

Section 6.2: Preconditions & Limitations
MapReduce limitations and conditions are discussed to enable students to understand complex situations better.

Section 6.3: Deploying a New MapReduce Version via Distributed Cache
Deployment of New MapReduce versions is discussed in this session. The steps involved in the deployment of New MapReduce versions involves 3 steps.

Section 6.4: MapReduce Archives and Classpath Configuration
Assigning a Classpath for MapReduce function and determination of configured path forms the core of the session.