Apache Hadoop 3.1.2, the brand new software to help

The recent update of Apache Hadoop 3.1.2 had the changes software engineers always intended in the Apache Hadoop- 2. Version. This version includes improvements and additional features from the previous Apache Hadoop, This version is available (GA) and let’s see the major changes below:

Overview

1. Yarn Service framework

Which supplies first class support and APIs to host long-running services inside the YARN. It acts as a container orchestration platform for managing containerized services on YARN. It supports both docker container and traditional process based on the containers in YARN.

First-class GPU scheduling and isolation are assigned for both docker/non-docker containers on YARN.

First-class FPGA scheduling and isolation are assigned for both docker/non-docker containers on YARN.

This version is built in such a way that it supports more expressive placement constraints in YARN, thereby playing an important role in the performance and flexibility of applications,

Applications which include long-running containers, such as services, machine-learning, and streaming workloads are benefitted by this version.

For example, it may be beneficial to co-locate the allocations of a job on the same rack (affinity constraints) to reduce network costs, spread allocations across machines (anti-affinity constraints) to minimize resource interference, or allow up to a specific number of allocations in a node group (cardinality constraints) to strike a balance between the two. Placement decisions also affect resilience. For example, allocations placed within the same cluster upgrade domain would go offline simultaneously.

2. In a queue instead of providing percentage based values, the mechanism helps to support administrators to identify and mention the absolute resources (X Memory, Y V Cores, Z GPUs, etc.). Thereby helping admins to take control and to configure the desired amount of resources for the given queue.

3. Heterogeneous storage:

 By building with heterogeneous storage, it brings in new storage type, PROVIDED, to the set of media in a Data Node.

Source: https://hadoop.apache.org/docs/r3.1.2/index.html

For more details refer to the above link.