Step-by-Step Guide to Install and Configure a Hadoop 2.8.2 Cluster

This guide shows how to download the Hadoop 2.8.2 source package from http://mirror.bit.edu.cn/apache/hadoop/common , extract it, and prepare a three‑node cluster consisting of a master (172.16.11.97) and two slaves (172.16.11.98, 172.16.11.99).

On the master node, the package is obtained with:

wget http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.8.2/hadoop-2.8.2.tar.gz
tar zxvf hadoop-2.8.2.tar.gz

Configuration files under hadoop-2.8.2/etc/hadoop are edited:

cd hadoop-2.8.2/etc/hadoop
vim hadoop-env.sh   # set JAVA_HOME
export JAVA_HOME=/usr/local/src/jdk1.8.0_152

vim yarn-env.sh   # set JAVA_HOME similarly

vim slaves
slave1
slave2

Key XML files are modified (shown here as code snippets):

<configuration>
  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://172.16.11.97:9000</value>
  </property>
  <property>
    <name>hadoop.tmp.dir</name>
    <value>file:/usr/local/src/hadoop-2.8.2/tmp</value>
  </property>
</configuration>

<configuration>
  <property>
    <name>dfs.namenode.name.dir</name>
    <value>file:/usr/local/src/hadoop-2.8.2/dfs/name</value>
  </property>
  ...
</configuration>

<configuration>
  <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>
</configuration>

<configuration>
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  ...
</configuration>

Required directories are created:

mkdir /usr/local/src/hadoop-2.8.2/tmp
mkdir -p /usr/local/src/hadoop-2.8.2/dfs/name
mkdir -p /usr/local/src/hadoop-2.8.2/dfs/data

Environment variables are added to ~/.bashrc on all nodes:

HADOOP_HOME=/usr/local/src/hadoop-2.8.2
export PATH=$PATH:$HADOOP_HOME/bin

The Hadoop directory is copied to the slave machines:

scp -r /usr/local/src/hadoop-2.8.2 root@slave1:/usr/local/src/hadoop-2.8.2
scp -r /usr/local/src/hadoop-2.8.2 root@slave2:/usr/local/src/hadoop-2.8.2

The namenode is formatted and the whole cluster started:

hadoop namenode -format
./sbin/start-all.sh

Cluster status can be checked with jps on each node and the YARN web UI is reachable at http://master:8088 . Common Hadoop commands are the same as in version 1.0, and the cluster can be stopped with:

./sbin/hadoop stop-all.sh