Big Data System Lab Note

Mon, 09 Oct 2023 00:00:00 +0000

Set up Hadoop environment

Apache Hadoop 3.3.6 – Hadoop: Setting up a Single Node Cluster.

set up java environment

java version requirements:

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Java+Versions

% curl -sL "https://builds.openlogic.com/downloadJDK/openlogic-openjdk/8u382-b05/openlogic-openjdk-8u382-b05-linux-x64.tar.gz" -o ./openjdk-8u382.tar.gz
% tar -xvf ./openjdk-8u382.tar.gz
% mv ./openlogic-openjdk-8u382-b05-linux-x64 ./usr/bin/openjdk-8u832
% sudo vim ~/.zshrc

add java to path:

# ~/.zshrc

#...
export JAVA_PATH="/usr/bin/openjdk-8u832/bin"
export PATH=$PATH:$JAVA_PATH

test path:

% source ~/.zshrc
% java -version 1 ↵
openjdk version "1.8.0_382-382"
OpenJDK Runtime Environment (build 1.8.0_382-382-b05)
OpenJDK 64-Bit Server VM (build 25.382-b05, mixed mode)

requisite software

ssh required, pdsh for better performance.

Java on CuSO4_Deposit

Big Data System Lab Note

Set up Hadoop environment

set up java environment

requisite software