Hi friends,
I have started learning Apache Spark and it is time now to share few things with you.
In this series related to Spark and Scala, we are going to see essential things that we should know regarding both Spark and Scala.
So as I always begin, we are going to start with Spark Installation.
For this series, we are going to use Spark 1.6.1 pre-bundled with Hadoop 2.6.0 and compatible with Scala 2.11.7.
Please use below steps in order to install Spark 1.6.1 on Ubuntu 14.04
| Following are the steps which you need to follow in order to install Spark 1.6.1 on Ubuntu 14.04 | |
| Assumptions : username is hduser | |
| Step 1 : JAVA Installation | |
| If you have already installed JAVA, then you can skip this step. | |
| If JAVA is not installed, then please follow below steps in order to install JAVA 1.7 | |
| command 1 : sudo apt-get update | |
| command 2 : sudo apt-get install openjdk | |
| Once above commands are executed successfully, then you can run following command for verification. | |
| $java -version | |
| Above command should give you output which might look something like this. | |
| java version "1.7.0_71" | |
| Java(TM) SE Runtime Environment (build 1.7.0_71-b13) | |
| Java HotSpot(TM) Client VM (build 25.0-b02, mixed mode) | |
| JAVA version may be different based on your installation. | |
| Step 2 : Scala Installation | |
| If you done Scala Installation, then you can skip this step, otherwise you can install Scala 2.11.6 with the help of <a href="https://milindjagre.wordpress.com/2016/09/07/scala-2-11-7-installation-on-ubuntu-14-04/"> this </a> link. | |
| Step 3 : Download Spark Package | |
| wget http://d3kbcqa49mib13.cloudfront.net/spark-1.6.1-bin-hadoop2.6.tgz | |
| Step 4 : Extracting Spark Package | |
| sudo cp spark-1.6.1-bin-hadoop2.6.tgz /usr/local/ | |
| cd /usr/local/ | |
| sudo tar -xvf spark-1.6.1-bin-hadoop2.6.tgz | |
| sudo mv spark-1.6.1-bin-hadoop2.6 spark | |
| cd | |
| Step 5 : Update .bashrc file with following contents | |
| #SPARK VARIABLES START | |
| export SPARK_HOME=/usr/local/spark | |
| export PATH=$PATH:$SPARK_HOME/bin | |
| #SPARK VARIABLES END | |
| $cd | |
| $nano .bashrc | |
| ***PASTE ABOVE MENTIONED CONTENT IN THE FILE ANYWHERE*** | |
| press CTRL + X | |
| press Y | |
| press ENTER | |
| Step 5 : SPARK Installation Verification | |
| For verifying SPARK installation, you can run following command. | |
| $spark-shell | |
| This completed SPARK Installation. | |
| Thanks for having a read. | |
| Cheers. |
When you type $spark-shell command, you should get following output.
If you are getting same output, then cheers, Spark is installed successfully.
