How To Install Apache Spark On Ubuntu 18.04

This brief tutorial shows students and new users how to install Apache Spark on Ubuntu 20.04 | xviii.04.

Apache Spark is an open source framework packaged with college-level libraries, including support for SQL queries, streaming data, machine learning and graph processing.

It is also capable of analyzing a large amount of data and distributed beyond clusters and processes the information in parallel.

If yous are a programmer who needs to produce seamless and create circuitous workflows, then Apache Spark is a peachy place to outset.

Getting started with installing Apache Spark on Ubuntu.

Install Java JDK

Apache Spark requires Java JDK. In Ubuntu the commands beneath can install the latest version.

sudo apt update sudo apt install default-jdk

After installing, run the commands beneath to verify the version of Java installd.

java --version

That should display similar lines as shown below:

openjdk 11.0.10 2022-01-19 OpenJDK Runtime Environment (build 11.0.ten+9-Ubuntu-0ubuntu1.twenty.04) OpenJDK 64-Flake Server VM (build 11.0.10+9-Ubuntu-0ubuntu1.twenty.04, mixed mode, sharing)

Install Scala

One parcel that yous'll also need to run Apache Spark is Scala. To install in Ubuntu, simply run the commands beneath:

sudo apt install scala

To verify the version of Scala installed, run the commands below:

scala -version

Doing that will display similar line below:

Scala code runner version 2.11.12 -- Copyright 2002-2017, LAMP/EPFL

Install Apache Spark

At present that you have installed required packages to run Apache Spark, proceed below to install it.

Run the commands below to download the latest version.

cd /tmp wget https://archive.apache.org/dist/spark/spark-2.four.vi/spark-two.iv.6-bin-hadoop2.seven.tgz

Next, extract the downloaded file and move it to the /opt directory.

tar -xvzf spark-2.4.half-dozen-bin-hadoop2.7.tgz sudo mv spark-two.4.vi-bin-hadoop2.vii /opt/spark

Adjacent, create environment variables to be able to execute and run Spark.

nano ~/.bashrc

And so add the lines at the bottom of the file and salvage.

export SPARK_HOME=/opt/spark export PATH=$PATH:$SPARK_HOME/bin:$SPARK_HOME/sbin

After that, run the commands below to utilise your environment changes.

source ~/.bashrc

Start Apache Spark

At this point, Apache Spark is installed and ready to use. Run the commands below to starting time it up.

commencement-master.sh

Next, start Spark work procedure by running the commands beneath.

first-slave.sh spark://localhost:7077

You lot can replace localhost host with the server hostname or IP address. When the procedure start, open your browser and scan to the server hostname or IP address.

http://localhost:8080

If you wish to connect to Spark via its control vanquish, run the commands beneath:

spark-beat

The commands above volition launch Spark beat.

Spark session available every bit 'spark'. Welcome to       ____              __      / __/__  ___ _____/ /__     _\ \/ _ \/ _ `/ __/  '_/    /___/ .__/\_,_/_/ /_/\_\   version 2.4.6       /_/           Using Scala version 2.11.12 (OpenJDK 64-Bit Server VM, Java 11.0.ten) Type in expressions to have them evaluated. Blazon :help for more than information.  scala>        

That should do it!

Decision:

This post showed you how to install Apache Spark on Ubuntu xx.04 | eighteen.04. If you notice whatever error above, delight use the form below to report.

Source: https://websiteforstudents.com/how-to-install-apache-spark-on-ubuntu/

Posted by: martinezmareake.blogspot.com

Share this post