Open main menu

CDOT Wiki β

Changes

GPU621/Apache Spark Fall 2022

161 bytes added, 15:32, 30 November 2022
Create And Set Up Spark
===Create And Set Up Spark===
Spark needs to be set up in a cluster so first we need to create a JavaSparkContext object, which tells Spark how to access a cluster. But To create a SparkContext you can also run it locally and act as first need to build a clusterSparkConf object that contains information about your application. We will talk about how to set up a spark in a cluster later. Now let's try to create a spark locally. To do that, we will need the following code:
//create and set up spark
92
edits