Changes

When the action is triggered after the result, new RDD is not formed like transformation. Thus, Actions are Spark RDD operations that give non-RDD values. The values of action are stored to drivers or to the external storage system. It brings laziness of RDD into motion. reduce(func)

Aggregate the elements of the dataset using a function func (which takes two arguments and returns one).

System.out.println(result);

count() returns the number of elements in RDD.

The action take(n) returns n number of elements from RDD. It tries to cut the number of partition it accesses, so it represents a biased collection. We cannot presume the order of the elements.

The action collect() is the common and simplest operation that returns our entire RDDs content to driver program.

When we have a situation where we want to apply operation on each element of RDD, but it should not return value to the driver. In this case, foreach() function is useful.

//map to only words

JavaRDD<String> wordsRDD = removeBlankLineRDD.flatMap(sentence -> Arrays.asList(sentence.split(" ")).iterator());

//create pair RDD

JavaPairRDD<String, Long> pairRDD = wordsRDD.mapToPair(word -> new Tuple2<>(word, 1L));

Ssh into the master node of the cluster. Then issue a copy command to copy the jar file to the master node using:

Then you can run the app using:

[[File: output spark.png | 800px]]