Changes

GPU621/Apache Spark Fall 2022

3 bytes removed, 21:16, 3 December 2022

→‎RDD APIs

JavaPairRDD<String, Long> levelRDD= logMessages.mapToPair(value ->new Tuple2<>(value.split(":")[0], 1L));

// (WARN,1) (ERROR,1) (ERROR,1) (FATAL,1) (WARN,1)

levelRDD .foreach(tuple2->System.out.println(tuple2));

1.5 reduceByKey(func)

JavaPairRDD<String, Long> levelRDD= logMessages.mapToPair(value ->new Tuple2<>(value.split(":")[0], 1L));

// (WARN,2) (ERROR,2) (FATAL,1)

levelRDD . reduceByKey ((a,b)->a+b) .forEach(t->System.out.println(t))

1.6 sortByKey()

92

edits