Changes

The common MapReduce parallel programming we have covered in this course was arguably made famous by Google. It was used by the company to process a massive data set sets in parallel on a distributed cluster in order to index the web for accurate and efficient search results. Apache Hadoop, the open source platform inspired by Google’s early proprietary technology has been one of the most popular big data processing frameworks. However, in recent years its usage has been declining in favor of other increasingly popular technologies, namely Apache spark. We will introduce the history and advantages (scalability, flexibility, resilience) that led to the popularization of Apache Hadoop for certain big data applications. Our project will focus on demonstrating how a particular use case performs in Apache Hadoop versus Apache spark.