GPU621/Apache Spark
GPU621/Apache Spark
The common MapReduce parallel programming we have covered in this course was arguably made famous by Google. It was used by the company to process a massive data sets in parallel on a distributed cluster in order to index the web for accurate and efficient search results. Apache Hadoop, the open-source platform inspired by Google’s early proprietary technology has been one of the most popular big data processing frameworks. However, in recent years its usage has been declining in favor of other increasingly popular technologies, namely Apache spark. We will introduce the history and advantages (scalability, flexibility, resilience) that led to the popularization of Apache Hadoop for certain big data applications. Furthermore, our project will focus on demonstrating how a particular use case performs in Apache Hadoop versus Apache spark, and how this relates to the rising and waning adoption of Spark and Hadoop respectively.
Group Members
- Akhil Balachandran
- Daniel Park
- Patrick O'Reilly
