48
edits
Changes
no edit summary
== What is Google App Engine ==
Google App Engine is a platform for building scalable web applications and mobile backends. It has a efficient Map Reduce Library that processes large set of data in map and reduce pattern in parallelism. This project will use this library to process a serious of searching history and produce a recommendation list for user. It support Java and Python and I will use Java in this project.
== Map and Reduce Library on GAE ==
In Map and Reduce Library, user code is only required for mapping and reduce function. The platform will shuffle and rearrange data order to make the upcoming reduce process extremely fast. This also reduces a lot of development work. For further boosting, the shuffle part can be rewritten Other big advantage comparing to boost up the speedMap Reduce on OpenMP is that programmer doesn't have to care about synchronization or resource allocation.
For further boosting, the shuffle part can be rewritten to boost up the speed. The original source code is [https://github.com/GoogleCloudPlatform/appengine-mapreduce/blob/master/java/example/src/com/google/appengine/demos/mapreduce/randomcollisions/CollisionFindingServlet.java here].
[[File:MPL.png|600px]]
== Map and Reduce Function ==
Map function:
In mapping process, program takes in a couple of user searching history as data. Choose a word randomly to compare, whenever a word among all searching history appears more than once it will be emitted (recorded).
[[File:map.png|600px]]
[[File:reduce.png|600px]]
In reduce process, sharded_num determines how many group of result will be left. If not defined, it's 0 by default. Lastly you will have a group of result that user searching for in common.
== Result ==
[[File:result.png|600px]]
== Conclusion ==
Goolge App Engine is a light wight but very powerful development platform that provides a lot of features. One of those futures that I used in this project is Map Reduce Library. It is very easy to use while doesn't require much coding. It process code in parallelism without any concern of synchronization.