1
edit
Changes
no edit summary
== Project Name ==
'''BuildAPI'''
== Project Description ==
The BuildAPI project consists of generating analytic reports which can be used for a multitude of purposes such as performance and usage tests or discovering rogue buildbot slave machines. Moreover, it will require querying a database for the information on the BuildBot jobs, which will then be used to generate the necessary reports.
Initial contacts: [[User:Armenzg|Armenzg]]
== Project Leader(s) ==
Andrew Singh [http://zenit.senecac.on.ca/wiki/index.php/User:Asingh]
== Project Contributor(s) ==
* [[User:Armenzg|Armenzg]]
* Google Groups - Pylon-Discuss [http://groups.google.com/group/pylons-discuss]
* IRC: aki, cthomas
* [[User:ctyler|Chris Tyler]]
== Project Details ==
* '''Updated Objectives (October 28th 2010) from [[User:Armenzg|Armenzg]]'''
* We will choose one of the following to do:
** 1) Generate graphs, charts, CSVs and CPU totals for infrastructure load blog posts
*** a) This is very useful and could move us forward towards having this information being published publicly for consumption
*** b) I highly encourage this one as understanding the mental model behind it is easier
** 2)Write a tool that analyzes our statusDB and figure out slaves that have been continually been burning jobs (sometimes it takes us several days to spot them)
''* Objective: To analyze and create reports from BuildBot data for multiple purposes. These reports will generated from querying buildbot databases to get the necessary information about jobs.
* It will be used for purposes such as:
:* identifying critical machines that should not be processing jobs
:* discovering rogue slaves
:* Generating reports on performance, usage, build times, downtime, and more''
* This project uses Pylons framework:
** Armen suggested that one student works on the first graph, and the other works on the second graph
** We will also be working on the documentation
== Resources ==
* Pylons-Discuss Group http://groups.google.com/group/pylons-discuss
* Mozilla Software Engineering group (see BuildAPI wiki)
* Channels such as #build, #pylons and #seneca or irc.mozilla.org and irc.freenode.net
== Project Plan ==
Goals for each release:
* 0.1
** Complete a local installation of Python, Pylons, MySQL (with databases snapshots loaded) and google virtualization
** Set up BuildAPI and be able to run the Paste server to use the pre-existing BuildAPI controllers
** Create my '''own''' controller, models or views to test and understand Pylons concepts
* 0.2
** Pull information from the database and display it on a webpage USING the MVC model and integrating with the provided query.py script
** Would like to generate a simple graph from the information that is pulled from the database
** Note: Still need to understand how JSON and Java is being used with the existing scripts. For now I am comfortable with the MVC model and can apply it practically, for the most part.
* 0.3
** Full documentation provided on how to set up BuildAPI: http://asdesigned.ca/sbr/BuildAPI-How-To-Setup.pdf
** Reflections on project and course posted on blog [http://andrewasdesigned.blogspot.com/]
** Placed the data into JSON output
== Components to Learn/Get an idea of ==
* Pylons Framework
** Read chapters from The Definitive Guide to Pylons book
* How to integrate GoogleAPI and JSON output with Pylons
* Python
* MVC (Model, View, Controller) Model
== Accomplishments ==
* Created my own "project" using a controller, model and template.
** The model uses the existing query.py file but I added my own queries.
** The controller is called project.py, imports my queries from the query.py file and renders the template project.mako with the query results
** I have removed the json because I do not fully understand it and I am taking it step by step
* BuildAPI set up on http://iraq.proximity.on.ca:5000
* Python, Mysql-Python, Google Visualizations API, BuildAPI installed locally on a virtual machine on my laptop
* Databases installed from sql dump file
* Able to run Paste server and BuildAPI pre-existing scripts
* Able to run in from both regularly and from a python virtual environment (used for testing)
* Created my own test script using a controller to display information from a selected table in the schedulerdb database
== Issues ==
* '''Update: Resolved all below issues'''
** I tried to install the same on our Seneca CDOT server: iraq.proximity.on.ca so that I could work off of the server but I ran into errors
** Tried regular installation as well as using a python virtual environment
** Serving content on iraq.proximity.on.ca
== Project News ==
* Completed Documentation for setting up BuildAPI
** See Blog Post or here
* November 18th, 2010
** Updated some information on the page
** Made substantial progress learning Pylons and MVC framework
* October 28th, 2010
* What has been done so far
** Please see my blog at: http://andrewasdesigned.blogspot.com/2010/10/beginning-previously-we-had-conference.html
** I have been moving forward and have installed BuildAPI locally on my machine to test it out
** Still need time to finish reading on Pylons!
* October 22nd, 2010
** New [http://people.mozilla.org/~catlee/schedulerdb-2010-10-22.sql.bz2 schedulerdb snapshot] available.
* October 14, 2010:
** Meeting of previously named "ScriptFactory" broke into three seperate projects: [[Mozharness]], [[BuildAPI]] and [[Release Simple Bugs]] -Mustafa
** The ScriptFactory group met today with Armen and Ctyler in a conference call
** We will separate into two groups and tackle Mozharness, BuildAPI or Release Simple Bugs
== Links ==
* Andrew Singh: http://andrewasdesigned.blogspot.com/
* Armen Zambrano Gasparnian: http://armenzg.blogspot.com/
* Planet CDOT: http://zenit.senecac.on.ca/wiki/index.php/Planet_CDOT
* BuildBot Architecture: http://buildbot.net/buildbot/docs/0.8.1/images/overview.png
* Database Snapshots: http://people.mozilla.org/~catlee/schedulerdbschema.png** and http://people.mozilla.org/~catlee/schema.png ''Updated by Andrew Singh''
