Delta debugging framework bonsai direction
Objective
To write some PERL code that can query Bonsai and use the information available in the place of CVS (as CVS lacks changesets).
Description
The project itself needs, for the most part, three features from the repository: checkin details over a certain period of time, history of files (through revisions), and the ability to extract diffs to see what changed between two versions of a source file.
This code would use a two-step approach: queries would be pulled using Bonsai, code would be checked out using a pure-CVS wrapper (akin to the style Richard used for SVN).
Why Bonsai?
Lots of functionality is pre-baked, only needing to be extracted somehow.
History/What We Tried
This was also published on the main Delta debugging framework project page.
- Initially was going for a straight wrapper around CVS ala the style Richard used for SVN.
- Tried to find some functionality within Bonsai that could make it easier.
- Talked to Reed Loden, he set up a repository for us to try with. Thanks Reed!
- Thought that there may be some additional (read: unpublished) tools that could be worked with. Got in contact with some of the "Project Participants" listed on [1]. Was told the person in particular wasn't a contributor (just submitted a bug report). They in turn pointed me to #mozwebtools.
- Lurked on #mozwebtools for a few weeks. Talked to 'justdave' about Bonsai. Reed Loden chimed up and informed me that Bonsai can output to XML using ?xml=1 on the query (score! thanks again).
- Researched some PERL parsing utilities. Tried out XML::LibXML for DOM-style parsing.
- Explored a live installation of Bonsai using The Mozilla Project's Bonsai Installation. Tried to map the various pages and parameters (of the querystrings) to the functionality behind.
- Downloaded The Bonsai Project source code. See this page for details on getting the Bonsai source. Unfortunately there wasn't as much documentation as we had hoped for.
- Poked and prodded around, finding the CGIs of interest (and their respective params).
Exploring Bonsai
Of Interest to the Delta Debugging Framework
These particular CGIs in the Bonsai project are of interest to the Delta Debugging Framework project:
| CGI File | Description | Possible Benefit |
|---|---|---|
| cvsblame.cgi | Can see who did changed what in the repository. | May be useful to find out who introduced problem code. |
| cvslog.cgi | A web interface for "cvs log" | Good for extracting commentary on the what/why a change was made. |
| cvsquery.cgi | Displays the results of querying a module with certain critera (who, dates, files, etc.). | Good for querying high-level results; can see all checkins. |
| Main screen of Bonsai that you use to query with. | Not necessary as the PERL script simulates this action. | |
| cvsview2.cgi | View CVS diffs. | To extract the changes between two versions of a file. |
| moduleanalyse.cgi | Show directories in a module. | Good to map out the module if not known ahead. |
| multidiff.cgi | Show ALL of the diffs. | Don't have to circumvent pesky pagination with the query results! |
| rview.cgi | Lets you browse the contents of a directory. | Perhaps useful in the discovery of the project contents in question. |
| showcheckins.cgi | Show some set of checkins in a bonsai hook. | Of questionable benefit at the current time. |
How/Particulars
In researching and exploring the options, we discovered that Bonsai can actually output to XML by attaching the param/value pair "xml=1" to the end of a query using the Bonsai CGI scripts. (Thanks go to Reed Loden for pointing this out to us.)
This enabled us to probe deeper and investigate interfacing with CVS by way of Bonsai by building a request in PERL, querying Bonsai, and parsing the XML response. Further research was put into finding an appropriate parsing means in PERL (no experience in this arena). Turned out that PERL has a DOM-style parsing module named XML::LibXML (among many other XML parsers available).
The XML::LibXML module is rather easy to use. For example, to query Bonsai for Checkins in the last 2 hours to the Mozilla trunk, and iterate through them, the below code works:
#!/usr/bin/perl
use LWP::Simple;
use XML::LibXML;
#query Bonsai
my $xmloutput = get "http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=HEAD&branchtype=match&dir=&file=&filetype=match&who=&whotype=match&sortby=Date&date=hours&hours=2&mindate=&maxdate=&cvsroot=%2Fcvsroot&xml=1";
my $parser=XML::LibXML->new;
my $domtree=$parser->parse_string($xmloutput);
my @checkins=$domtree->getElementsByTagName("ci");
foreach my $ci (@checkins) {
# who and when, gross dumpage
print $ci->getAttribute("who") . " @ " . $ci->getAttribute("date") . "\n";
}
Concerns and Thoughts
Accessing information through Bonsai wouldn't be overly difficult. However, trying to make sense of changesets themselves (the concept of, and what constitutes) might be difficult as CVS doesn't have the notion of a changeset. This means that either the programmer would either define a set of rules on what does/does not make a changeset OR there is something that already exists in Bonsai (which has been overlooked--and that's not hard to do).
Additionally, it is apparent that not all CGIs of Bonsai support the xml parameter (and subsequent output). This works on cvsquery.cgi, but is lacking (from what we can see) in the others. Had we enough time, this might not have even worked.
Roadblocks
- No previous exposure to Bonsai, CVS or source repository concepts in addition to a total absence of PERL skills.
- By the time we discovered how to do it, we ran out of time in the course.