Portable DXR

Name of Project

Portable Dehydra Cross Referene (PDXR)

Description

This project aims to turn the DXR into an extension that can be installed in Firefox or distributed as a XUL/Prism application. This will allow developers with limited or no internet connection to access the Mozilla source code and documentation in an offline environment. The extension will also provide a low-bandwidth solution for applying updates on a timed interval or at the user's command.

The DXR is a compilation of Mozilla developer documentation. The plan is for this material to be distributed on a Fedora Live DVD that will allow the user to access these materials offline or, alternatively, install them for use on a Windows machine.

Currently the Local MXR and MDRK exist as web applications that run only on Fedora and require the installation of MySQL, Perl and apache. These projects have to be setup manually and are not installable. Our part in this project would be to convert these projects so that they can be installed cross-platform as a Mozilla Firefox extension.

Leaders

James Boston

Samer Ziadeh

Jerry Pau

Details

Related Projects

• Mozilla Developer Resource Kit: This is the over-arching project of which the PDXR is but one piece.
• Creating an MDRK spin: This is how this project will eventually be distributed.
• Create a local mxr: It is hoped that by solving the more difficult problem of packaging the dxr, the problem of packaging other materials such as an mxr or developer documentation will also be solved.

Technical Overview

Business Rules

• Must not sell project for profit
• Commit to lowest file size bandwidth

About The Source Code

• Backend: Perl
• Database: SQLite
• Interface: XUL

Perl Interpreter

The Perl parser will be needed by the files in charge of talking to the internet and the database. It should be able to connect to the update server and download the update log, then apply the updates to the database accordingly. So far we found perlembed. It allows us to use Perl from C, which means that we could compile our program with headers that will be able to alternate between Windows, Linux, and Mac.

There is also another nice find. This one is called Strawberry Perl.

"A 100% Open Source CPAN-capable Perl for Windows® computer that works exactly the same as Perl everywhere else."

Other Finds:

• PerlMonks
• use Perl
• Wikipedia Perl

Web Server

We are still researching for a light-weight webserver. This section be updated regularly.

Our picks (so far):

• SHTTPD
• dlib C++
• Cheetah Server
• Pure Perl CGId
• nginx (Embedding example code)

List of web servers:

• Wikipedia's List
• IBM's List

Features

• Running server side Mozilla code on the local machine. The extension will have an embedded server that will handle the server-side scripting.
• Notify the user that their database may be out-of-date 30 days after last update.
• Option to allow compression of data download.

Users

• Developers with limited or slow internet connection

Architecture

The architecture is still undergoing development and this section will be updated regularly.

High level conceptualization

Mozilla-Central

Mozilla-Central is the main repository for Mozilla source code. This is the raw fodder out of which documentation will be created. How it is created doesn't really appear on our diagram unless you count the "How does this happen?" note.

Well, how it happens is through a static analysis tool called Dehydra. In layman's terms, Dehydra reads the Mozilla source code and annotates it in a way that can be turned into documentation in an automated fashion. This part is another project entirely and as it relates to our project, the relevant work is being done by David Humphrey. However, our team will need to setup the Dehyra tool to feed our update server.

Update Server

Although this project is all about building a tool for browsing documentation off-line, that documentation is in a constant state of flux. It would be useful to be able to refresh it when need be. The update server needn't be terribly complex. A simple Apache server and some CGI work should suffice.

However, it needs to be worked out exactly what kind of update is being pushed. Is it a binary diff of the sqlite database? And if so, will it need to keep a collection of diffs based on a original dataset in order to service users who haven't sync'd for some time. There's also the question of whether the update mechanism build into Mozilla for extensions is useful to us. We need to do some research on how extension updates work.

Application

Web Browser

The application front-end holder. It provides a way for the application to be accessed. It is not limited to Firefox web browser only, it can also be run as a Prism application.

Extention

Portable DXR (our browsable source code documentation) is an application that lives inside our extension. The CGI tools exist as part of an embeddable web server. When you remove the DXR part, what you are left with is the platform on which it runs. It's also worth pointing out that the SQLite support is available in two ways. Local storage capabilities using SQLite are build into Mozilla applications and can be accessed by an extension using C++, JavaScript, Python. However, SQLite database files can be accessed by any software that understands the format, so SQLite support will also be available through the cgi tools. The DXR application will probably use the CGI tools to access SQLite files if our presentation layer is traditional XHTML. But if we decide to use XUL, we can use the browser's built in functionality directly from JavaScript. In either case, I suspect we will end up leveraging the built in sqlite support for the updates. But we need to do some research on that.

DXR

The presentation. The front-end is mostly implemented by David Humphrey on his DXR. We need to modify the UI to implement the following functionalities:

• Navigation
• Database querying
• Update services
• Data transaction

UI

• Navigation
• Tree:
  • Source Code
  • Documentation (Source code documentation)
  • Help (PDXR Documentation)
  • Options
    • Automatic updates
    • Prompted updates
    • Session saving

Roadmap

• Port DXR (1 week)

• Modify UI (4 weeks total)
  • Left panel:
    • Source Code
    • Documentation (Source code documentation) (1 week)
    • Help (PDXR Documentation) (2 weeks)
    • Options (1 week)
      • Automatic updates
      • Prompted updates
      • Session saving

• Implement the web server (1 week)
  • Test the web server on the different platforms

• Implement the Perl parser (1 week)
  • Test Perl scripts on the different platforms

• Code the CGI tools (4 weeks)
  • Perl talks to the update server
  • Perl updates the database

News & Updates

Meetings (Chronological)

Sept 26, 2008: Discussion with David Humphrey about project scope.

Oct 2, 2008: Discussion with David Humphrey about technical details of DXR.

Oct 8, 2008: Team meeting to discuss Work Breakdown Structure (WBS)

Oct 16, 2008: Team meeting to discuss technical details of project

Oct 29, 2008: Further discussions with David Humphrey about documenting the project.

Oct 30, 2008: Group meeting to map out high level design.

Nov 5, 2008: Team meeting with David Humphrey about progress.

Team Blogs

• James Boston: All relevant posts are tagged 'pdxr'.
• Samer Ziadeh
• Jerry Pau

IRC

• #PDXR
• #seneca

TODO

• Need to get a perl (and python?) interpreter into the extension or into an installer so it lives locally as well (by 27/11/08) (Samer Ziadeh).
  • Must work within the extension
  • It probably doesn't need to be ligthweight.
  • Needs to work with the extension on Windows, Mac, and Linux

• Need to figure out how to mash a web server into an extension. (by 27/11/08) (Jerry Pau)
  • Which Mashup should we use?
    • Consumer Mashups
    • Data Mashups
    • Business Mashups
  • Do we make sure the web server is up to date?
    • How would we do that?
    • Are we going to add options in the setting regarding to the updates?
      • Automatically check for updates
      • Prompt before download updates
      • Prompt before installing updates
      • Never ask again

• Explain how these work: (by 27/11/08) (James Boston)
  • Get updates from David's DXR or Mozilla-Central.
  • Push updates to the extension.

• Design of the interface of PDXR that will be using Dehydra (by 27/11/08) (Jerry Pau & Samer Ziadeh).