Mercurial history browsing

Project Name

Adding Functionality to the Browser Based UI for Mercurial History Browsing, hgweb

Project Description

Mozilla is just starting to use an exciting new distributed version control tool: Mercurial.

One of the exciting things about Mercurial is that the history—the list of all the changes that have ever been checked in—is not linear. It frequently has branches and merges. This is actually a good, important feature, as you know if you've read a little about distributed version control. The downside is that the history becomes a maze of twisty little passages. Benjamin Smedberg's demo shows the history of a repository with lots of merges (each box is a check-in; you can click on the boxes to move around).

There have been a couple attempts at showing history in an intuitive, graphical way. Mercurial comes with a web UI for browsing the repository, including history (here's what it looks like). It also comes with an "hg glog" extension that draws history as ASCII art, and an "hg view" extension that does roughly the same thing with a little GUI. Sadly, none of these applications qualifies as awesome.

The project's goal is to rectify this sad situation by writing an awesome browser-based UI for navigating Mercurial repository history.

Resources: jorendorff, bsmedberg

Project Leader(s)

Siddharth Kalra

Project Contributor(s)

• Tiago Moreira

Project Mentors

• Jason Ordendorff (jorendorff)
• Dirkjan Ochtman (djc)
• Ted Mielczarek (ted)
• Benjamin Smedberg (bsmedberg)

Project Ideas

September 23rd, 2008

Hgweb ideas

Chat with jorendorff We have to make sure you understand hg we have to decide on some very small thing that you can get running very soon we should talk it over and make sure it's clear how to do it we should plan to meet up later and brainstorm some, once you've had a chance to play around with hg Ideas from the blog post by jorendorff jorendorff's ideas History scroll forever, up and down. Something like a tumblelog Able to identify whether what one is looking at is in the mainline or a branch Major lines of development color coded i.e. blue for mozilla-central, orange for tracemonkey etc... This is possible using information from those repos’ pushlogs Pushlog info could be used to make those special paths relatively straight, with other smaller branches and mini-merges happening to the side or hidden by default with some sort of collapse/expand widget Show more info about each changeset, but only in an unobtrusive way Be able to know which dirs were touched and roughly the size of the diffs Be able to filter the history by file or dir A vertical timeline Location of a changeset on screen would tell something about when it was developed or pushed Be able to zoom in and out and see weeks, months, years of work Showing dates of when changes were "pushed" to mozilla-central, not the dates they were "committed" This makes the history linear and to some degree shows if a merge is big or small Jesse Ruderman Ideas Consider starting with the pushloghtml page, which the Moz community seems to use, rather than the shortlog page Gc Ideas Search history restricted to a particular data range and dirs because when a bug is found to start on a particular build from a month ago, need to find what was changed since the previous build, in relevant dirs RyanVM Ideas Be able to view more than 10 entries at a time in the changelog Jonas Sicking Ideas Uses HG web to digg thorugh logs for a given file Wondering "why was this code changed" or "what is this code trying to do" Today it is painful to dig this info up HG Blame allows you to see who last changed a given line, but many times that is just cleanup patches that don't change the functionality ability to see who made changes that actually changed the functionality Bonsai had better functionality Showed checkin comment for every given line and jump to that revision You could then manually step on revision back since the version numbers were numeric HG log should be able contain links to the previous version for every line Boris Bonsai had the feature of being able to see the list of files touched by a checkin Clint Anything to make hunting down regression ranges Able to query by the person that checked in the patch and by source code directory.

January 15th, 2009

DPS911 - Project Description and Plan

I will be continuing my work on hgweb for this new course. I’ve had a talk with jorendorff and ted regarding the bugs I can try to fix. I have various options that I can take on. There are definitely things available that I can continue to work on throughout the semester. Ideas for Releases bug 459727 - pushloghtml should show more than 10 entries at a time There are still issues with this bug that need to be fixed. The basic functionality works but there are some things that still need fixing. Things like the buglink functionality, loading more entries according to screen size, optimizing the code for efficiency etc still requires work. History that is filtered by directory Filtering the pushlog according to the directories touched. According to jorendorff this is a large feature that will require work. I told him that we were required to put out releases every 2 weeks so he recommended that I choose one large feature to work on (this one). I can implement this feature and improve its functionality for at least a few releases. Hg Annotate is very slow This is a bug that people have been working on for a while. Files using Hg Annotate are taking forever to load in any browser in comparison to loading the same file in bonsai. This is a medium difficulty bug that I could fix in maybe 2 releases or so. Expand/Collapse functionality for files touched Last semester I fixed a bug that added files touched functionality to the pushlog. My bug passed the review but now I want to add expand/collapse functionality for the files touched. Goals My goals are to give priority to the bugs that users want done. I will probably start by working on the pushloghtml should show more than 10 entries at time bug and then see what I want to work on next. Ideally I would like to get all the four ideas I have listed above done but it is hard to say whether that will be possible. Inevitably there will be issues with my patches and I will have to probably dedicate some releases to bug fixing. Possible Problems Last semester’s experience with this stuff means that I won’t have a large learning curve. It should be smooth sailing but problems will definitely creep up. I will probably get stuck and will require help. In that scenario I will need to consult somebody with more knowledge than myself. Usually ted, jorendorff and djc are available but if they are not, then there might be issues. My project doesn’t involve waiting for anything or for anybody to do anything (except when I will be waiting for reviews) so in the end everything is on me to get things working.

Project Details

v0.1 Release Details

• to get pushloghtml working on my machine (some progress made on this part)
• to locate the source files that dictate the amount of pushlog entries that get displayed and then alter it to display more entries
  • Turns out that there was some miscommunication and I need to add an AJAX scrolling feature that dynamically loads more changesets
• to add code that allows expand/collapse functionality (apparently all I need is a js lib according to humph)
  • Turns out that there was another miscommunication as Ted has already added this feature. Look here. However the patch has some problems that will need to be fixed. I might fix this for my 0.2 release (undecided for now)
• To release v0.1 on time

v0.2 Release Details (More Information) (Final result)

• To continue work on bug 459727
  • Implement a loader GIF (View here)
  • Getting the correct Chronological order when adding more entries OnScroll (View here) (Final result)
  • Getting merge changesets to show up correctly when adding more entries OnScroll (View here) (final result)
    • Making sure the new entries that load OnScroll comply with bug 445560 (Expand/Collapse feature) ==> REMOVED from v0.2 - this functionality should wait until the expand/collapse functionality itself is approved and goes live. This should be a separate bug
  • To make buglinks work
  • To make changeset links work
  • To get the correct date formating
  • On first page load show 20 entries instead of 10 (View here)
  • To get the correct author names (right now, sometimes it returns the email address while other times the name of the author) (View solution here)
• To fix ted's patch for bug 445560 (View here)
• To implement "the which files were touched by a changeset" feature for bug 448707 (View here)

v0.3 Release Details (View here)

• Problem with my patch for bug 445560 (View here)
  • Fixed problems with my patch by adding 2 new patches (View here)
    • Fix 1: Should use ids instead of dates
    • Fix 2: Should store unique identifiers in the class instead of the id of the tag
• My patch for bug 448707 approved! (View here)
• Implementing the Graph view for the pushlog
  • Downloading Dirkjan's (djc) Graph view's source code (View here)
  • Getting djc's Graph view running on my computer (localhost) with hg serve (View here)
  • Understanding how djc's Graph view works (View here) (View here 2)
  • Coding the Graph view for the pushlog
    • 1st interation - Make it look similar to the coal template's graph view (View here)
    • Final Result of the Graph view (View here)
      • Added bug link feature
      • Formatted correctly to look similar to other gitweb_mozilla pages
      • Added on mouse over tooltip functionality
• Problem with my patch for bug 459727 (View here) (fix for my patch)
  • To calculate the max use SELECT COUNT(*) FROM ... instead of asking for all the rows and then counting them
  • getMaxEntries called, then 'start' used once the user scrolls. But the user might scroll before the first result comes back, while start is still zero -- a race condition.
  • The JS code isn't consistently indented
  • Patch seems to contain some non-ASCII characters. Bugzilla doesn't render it properly. Consider using an HTML entity reference, like – or —, instead of an actual Unicode character.
  • In a few places the b tag is used where the server uses the strong tag; be consistent
  • Loads more results when the user scrolls. What if the initial results don't fill up the window?
  • Better to move CSS and formatting to stylesheets where possible
• Problem #1 explained in comment #9 by jorendorff still existed with my new patch (it is fixed, view here)
• Improved the bug link feature for bug 459727 (View here)
• My patch for bug 445560 approved! (View here)
• v0.3 complete!

Project News

September 18th, 2008

• Picked the Mercurial History Browsing project

September 23rd, 2008

• Had the first conversation with jorendorff
  • Talked what I know about mercurial and python
• Chatted with jorendorff, ted and djc about starting on the project.
  • Brainstormed some ideas about what to do during the project
• Jorendorff made a blog post calling for ideas about the project

September 25th, 2008

• Chatted with jorendorff on setting my development environment
• Email conversation with jorendorff, asked some preliminary questions about the project and a hello world type of activity
  • jorendorff replied
    • I replied to jorendorff concerning setting up my development environment

September 26th, 2008

• Email conversation with jorendorff about setting up the development environment

September 28th, 2008

• Played around with hg. Details here
• Email conversation with jorendorff, asked for help on setting up the development environment and suggestions on a v0.1 release

September 29th, 2008

• Email conversation with djc about setting up hg_templates
• Email conversation with jorendorff about having the development environment completed and ready to use and picking what I will be working on for my v0.1 release ideas

October 1st, 2008

• Chatted with ted about getting a hold of the pushlog database
• Email conversation with jorendorff about setting up pushloghtml on my development environment and some questions/concerns regarding pushloghtml

October 2nd, 2008

• Development Environment almost setup and running except for pushloghtml
• Decided on v0.1 release (tentative, still need to clear it with humph)
• Email conversation with jorendorff clarifying some concerns about pushloghtml
• Email conversation with ted explaining how to setup pushloghtml, concerning the buglink.py file

October 3rd, 2008

• Decided on v0.1 release (confirmed with humph). Updated v0.1 release blog post
• Email conversation asking about simplejson and pysqlite2 modules
• Email conversation with bsmedberg explaining simplejson and pysqlite2 questions

October 5th, 2008

• Problems with getting pushloghtml to work here. Work in progress...

October 6th, 2008

• Multiple email conversations asking djc and bsmedberg about a "no module found" error relating to simplejson and pysqlite2
  • Conversation with me asking why the error is happening
  • djc replies asking whether I get a traceback
  • I reply with the traceback output
  • bsmedberg replies with possible causes
  • I replied with various things that I have tried to get it to run properly

October 7th, 2008

• Another email conversation with djc, discussing ideas about the causes of the "no module found" error

October 8th, 2008

• Email conversation with djc asking questions about building Mercurial from source. I was having some trouble with it

October 9th, 2008

• Email conversation with djc discussing further issues with building Mercurial from source
• Email to let jorendorff, djc, ted and bsmedberg know that the chronic problems with Mercurial have been resolved
• Development environment setup. Ready for coding to begin!

October 13th, 2008

• Filed the bug: pushloghtml should show more than 10 entries at a time
• ted has already filed a bug for the expand/collapse feature: html pushlog should fold merge changesets into one line
• Chatted with jorendorff and ted about setting up my development environment
• Chatted with ted about the expand/collapse feature

October 14th, 2008

• Applied Ted's patch to my development environment to fix the bug: html pushlog should fold merge changesets into one line, check out my blog post for details
• Chatted with djc about pushloghtml
  • This is where the confusion occured, I thought they wanted to load more items staticly when they wanted an OnScroll features
  • I asked them what AJAX library I should used
• Chatted with ted about a problem with applying his patch

October 16th, 2008

• Email conversation with ted discussing some questions about implementing an AJAX scrolling feature to load more pushlog changesets
• ted replied with possible solutions/steps I could take

October 17th, 2008

• Email conversation asking questions about the json-pushes script

October 18th, 2008

• Made a blog post regarding bug 459727 - Pushloghtml Should Show More Than 10 Entries, Working on v0.1 Release
• Email conversation asking ted about xmlHttpRequest and JSON questions

October 19th, 2008

• Edited the blog post regarding bug 459727 - Pushloghtml Should Show More Than 10 Entries, Working on v0.1 Release

October 23rd, 2008

• Email conversation with djc regarding bug 459727. In particular asking about the JSON script that returns turns the data from the server side, which is gotten via an xmlHttpRequest()

October 29th, 2008

• Had a conversation with djc regarding bug 459727
  • Discussed how my project was progressing
  • Asked him a question related to the email conversation we had
• Had a conversation with ted regarding the bug 445560 expand/collapse feature
  • Asked him what his patch doesn't do properly
  • Wanted to find out what needed to be fixed

November 3rd, 2008

• Had a conversation with djc regarding the date formatting for bug 459727
  • Asked how the date is formatted in the way it is right now

November 4th, 2008

• Had a conversation relating to v0.2 release with jorendorff (decided what I will include in v0.2)
• [http://sidkalra.com/files/mercurial/email/conversation_email_Nov04-08.txt Email conversation with djc regarding bug 459727 (the OnScroll feature). Asked questions the data that the JSON script returns
  • djc replied with possible solutions/methods to solving my problems

November 5th, 2008

• Made a blog post relating to bug 459727

November 6th, 2008

• Had a discussion with jorendorff about how to implement the which files were touched by a changeset feature

November 7th, 2008

• Made a blog post discussing the details for v0.2 release
• Made a blog post about a problem with implementing merge changesets for bug 459727

November 8th, 2008

• Made a blog post adding a loader GIF for bug 459727
• Made a blog post about fixing bug 445560

November 10th, 2008

• Had a discussion with ted about the which files were touched by a changeset feature
  • Asked him about the back-end code related to this bug
• Had a discussion with jorendorff about the which files were touched by a changeset feature
  • Showed him what I had done so far and asked him if he was OK with my implementation

November 11th, 2008

• Made a blog post about fixing bug 448707
• Had a chat with jorendorff and ted regarding whether bug 459727 should comply with bug 445560 from the get go
• Made a blog post about showing the correct author for new entries OnScrollDown for bug 459727
• Made a blog post regarding bug 459727: On initial PageLoad the pushlog should load 20 entries instead of 10
• Made a blog post concerning a problem with the chronological order for bug 459727

November 12th, 2008

• Had a discussion with ted and bsmedberg about the issue explained by the blog post here
• Edited the blog post about not getting the right author name. The edit provides a solution to the problem
• Edited the blog post detailing my problem with ctx.parents()[i].description()
• Chatted with ted about a problem with not getting the correct author name. Sometimes the email was being shown while other times the actual name was showing up

November 13th, 2008

• Had a long discussion with jorendorff about fixing the issue described by the blog post here
• Edited the blog post detailing the problem and fix with ctx.parents()[i].description()

November 15th, 2008

• Made a blog post to add merge changeset functionality to bug 459727

November 16th, 2008

• Posted patch for bug 459727
• Posted patch for bug 445560
• Posted patch for bug 448707

November 19th, 2008

• Had a discussion with jorendorff about what to do for my v0.3 release

November 24th, 2008

• Made a blog post about looking forward to my v0.3 release
• Had a discussion with ted about fixing my patch for bug 445560
  • ted identified a problem with the patch
  • I said I would fix it
  • Made a blog post regarding this discussion
• Posted new hgpoller patch for bug 445560
• Posted new hg_templates patch for bug 445560
  • Made a blog post about the two new patches for bug 445560
• Emailed djc asking questions about his graph view implementation

November 29th, 2008

• Email conversation with djc. He replied to my earlier email asking about his graph view implementation

December 1st, 2008

• Had a discussion with jorendorff on the Graph view
• Made celebratory blog post about my patch for bug 448707 being accepted
• Made a blog post about understanding how graph data is passed from the server side to the client side

December 2nd, 2008

• Made a blog post about completing the 1st iteration for the graph (View here)
• Made a blog post about problems identified by jorendorff concerning my patch for bug 459727

December 3rd, 2008

• Added a new patch for hgpoller for bug 459727
• Added a new patch for hg_templates for bug 459727
• Blogged about the above patches here
• Made a blog post explaining a fix concerning a problem with my newly submitted patch (see above) for bug 459727. Problem #1 explained in comment #9 by jorendorff still existed even with my new patch

December 4th, 2008

• Made a blog post about improving the bug link feature for bug 459727

December 5th, 2008

• Had a chat with jorendorff and ted
  • Talked about who should be the reviewer for bug
  • Talked about continuing on and working on this project next semester
  • Final discussion for the project for this semester
• Made a blog post about another patch of mine getting approved for bug 445560

December 7th, 2008

• Made a blog post about implementing the graph view
• Made a blog post about completing v0.3 release

START OF DPS911

January 2009

Date		Comments
January 13th, 2009		Had a discussion with jorendorff and ted about what I will be working on this semester.

Project Bugs

• bug 459727 - pushloghtml should show more than 10 entries at a time
• bug 445560 - html pushlog should fold merge changesets into one line
• bug 448707 - display files in pushlog
• bug 468089 - pushlog should have a graph view

External Links

• Distributed Version Control
• Mercurial
• Mercurial Basics
• mercurial-central shortlog page
• graph view
• better shortlog
• mozilla-central pushlog
• Desired features for hgweb
• jQuery tutorials
• Using XMLHttpRequest
• bonsai
• hg annotate problem
  • hg annotate output on hgweb is painfully slow
  • http://www.selenic.com/mercurial/bts/issue1310