Open main menu

CDOT Wiki β

Changes

Code Indexer

475 bytes removed, 13:19, 16 January 2007
Status
==IRC Talk about CVSUP==
Ok, what i would be interested to know is if i could get fully versioned histories of specific branches, not a versioned copy of the whole code base, due to technical limitations. If this versioning information has a bug number and a reviewer, i can easily implement Blame and version histories into the OpenGrok, making it ideal (imo). Please add your comments as if this were the talk page
 
 
 
==Project Name==
Source Code Indexing Service Analysis
==Project Contributor(s)==
==Project News and Details======Status====* I got OpenGrok now have some time to server, but unfortunatley it gives me a read error permission, so I am re-do ing resume work on the installationproject. --[[User:John64|John64]] 20:15, 23 October 2006 (EDT) This time I will manage my time more wisely* Due to a huge number of assignments <strike>I have, this project has stalled until at least reading weektoo many assignments. I need to do work on them, but I should have lots of more free time during or soon after reading week to spend on this! --the Christmas break.</strike> ==Options==* Help with [http://lxr.mozilla.org/ LXR]/[Userhttp:John64|John64//landfill.mozilla.org/mxr-test/ MXR]/[http://www.mozilla.org/bonsai.html Bonsai] 10:02, 19 October 2006 (EDT)development.*'''Update:''' Since Make a sort-of branch respectful version of [[OpenGrok ]] but this would be a very shoddy implementation that doesn't really do file what it should* Setup one OpenGrok per active branch of the Mozilla Project, this would have no version trackinghistory whatsoever, There is no point in indexing the entire Mozilla code baseapart from file dates. I will be pulling down a copy * [Re]write major portions of how [[OpenGrok]] deals with history and changesets and the Mozilla source tree likes, this is my personal preference.* Try to fit [http://www.cenqua.com/fisheye/index instead.html Fisheye] into the current development model, but it seems this might be more like finding a problem for a solution. This shouldn't have is a very very powerful tool, but it is not really like LXR or OpenGrok, it is more useful to analyze CVS/SVN histories more than search for functions, files, definitions and the intense footprint as an rsync'd Mozilla tree --[[User:John64|John64]] 00:01, 10 October 2006 (EDT)* likes. Fisheye is also '''Issue:extremely''' OpenGrok is very demandingslow. It has allready filled Within my lan, it takes a 12GB VMware imagelong time to do any queries, and i still dont over the internet it is impossible. I don't know how large why, especially since OpenGrok uses the index growssame basic technologies. This could be (10 minutes plus for one page load on a problemlan connection, in its defence, it is still indexing the code) ====Why I like OpenGrok====Apart from the fact that it does not because i dont have spacesupport branches, but because this may be too much for is in my opinion the school serverperfect tool. Likely OpenGrok would need a dedicatedIt is fast, open souce and most importantly, it makes really easy to navigate, well thought out pages that just work. Because of the way OpenSolaris does file versions for their code, reasonibly fast machinethey don't use branches at all. It seems to be OpenSolaris currently uses a linear method of file versioning, they don't use branches, they use versions as a very powerfulsory of branch, extremely well done application but it basically the idea that Office 12 is the "2007 branch" and Office 11 is missing the ability to associate "blame2003 branch" from what I see. I also donMozilla doesn't know how do this, so it takes care of file versioningwould be nessecary to implement this feature. This happened when indexing the entire 2.9GB mozilla CVSROOTLuckily, however, OpenGrok is very modularized and atomic in nature. If you take a look at go to the example OpenGrok I link topage, you will see that its can get a really nice application!* Switched to a local VM from my P4 --[[User:John64|John64]] 21:44more complete explanation, 4 October 2006 (EDT)* Set Javabut the basic jist of it is that there are many "Guru's location in /etc/default/tomcat5 --[[User:John64|John64]] 20:15", 3 October 2006 (EDT)* Figured out port for Tomcat5, 8180 privateeach with a task. Setting up port forwarding as we speak using public port 81 Oct 3, 2006* Using personal Machine instead The files are first read by the History Guru who looks at the file and decides what type of versioning the VM due file uses. Once the versions have been analyzed, they are passed on to the file analyzer guru who then decides what type of file it being double NAT-edis, and thus inaccessible from outside passes it on to a file type analyzer. The allows for portions of the code to be changed without changing the host machine Oct 2whole system, 2006* Set up server using Ubuntu 6.06LTS so if we wanted to be able to do special things with Linux 2XUL/XPCOM as far as how to handle its symbols, we would write one module which is not dependent at all on any other file analyzer.6 The same way, if Mozilla switchs to SVN, we would just port the branching support to SVN.17ck1 Oct 2 On the chances that Mozilla switches to something other than CVS or SVN, 2006* RSyncing machine Oct 2, 2006a HistoryGuru could be written for that type of versioning history. The OpenGrok project is under the [http://www.sun.com/cddl/ CDDL] which [http://www.sun.com/cddl/CDDL_MPL_redline.pdf derives from the MPL 1.1]
====Candidates====* [http://lxr.mozilla.org/ LXR]/[http://landfill.mozilla.org/mxr-test/ MXR]/[http://www.mozilla.org/bonsai.html Bonsai] - Not working on setting one up because there is already one* [http://gonzui.sourceforge.net Gonzui] - Impressive looking thing. This will be my first target to setup due to its simplicity and apparently very powerful nature* [[OpenGrok]] - As this is becoming my main focusIn closing, I am breaking really like the OpenGrok project because it out into its own page.* [http://www.cenqua.com/fisheye/index.html Fisheye] - Commercial Solution that is free (as in beer) for open source projects.* Hosted by Tigris - I forgot the name'''very''' fast, but its hosted by "Tigris" '''very''' powerful and it looked pretty good. If you know the name, please edit as appropriate* [http://savannah.nongnu.org/projects/horus Horus] - Not really what is needed, but its a nice interface for programming students own code. I will not be actively doing anything to it.* [http://bazaar-vcs.org Bazaar] - I dont really know what this is, it might be what is needed, but it might be something irrelevant.* [http://sourceforge.net/projects/sourcenav/ Sourcenav] - investigating this project* [http://google.com/codesearch Google Code Search] - No way to specify the Mozilla project only'''VERY''' modular!
====Links====
* [https://sparc.senecacollege.ca/portal.php?project&pid=23 Official Blurb] just in case I forget what I am doing :P
* [http://matrix.senecac.on.ca/~jhford/ John's School Page]
* [http://cvs2svn.tigris.org CVS2SVN] Tool to convert CVS to SVN. Will be used to test SVN interop.
* [http://endeveloper.wikipediamozilla.org/wikien/Apache_Tomcat Tomcat on Wikipediadocs/Mozilla_Source_Code_Via_CVS Devmo:CVS Checkout]* [http://developer.mozilla.org/en/docs/Mozilla_Source_Code_Via_CVS Rsyncing_the_CVS_Repository Devmo: Rsyncing the CVS Checkout]* [http://developer.mozilla.org/en/docs/Rsyncing_the_CVS_Repository Rsyncing the CVS_Tags Devmo: CVSTags]- to get the branches to checkout
* [http://www.ubuntuforums.org/showthread.php?t=219985 Tomcat5 on Ubuntu]
* [http://www.onjava.com/pub/a/onjava/2003/06/25/tomcat_tips.html Tomcat Tips]
* [http://atucker.typepad.com/blog/2005/11/a_new_source_br.html Blog Entry] on OpenGrok
* [http://ubuntuforums.org/showthread.php?t=124431 Java5 on Ubuntu] - "sudo update-alternatives --config java" and "apt-get remove --purge java-gcj-compat"
* [http://tomcat.apache.org/tomcat-4.1-doc/security-manager-howto.html Tomcat4 Doc] - possibilities for setting up catalina.policy correctly
* [http://developer.mozilla.org/en/docs/CVS_Tags CVS Tags] - to get the branches to checkout
* [http://tomcat.apache.org/tomcat-5.5-doc/ssi-howto.html Tomcat SSI Howto]
* [http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/cvsup.html#CVSUP-CONFIG-VERS CVSUP Guide for FreeBSD]
* [http://www.mozilla.org/docs/jargon.html Mozilla Jargon]
* [http://www.deitel.com/CodeSearchEngines/CodeSearchEngines_ResourceCenter_MerobaseOpenGrokCodeProject.html link 45Misc==Notes on Accessing Test Server==If you want to access the test server through anything other than port 80, you are going to have to type in the following address into a browser and note the IP address you get in your address bar. This is because I have dynamic DNS. Everything, including the source itself, will be in the http root for easy access to the files. This is not optimal, and will not stay this way once things advance. * [http://superfindwww.sun.bouncemecom/cddl/CDDL_why_details.net Superfindhtml CDDL] - Will resolve as www.no-ip.com computer, which is why you have to use a browser to get explanation of the diferences between the IP ==Questions==Please edit in an answer if you knowMPL and CDDL
==Pulling CVS==
This code will pull the CVS for the branches specified in @branches, or it did at some point, your mileage may vary
<pre>
#!/usr/bin/perl
# Pull CVS from the mozilla project server
# Where you want the branch folders
my $src_root = "/var/mozilla";
# Where is your make executable run.sh for opengrok? (Mozilla needs 3.80, nothing elseor equivalent script to start the indexer)my $make opengroker = "/homevar/jhfordopengrok/oldmake/makerun.sh";
# Where is your run.sh for opengrok??server?my $opengroker cvsserver = "/var/opengrok':pserver:anonymous@hera.senecac.on.ca:/run.sh"cvsroot';
# Branches to be pulled
my @branches = (
"HEAD", "REFLOW_20061031_BRANCHMOZILLA_1_8_BRANCH",
"MOZILLA_1_8_0_BRANCH",
"AVIARY_1_0_1_20050124_BRANCH", "REFLOW_20061031_BRANCH" ); # Descriptions for each branch, don't delete old ones for the sake of deleting themmy %descriptions = ( "HEAD" => "Trunk - development branch", "MOZILLA_1_8_BRANCH" => "Firefox 2.0 - development branch", "MOZILLA_1_8_0_BRANCH" => "Firefox 1.5 - maintainance branch", "MOZILLA_1_7_BRANCH" => "Firefox 1.0 - maintainance branch", "AVIARY_1_0_1_20050124_BRANCH" => "Suite - maintainance branch", "REFLOW_20061031_BRANCH" => "Reflow Refactoring"
);
# Open the file or
open BRANCHLIST, ">$branchlistpath" or die "Could not open file";
 
# Clear out what ever source was there
system ("rm -rf ${src_root}/*");
foreach (@branches){
# Download the makefile, then checkout from the makefile
system("
mkdir ${src_root}/$_;
cd ${src_root}/$_;
cvs -d :pserver:anonymous:anonymous\x40cvs-mirror.mozilla.org:/cvsroot ${cvsserver} co -r $_ mozilla/client.mk;
make -f ${src_root}/${_}/mozilla/client.mk checkout MOZ_CO_PROJECT=all;
");
system ("bash $opengroker");
 
</pre>
1
edit