Difference between revisions of "Add support for more compilers to distcc"
Pcallaghan (talk | contribs) (added complete list of present cygwin packages) |
Pcallaghan (talk | contribs) m (fixed instructions) |
||
(5 intermediate revisions by the same user not shown) | |||
Line 19: | Line 19: | ||
<li>[[User:Vlam6|Vincent Lam]]</li> | <li>[[User:Vlam6|Vincent Lam]]</li> | ||
<li>[[User:Vbala|Vijey Balasundaram]]</li> | <li>[[User:Vbala|Vijey Balasundaram]]</li> | ||
+ | <li>[[User:Simon|Simon Jung]]</li> | ||
</ul> | </ul> | ||
Line 30: | Line 31: | ||
== Project Details/News == | == Project Details/News == | ||
+ | === 0.3 release (Under construction)=== | ||
+ | ==== Code ==== | ||
+ | All the code for the 0.3 release can be found [http://zenit.senecac.on.ca/wiki/imgs/Pcallaghanzeropointthree.zip here]. This contains three files: | ||
+ | <ul> | ||
+ | <li>pcallaghanzeropointthreeWHOLE.txt - the entire patch</li> | ||
+ | <li>pcallaghanzeropointthreeCODE.txt - the changes to the src directory</li> | ||
+ | <li>pcallaghanzeropointthreeTEST.txt - the changes to test/distcctest.py</li> | ||
+ | </ul> | ||
+ | These should be applied from the trunk directory. NOTE: use either pcallaghanzeropointthreeWHOLE.txt OR both the others, depending on your preference. The end result is the same. For more details see the instructions section. | ||
− | + | ==== Instructions ==== | |
+ | In order to use this program, several steps must be completed. | ||
− | < | + | <ol> |
− | <li> | + | <li>download the Mono C# [http://www.mono-project.com/Main_Page compiler]</li> |
− | <li> | + | <ul> |
− | <li> | + | <li>put mono in your PATH</li> |
− | <li> | + | </ul> |
− | <li> | + | <li>set up an environment variable DISTCC_MCS_HOSTS=localhost</li> |
− | <li> | + | <li>Checkout the distcc source</li> |
− | + | <ul> | |
− | <li> | + | <li>svn co svn://cdot.senecac.on.ca/distcc/trunk</li> |
− | <li> | + | <li>You should get revision 117</li> |
− | <li> | + | </ul> |
− | <li> | + | <li>If you are using windows [http://www.cygwin.com/ get cygwin]</li> |
− | < | + | <ul> |
+ | <li>configure cygwin by running the setup file that gets downloaded. I used the mirror: http://mirror.rhsmith.umd.edu</li> | ||
+ | <li>NOTE: This is the step most errors will stem from, so pay careful attention to detail.</li> | ||
+ | <ul> | ||
+ | <li>In addition to the packages downloaded by default, get the following packages:</li> | ||
+ | <ul> | ||
<li>binutils 20060817-1 (Devel)</li> | <li>binutils 20060817-1 (Devel)</li> | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
<li>diffutils 2.8.7-1 (Utils)</li> | <li>diffutils 2.8.7-1 (Utils)</li> | ||
− | + | <li>gcc 3.4.4-3 (Devel) NOTE: MUST be version 3.4.4-3 </li> | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | <li>gcc 3.4.4-3 (Devel) | ||
− | |||
<li>gcc-core 3.4.4-3 (Devel)</li> | <li>gcc-core 3.4.4-3 (Devel)</li> | ||
<li>gcc-g++ 3.4.4-3 (Devel)</li> | <li>gcc-g++ 3.4.4-3 (Devel)</li> | ||
− | |||
− | |||
− | |||
− | |||
<li>gcc-mingw 20040810-1 (Devel)</li> | <li>gcc-mingw 20040810-1 (Devel)</li> | ||
− | |||
<li>gcc-mingw-core 20050522-1 (Devel)</li> | <li>gcc-mingw-core 20050522-1 (Devel)</li> | ||
<li>gcc-mingw-g++ 20050522-1 (Devel)</li> | <li>gcc-mingw-g++ 20050522-1 (Devel)</li> | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
<li>mingw_runtime 3.13-1 (Devel, Libs)</li> | <li>mingw_runtime 3.13-1 (Devel, Libs)</li> | ||
− | <li> | + | <li>python 2.5.1-2 (Interpreters, Python)</li> |
− | + | <li>subversion 1.4.5-2 (Devel)</li> | |
− | |||
− | |||
− | <li> | ||
<li>openssl 0.9.8g-1 (Libs, Net)</li> | <li>openssl 0.9.8g-1 (Libs, Net)</li> | ||
− | |||
− | |||
− | |||
<li>patch 2.5.8-9 (Utils)</li> | <li>patch 2.5.8-9 (Utils)</li> | ||
<li>patchutils 0.2.31-1 (Devel, Libs, Text)</li> | <li>patchutils 0.2.31-1 (Devel, Libs, Text)</li> | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
<li>tcltk 20060202-1 (Libs)</li> | <li>tcltk 20060202-1 (Libs)</li> | ||
− | <li> | + | <li>zlib 1.2.3-2 (Libs)</li> |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
<li>w32api 3.10-1 (Libs)</li> | <li>w32api 3.10-1 (Libs)</li> | ||
− | <li> | + | <li>libapr1 1.2.11-1 (Libs)</li> |
− | <li> | + | <li>libaprutil1 1.2.10-1 (Libs)</li> |
− | <li> | + | <li>libneon26 0.26.4-1 (Libs)</li> |
− | <li> | + | <li>libcharset1 1.11-1 (Libs)</li> |
− | <li> | + | </ul> |
− | <li> | + | <li>modify the cygwin.bat file</li> |
− | <li> | + | <li>Copy and paste the following file, then change the paths so they accurately represent your system. It's VERY important</li> |
− | <li> | + | <li>The particular lines to change (if yours are different) are: 4,9,12,16,19,22</li> |
− | <li> | + | <li>If you copy and paste you should not have to make any changes past line 22.</li> |
− | <li> | + | <pre>@echo off |
− | <li> | + | |
− | <li> | + | C: |
− | <li> | + | chdir C:\cygwin\bin |
− | <li> | + | |
− | <li> | + | rem --- Set HOME so that cvs and ssh work correctly |
− | </ | + | rem --- cvs uses HOME to locate your .cvspass file, and ssh to locate your .ssh file |
+ | rem --- if you are using ssh, your HOME should match the home directory specified in /etc/passwd. See http://www.cygwin.com/faq/faq0.html. | ||
+ | set HOME=C:\home | ||
+ | |||
+ | rem --- Set VCVARS to wherever the MSVC vcvars.bat file is found | ||
+ | set VCVARS=C:\ff\vcvars32.bat | ||
+ | |||
+ | rem --- Set MSSDK to wherever the MS SDK is installed | ||
+ | rem --- Only required for MSVC7 or the Free MSVC editions that don't come with an SDK | ||
+ | set MSSDK=C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2 | ||
+ | |||
+ | rem --- Set MOZ_TOOLS to wherever you have the moztools packaged installed | ||
+ | set MOZ_TOOLS=C:\moztools | ||
+ | rem --- Set MONO to wherever you have mono installed. | ||
+ | rem --- Make sure there are no spaces in the path, otherwise the unit tests won't work | ||
+ | set MONO=C:\Mono-1.2.5.1 | ||
+ | |||
+ | rem --- Set CYGWINBASE to wherever cygwin is installed | ||
+ | rem --- Do not use CYGWIN or else cygserver, cygrunsrv, and Cygwin services will not function properly | ||
+ | rem --- Variable CYGWIN is also used to modify Cygwin's behaviour a little bit. | ||
+ | set CYGWINBASE=C:\cygwin | ||
+ | rem --- Make sure Cygwin does not print out a DOS style path warning | ||
+ | set CYGWIN=nodosfilewarning | ||
+ | |||
+ | rem --- Prepend Cygwin path | ||
+ | rem --- This is necessary so that cygwin find is ahead of windows find.exe in the PATH, but cgywin link is after MSVC link.exe. | ||
+ | set PATH=%CYGWINBASE%\bin;%PATH% | ||
+ | set PATH=%MONO%\bin;%PATH% | ||
+ | rem --- Set MSVC environment vars | ||
+ | call "%VCVARS%" | ||
+ | |||
+ | rem --- Prepend SDK paths | ||
+ | rem --- Only required for MSVC7 or the Free MSVC editions that don't come with an SDK | ||
+ | set PATH=%MSSDK%\bin;%PATH%; | ||
+ | set INCLUDE=%MSSDK%\include;%MSSDK%\include\atl;%INCLUDE% | ||
+ | set LIB=%MSSDK%\lib;%LIB% | ||
+ | |||
+ | rem --- moztools comes last | ||
+ | set PATH=%PATH%;%MOZ_TOOLS%\bin;C:\ff\distcc\src; | ||
+ | |||
+ | |||
+ | rem --- Now the PATH variable contains: | ||
+ | rem MS-SDK; MSVC; Cygwin; Windows; glib/libIDL; Moztools | ||
+ | |||
+ | rem --- Typically the last thing the script does is launch a cygwin shell | ||
+ | rem --- watch for your ~/.profile and /etc/profile which may overwrite your carefully setup PATH! | ||
+ | bash --login -i | ||
+ | </pre> | ||
+ | </ul> | ||
+ | </ul> | ||
+ | <li>get the 0.3 [http://zenit.senecac.on.ca/wiki/imgs/Pcallaghanzeropointthree.zip patch]</li> | ||
+ | <li>apply the patch</li> | ||
+ | <ul> | ||
+ | <li>from the trunk directory</li> | ||
+ | <li>patch -p0 < pcallaghanzeropointthreeWHOLE.txt</li> | ||
+ | <li>OR</li> | ||
+ | <ul> | ||
+ | <li>patch -p0 < pcallaghanzeropointthreeCODE.txt</li> | ||
+ | <li>patch -p0 < pcallaghanzeropointthreeTEST.txt</li> | ||
+ | </ul> | ||
+ | </ul> | ||
+ | <li>configure and build distcc</li> | ||
+ | <ul> | ||
+ | <li>from the trunk directory</li> | ||
+ | <li>./configure && make && make install</li> | ||
+ | </ul> | ||
+ | <li>get a simple C# [http://docs.google.com/Doc?id=dfkp8fkz_1hbrfkk file]</li> | ||
+ | <li>compile it</li> | ||
+ | <ul> | ||
+ | <li>distcc mcs hello.cs</li> | ||
+ | </ul> | ||
+ | <li>run it</li> | ||
+ | <ul> | ||
+ | <li>./hello</li> | ||
+ | </ul> | ||
+ | <li>As an alternative to steps 9 through 11, find or write some other c# files and test them.</li> | ||
+ | <li>Tell me how it went.</li> | ||
+ | </ol> | ||
+ | |||
+ | === Plans for 0.3 release === | ||
+ | While all the required functions exist, unit tests would streamline the testing process and ensure each function does exaclty what it's supposed to. Therefore, the major component of the 0.3 release is going to be a series of tests for the functions that can be run after any changes. In addition, (Time permitting) I'll clean up the instructions and make them clearer. | ||
+ | |||
+ | Having found that these instructions are inadequate, I'm setting out to fix them. First up is configuring cygwin. The Instructions given on the linked page aren't enough to compile distcc with. There's a list of some 165 packages I had. Most of them were obviously extraneous and quite a few were downloaded by default. So I'm down to about 40 to test. The required ones will be listed with the 0.3 release instructions. | ||
− | === 0.25 Release === | + | === 0.25 Release (Obsolete) === |
A new [http://docs.google.com/Doc?id=dfkp8fkz_3hqpw4f patch] has been created. All required functions are now complete for the Mono C# compiler. Now is a great time for more contributors to test this project and try to find (or fix) any bugs that pop up. I'm still having trouble with SVN, but I expect to get that sorted out soon. | A new [http://docs.google.com/Doc?id=dfkp8fkz_3hqpw4f patch] has been created. All required functions are now complete for the Mono C# compiler. Now is a great time for more contributors to test this project and try to find (or fix) any bugs that pop up. I'm still having trouble with SVN, but I expect to get that sorted out soon. | ||
Latest revision as of 04:26, 14 December 2007
Contents
Project Name
Add support for more compilers to distcc
Project Description
An earlier project added multi-compiler support to distcc, and then MSVC support. The framework is now in place to support even more compilers. Add support for another compiler; either Java or C#.
Necessary Skills: C, Python, Makefile
Project Leader(s)
Project Contributor(s)
Some options for contributions:
- publish/find some more c# files to test with
- NOTE: Please ensure the files compile with Mono c# before sending them.
- test this on different OSs & networks
- Make a comprehensive list of the set-up steps on one page, unlike the scattered links there are now. No coding required, but time consuming.
Project Details/News
0.3 release (Under construction)
Code
All the code for the 0.3 release can be found here. This contains three files:
- pcallaghanzeropointthreeWHOLE.txt - the entire patch
- pcallaghanzeropointthreeCODE.txt - the changes to the src directory
- pcallaghanzeropointthreeTEST.txt - the changes to test/distcctest.py
These should be applied from the trunk directory. NOTE: use either pcallaghanzeropointthreeWHOLE.txt OR both the others, depending on your preference. The end result is the same. For more details see the instructions section.
Instructions
In order to use this program, several steps must be completed.
- download the Mono C# compiler
- put mono in your PATH
- set up an environment variable DISTCC_MCS_HOSTS=localhost
- Checkout the distcc source
- svn co svn://cdot.senecac.on.ca/distcc/trunk
- You should get revision 117
- If you are using windows get cygwin
- configure cygwin by running the setup file that gets downloaded. I used the mirror: http://mirror.rhsmith.umd.edu
- NOTE: This is the step most errors will stem from, so pay careful attention to detail.
- In addition to the packages downloaded by default, get the following packages:
- binutils 20060817-1 (Devel)
- diffutils 2.8.7-1 (Utils)
- gcc 3.4.4-3 (Devel) NOTE: MUST be version 3.4.4-3
- gcc-core 3.4.4-3 (Devel)
- gcc-g++ 3.4.4-3 (Devel)
- gcc-mingw 20040810-1 (Devel)
- gcc-mingw-core 20050522-1 (Devel)
- gcc-mingw-g++ 20050522-1 (Devel)
- mingw_runtime 3.13-1 (Devel, Libs)
- python 2.5.1-2 (Interpreters, Python)
- subversion 1.4.5-2 (Devel)
- openssl 0.9.8g-1 (Libs, Net)
- patch 2.5.8-9 (Utils)
- patchutils 0.2.31-1 (Devel, Libs, Text)
- tcltk 20060202-1 (Libs)
- zlib 1.2.3-2 (Libs)
- w32api 3.10-1 (Libs)
- libapr1 1.2.11-1 (Libs)
- libaprutil1 1.2.10-1 (Libs)
- libneon26 0.26.4-1 (Libs)
- libcharset1 1.11-1 (Libs)
- modify the cygwin.bat file
- Copy and paste the following file, then change the paths so they accurately represent your system. It's VERY important
- The particular lines to change (if yours are different) are: 4,9,12,16,19,22
- If you copy and paste you should not have to make any changes past line 22.
- get the 0.3 patch
- apply the patch
- from the trunk directory
- patch -p0 < pcallaghanzeropointthreeWHOLE.txt
- OR
- patch -p0 < pcallaghanzeropointthreeCODE.txt
- patch -p0 < pcallaghanzeropointthreeTEST.txt
- configure and build distcc
- from the trunk directory
- ./configure && make && make install
- get a simple C# file
- compile it
- distcc mcs hello.cs
- run it
- ./hello
- As an alternative to steps 9 through 11, find or write some other c# files and test them.
- Tell me how it went.
@echo off C: chdir C:\cygwin\bin rem --- Set HOME so that cvs and ssh work correctly rem --- cvs uses HOME to locate your .cvspass file, and ssh to locate your .ssh file rem --- if you are using ssh, your HOME should match the home directory specified in /etc/passwd. See http://www.cygwin.com/faq/faq0.html. set HOME=C:\home rem --- Set VCVARS to wherever the MSVC vcvars.bat file is found set VCVARS=C:\ff\vcvars32.bat rem --- Set MSSDK to wherever the MS SDK is installed rem --- Only required for MSVC7 or the Free MSVC editions that don't come with an SDK set MSSDK=C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2 rem --- Set MOZ_TOOLS to wherever you have the moztools packaged installed set MOZ_TOOLS=C:\moztools rem --- Set MONO to wherever you have mono installed. rem --- Make sure there are no spaces in the path, otherwise the unit tests won't work set MONO=C:\Mono-1.2.5.1 rem --- Set CYGWINBASE to wherever cygwin is installed rem --- Do not use CYGWIN or else cygserver, cygrunsrv, and Cygwin services will not function properly rem --- Variable CYGWIN is also used to modify Cygwin's behaviour a little bit. set CYGWINBASE=C:\cygwin rem --- Make sure Cygwin does not print out a DOS style path warning set CYGWIN=nodosfilewarning rem --- Prepend Cygwin path rem --- This is necessary so that cygwin find is ahead of windows find.exe in the PATH, but cgywin link is after MSVC link.exe. set PATH=%CYGWINBASE%\bin;%PATH% set PATH=%MONO%\bin;%PATH% rem --- Set MSVC environment vars call "%VCVARS%" rem --- Prepend SDK paths rem --- Only required for MSVC7 or the Free MSVC editions that don't come with an SDK set PATH=%MSSDK%\bin;%PATH%; set INCLUDE=%MSSDK%\include;%MSSDK%\include\atl;%INCLUDE% set LIB=%MSSDK%\lib;%LIB% rem --- moztools comes last set PATH=%PATH%;%MOZ_TOOLS%\bin;C:\ff\distcc\src; rem --- Now the PATH variable contains: rem MS-SDK; MSVC; Cygwin; Windows; glib/libIDL; Moztools rem --- Typically the last thing the script does is launch a cygwin shell rem --- watch for your ~/.profile and /etc/profile which may overwrite your carefully setup PATH! bash --login -i
Plans for 0.3 release
While all the required functions exist, unit tests would streamline the testing process and ensure each function does exaclty what it's supposed to. Therefore, the major component of the 0.3 release is going to be a series of tests for the functions that can be run after any changes. In addition, (Time permitting) I'll clean up the instructions and make them clearer.
Having found that these instructions are inadequate, I'm setting out to fix them. First up is configuring cygwin. The Instructions given on the linked page aren't enough to compile distcc with. There's a list of some 165 packages I had. Most of them were obviously extraneous and quite a few were downloaded by default. So I'm down to about 40 to test. The required ones will be listed with the 0.3 release instructions.
0.25 Release (Obsolete)
A new patch has been created. All required functions are now complete for the Mono C# compiler. Now is a great time for more contributors to test this project and try to find (or fix) any bugs that pop up. I'm still having trouble with SVN, but I expect to get that sorted out soon.
to use this patch:
- get cygwin
- configure cygwin
- download the Mono C# compiler
- put mono in your PATH
- set up an environment variable DISTCC_MCS_HOSTS=localhost
- Checkout the distcc source
- svn co svn://cdot.senecac.on.ca/distcc/trunk
- You should get revision 117
- cd trunk && ./configure && make && make install
- put distcc in your PATH, before cygwin
- get the 0.25 patch
- apply the patch
- from the trunk directory
- patch -p0 < pcallaghanzeropointtwofive.txt
- configure and build distcc
- from the trunk directory
- ./configure && make && make install
- get a simple C# file
- compile it
- distcc mcs hello.cs
- run it
- ./hello
- As an alternative to steps 9 through 11, find or write some other c# files and test them.
- Tell me how it went.
0.2 Release (Obsolete)
A new patch has been created. All except one of the required functions (dcc_mcs_scan_args) are now complete. As the work required for this update turned out to be a lot less than I anticipated, a 0.25 patch should be released soon that will implement that last function. An SVN branch is not available yet, but should also be ready in time for 0.25.
to use this patch:
- Either reverse the first patch or get a non-patched version of the trunk
- Apply the new patch
Goal for 0.2 release
Continue work on the previously identified functions. As of 0.1 several were still little more than shells, by the completion of 0.2 all except dcc_mcs_scan_args should be fully completed. A supplementary goal is to set up the source as a branch of the existing trunk on cdot, so testers may just check out the new code rather than having to check out the trunk and patch it.
0.1 Release (Obsolete)
A compiler has been created. Note that it is by no means anywhere near completion. That's why this is the 0.1 release.
This patch has been tested on Windows XP, using cygwin to tell distcc to compile a single, simple c# file. To use it:
- get cygwin
- configure cygwin
- download the Mono C# compiler
- put mono in your PATH
- set up an environment variable DISTCC_MCS_HOSTS=localhost
- Checkout the distcc source
- svn co svn://cdot.senecac.on.ca/distcc/trunk
- You should get revision 117
- cd trunk && ./configure && make && make install
- put distcc in your PATH, before cygwin
- get the 0.1 patch
- apply the patch
- from the trunk directory
- patch -p0 < pcallaghanzeropointone.txt
- configure and build distcc
- from the trunk directory
- ./configure && make && make install
- get a simple C# file
- compile it
- distcc mcs hello.cs
- run it
- ./hello
Goal for 0.1 Release
- Add support for Mono's cSharp compiler[1]. By the 0.1 release, Distcc should be able to at least compile C# locally. While the earlier work done on Distcc With MSVC suggests that only a hand-full of functions need to be coded in order for it to add a new compiler, both Java and C# compile in a manner significantly different than C. This will likely cause some trouble shortly...regardless, the following functions are the minimum which need to be created:
- arg.c
- dcc_<compilerName>_set_action_opt
- dcc_<compilerName>_set_output
- dcc_<compilerName>_scan_args
- strip.c
- dcc_<compilerName>_strip_local_args
- dcc_<compilerName>_strip_dasho
- filename.c
- dcc_<compilerName>_is_source
- dcc_<compilerName>_is_preprocessed
- dcc_<compilerName>_is_object
- dcc_<compilerName>_preproc_exten
- hosts.c
- dcc_<compilerName>_get_hostlist
Known Bugs
See the list within Distcc With MSVC