Add support for more compilers to distcc

From CDOT Wiki
Jump to: navigation, search

Project Name

Add support for more compilers to distcc

Project Description

An earlier project added multi-compiler support to distcc, and then MSVC support. The framework is now in place to support even more compilers. Add support for another compiler; either Java or C#.

Necessary Skills: C, Python, Makefile

Project Leader(s)

Peter Callaghan (pcal)

Project Contributor(s)

Some options for contributions:

  • publish/find some more c# files to test with
  • NOTE: Please ensure the files compile with Mono c# before sending them.
  • test this on different OSs & networks
  • Make a comprehensive list of the set-up steps on one page, unlike the scattered links there are now. No coding required, but time consuming.

Project Details/News

0.3 release (Under construction)

Code

All the code for the 0.3 release can be found here. This contains three files:

  • pcallaghanzeropointthreeWHOLE.txt - the entire patch
  • pcallaghanzeropointthreeCODE.txt - the changes to the src directory
  • pcallaghanzeropointthreeTEST.txt - the changes to test/distcctest.py

These should be applied from the trunk directory. NOTE: use either pcallaghanzeropointthreeWHOLE.txt OR both the others, depending on your preference. The end result is the same. For more details see the instructions section.

Instructions

In order to use this program, several steps must be completed.

  1. download the Mono C# compiler
    • put mono in your PATH
  2. set up an environment variable DISTCC_MCS_HOSTS=localhost
  3. Checkout the distcc source
  4. If you are using windows get cygwin
    • configure cygwin by running the setup file that gets downloaded. I used the mirror: http://mirror.rhsmith.umd.edu
    • NOTE: This is the step most errors will stem from, so pay careful attention to detail.
      • In addition to the packages downloaded by default, get the following packages:
        • binutils 20060817-1 (Devel)
        • diffutils 2.8.7-1 (Utils)
        • gcc 3.4.4-3 (Devel) NOTE: MUST be version 3.4.4-3
        • gcc-core 3.4.4-3 (Devel)
        • gcc-g++ 3.4.4-3 (Devel)
        • gcc-mingw 20040810-1 (Devel)
        • gcc-mingw-core 20050522-1 (Devel)
        • gcc-mingw-g++ 20050522-1 (Devel)
        • mingw_runtime 3.13-1 (Devel, Libs)
        • python 2.5.1-2 (Interpreters, Python)
        • subversion 1.4.5-2 (Devel)
        • openssl 0.9.8g-1 (Libs, Net)
        • patch 2.5.8-9 (Utils)
        • patchutils 0.2.31-1 (Devel, Libs, Text)
        • tcltk 20060202-1 (Libs)
        • zlib 1.2.3-2 (Libs)
        • w32api 3.10-1 (Libs)
        • libapr1 1.2.11-1 (Libs)
        • libaprutil1 1.2.10-1 (Libs)
        • libneon26 0.26.4-1 (Libs)
        • libcharset1 1.11-1 (Libs)
      • modify the cygwin.bat file
      • Copy and paste the following file, then change the paths so they accurately represent your system. It's VERY important
      • The particular lines to change (if yours are different) are: 4,9,12,16,19,22
      • If you copy and paste you should not have to make any changes past line 22.
        • @echo off

C:
chdir C:\cygwin\bin

rem --- Set HOME so that cvs and ssh work correctly
rem --- cvs uses HOME to locate your .cvspass file, and ssh to locate your .ssh file
rem --- if you are using ssh, your HOME should match the home directory specified in /etc/passwd. See http://www.cygwin.com/faq/faq0.html.
set HOME=C:\home

rem --- Set VCVARS to wherever the MSVC vcvars.bat file is found
set VCVARS=C:\ff\vcvars32.bat

rem --- Set MSSDK to wherever the MS SDK is installed
rem --- Only required for MSVC7 or the Free MSVC editions that don't come with an SDK
set MSSDK=C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2

rem --- Set MOZ_TOOLS to wherever you have the moztools packaged installed
set MOZ_TOOLS=C:\moztools
rem --- Set MONO to wherever you have mono installed.
rem --- Make sure there are no spaces in the path, otherwise the unit tests won't work
set MONO=C:\Mono-1.2.5.1

rem --- Set CYGWINBASE to wherever cygwin is installed
rem --- Do not use CYGWIN or else cygserver, cygrunsrv, and Cygwin services will not function properly
rem --- Variable CYGWIN is also used to modify Cygwin's behaviour a little bit.
set CYGWINBASE=C:\cygwin
rem --- Make sure Cygwin does not print out a DOS style path warning
set CYGWIN=nodosfilewarning

rem --- Prepend Cygwin path
rem --- This is necessary so that cygwin find is ahead of windows find.exe in the PATH, but cgywin link is after MSVC link.exe.
set PATH=%CYGWINBASE%\bin;%PATH%
set PATH=%MONO%\bin;%PATH%
rem --- Set MSVC environment vars
call "%VCVARS%"
 
rem --- Prepend SDK paths
rem --- Only required for MSVC7 or the Free MSVC editions that don't come with an SDK
set PATH=%MSSDK%\bin;%PATH%;
set INCLUDE=%MSSDK%\include;%MSSDK%\include\atl;%INCLUDE%
set LIB=%MSSDK%\lib;%LIB%

rem --- moztools comes last
set PATH=%PATH%;%MOZ_TOOLS%\bin;C:\ff\distcc\src;


rem --- Now the PATH variable contains:
rem MS-SDK; MSVC; Cygwin; Windows; glib/libIDL; Moztools

rem --- Typically the last thing the script does is launch a cygwin shell
rem --- watch for your ~/.profile and /etc/profile which may overwrite your carefully setup PATH!
bash --login -i
  5. get the 0.3 patch
  6. apply the patch
    • from the trunk directory
    • patch -p0 < pcallaghanzeropointthreeWHOLE.txt
    • OR
      • patch -p0 < pcallaghanzeropointthreeCODE.txt
      • patch -p0 < pcallaghanzeropointthreeTEST.txt
  7. configure and build distcc
    • from the trunk directory
    • ./configure && make && make install
  8. get a simple C# file
  9. compile it
    • distcc mcs hello.cs
  10. run it
    • ./hello
  11. As an alternative to steps 9 through 11, find or write some other c# files and test them.
  12. Tell me how it went.

Plans for 0.3 release

While all the required functions exist, unit tests would streamline the testing process and ensure each function does exaclty what it's supposed to. Therefore, the major component of the 0.3 release is going to be a series of tests for the functions that can be run after any changes. In addition, (Time permitting) I'll clean up the instructions and make them clearer.

Having found that these instructions are inadequate, I'm setting out to fix them. First up is configuring cygwin. The Instructions given on the linked page aren't enough to compile distcc with. There's a list of some 165 packages I had. Most of them were obviously extraneous and quite a few were downloaded by default. So I'm down to about 40 to test. The required ones will be listed with the 0.3 release instructions.

0.25 Release (Obsolete)

A new patch has been created. All required functions are now complete for the Mono C# compiler. Now is a great time for more contributors to test this project and try to find (or fix) any bugs that pop up. I'm still having trouble with SVN, but I expect to get that sorted out soon.

to use this patch:

  1. get cygwin
  2. configure cygwin
  3. download the Mono C# compiler
    • put mono in your PATH
  4. set up an environment variable DISTCC_MCS_HOSTS=localhost
  5. Checkout the distcc source
  6. get the 0.25 patch
  7. apply the patch
    • from the trunk directory
    • patch -p0 < pcallaghanzeropointtwofive.txt
  8. configure and build distcc
    • from the trunk directory
    • ./configure && make && make install
  9. get a simple C# file
  10. compile it
    • distcc mcs hello.cs
  11. run it
    • ./hello
  12. As an alternative to steps 9 through 11, find or write some other c# files and test them.
  13. Tell me how it went.


0.2 Release (Obsolete)

A new patch has been created. All except one of the required functions (dcc_mcs_scan_args) are now complete. As the work required for this update turned out to be a lot less than I anticipated, a 0.25 patch should be released soon that will implement that last function. An SVN branch is not available yet, but should also be ready in time for 0.25.

to use this patch:

  • Either reverse the first patch or get a non-patched version of the trunk
  • Apply the new patch


Goal for 0.2 release

Continue work on the previously identified functions. As of 0.1 several were still little more than shells, by the completion of 0.2 all except dcc_mcs_scan_args should be fully completed. A supplementary goal is to set up the source as a branch of the existing trunk on cdot, so testers may just check out the new code rather than having to check out the trunk and patch it.

0.1 Release (Obsolete)

A compiler has been created. Note that it is by no means anywhere near completion. That's why this is the 0.1 release.

This patch has been tested on Windows XP, using cygwin to tell distcc to compile a single, simple c# file. To use it:

  1. get cygwin
  2. configure cygwin
  3. download the Mono C# compiler
    • put mono in your PATH
  4. set up an environment variable DISTCC_MCS_HOSTS=localhost
  5. Checkout the distcc source
  6. get the 0.1 patch
  7. apply the patch
    • from the trunk directory
    • patch -p0 < pcallaghanzeropointone.txt
  8. configure and build distcc
    • from the trunk directory
    • ./configure && make && make install
  9. get a simple C# file
  10. compile it
    • distcc mcs hello.cs
  11. run it
    • ./hello

Goal for 0.1 Release

- Add support for Mono's cSharp compiler[1]. By the 0.1 release, Distcc should be able to at least compile C# locally. While the earlier work done on Distcc With MSVC suggests that only a hand-full of functions need to be coded in order for it to add a new compiler, both Java and C# compile in a manner significantly different than C. This will likely cause some trouble shortly...regardless, the following functions are the minimum which need to be created:

  • arg.c
    • dcc_<compilerName>_set_action_opt
    • dcc_<compilerName>_set_output
    • dcc_<compilerName>_scan_args
  • strip.c
    • dcc_<compilerName>_strip_local_args
    • dcc_<compilerName>_strip_dasho
  • filename.c
    • dcc_<compilerName>_is_source
    • dcc_<compilerName>_is_preprocessed
    • dcc_<compilerName>_is_object
    • dcc_<compilerName>_preproc_exten
  • hosts.c
    • dcc_<compilerName>_get_hostlist

Known Bugs

See the list within Distcc With MSVC

Retrieved from "https://wiki.cdot.senecapolytechnic.ca/w/index.php?title=Add_support_for_more_compilers_to_distcc&oldid=14026"