Difference between revisions of "Distcc With MSVC"

From CDOT Wiki
Jump to: navigation, search
(→‎Project News)
(Added link for compiler porters)
 
(51 intermediate revisions by 4 users not shown)
Line 5: Line 5:
 
= Project Description =
 
= Project Description =
  
−
The goal of this project is to speed up Windows Mozilla Builds by spreading object compilations over several machines. This can be done by using an open source product called distcc created by Martin Pool. Unfortunately this is only available on *nix machines using gcc. Our project involves porting this software to Windows and use [http://msdn.microsoft.com/visualc/ Microsoft's Visual C++ compiler] using cygwin.
+
The goal of this project is to speed up Windows Mozilla Builds by spreading object compilations over several machines. This can be done by using an open source product called distcc created by Martin Pool. Unfortunately this is only available on *nix machines using gcc. Our project involves porting this software to Windows and use [http://msdn.microsoft.com/visualc/ Microsoft's Visual C++ compiler] using cygwin and msys.
  
 
= Project Leader(s) =
 
= Project Leader(s) =
Line 19: Line 19:
 
* [[User:Bhearsum|Ben Hearsum]] - for the windows box and support! Thank you!
 
* [[User:Bhearsum|Ben Hearsum]] - for the windows box and support! Thank you!
 
* [[User:JasonSpiro|Jason Spiro]] - For helping figure out why cl wouldn't execute in the unit testing framework and assisting in finding the right path to solve this issue.
 
* [[User:JasonSpiro|Jason Spiro]] - For helping figure out why cl wouldn't execute in the unit testing framework and assisting in finding the right path to solve this issue.
 +
* [[User:Pcvitori | Phil]] - helped with changing the page on helping people do research into cl options for us.
 +
* Moe
 +
* [[User:RealMarkP|Mark P. (aka RealMarkP)]]
 +
* David Humphrey
 +
* Jeff
 +
* Melz
 +
 +
= Product Documentation =
 +
The project has been focused on getting distcc to work with Mozilla, mainly because Mozilla uses GNU Makefiles which can run concurrent jobs. We have started a [[distcc_mozilla_tutorial| tutorial]] in order to help testers/developers.  We have also written documentation for others who might wish to [[Adding Compilers to Distcc|add support to distcc for other compilers]] (e.g., Java, C#, etc.).
 +
 +
== Current Status ==
 +
* Some of the Test Cases may be incorrect. Neither Tom nor Cesar have extensive experience in CL options. So it may not distribute when it should, or it may distribute when it shouldn't.
 +
 +
== What's still to do ==
 +
 +
<!-- Taken from ben's project page -->
 +
<table style="width: 100%;" class="standard-table" cellborder=1 border=1 cellpadding=2 cellspacing=0>
 +
 +
<tr>
 +
<th>Task</th>
 +
<th style="width: 10%>Priority</th>
 +
<th style="width: 30%">Status</th>
 +
</tr>
 +
 +
<tr>
 +
<td>Fix documentation, including website, manpage, and tutorials</td>
 +
<td style="background-color: #ff671e; text-align: center; font-weight: bold; font-size: bigger">Medium</td>
 +
<td>Neglected. I think Tom has worked on the man page, Cesar has worked on the tutorial</td>
 +
</tr>
 +
 +
<tr>
 +
<td>
 +
* find a suitable replacement for fork()
 +
** using windows threads. Need to use #ifdef directives to switch between unix fork and threads. But perhaps where is a better way to do this.
 +
** Perhaps talk to Martian Pool to see if he has any ideas? He hasn't been in the mailing list for a while.
 +
</td>
 +
<td style="background-color: red; text-align: center; font-weight: bold; font-size: bigger">High</td>
 +
<td>Have not started</td>
 +
</tr>
 +
 +
<tr>
 +
<td>
 +
* Send .pdb files to allow debug builds. Currently blocking mozilla from ever using distcc.
 +
** looking at the protocol. Hopefully we don't have to changed a lot.
 +
</td>
 +
<td style="background-color: red; text-align: center; font-weight: bold; font-size: bigger">High</td>
 +
<td>Have not started</td>
 +
</tr>
 +
 +
<tr>
 +
<td>
 +
* Port distcc to use mingw
 +
** Properly link in winsock library ([http://www.foobartastic.com/2007/02/27/distcc-alpha-2-and-moving-to-mingw/ fixed] using libtool)
 +
** use windows threads (another high priority item)
 +
** locate and replace unix functionality that existed in cygwin, but doesn't exist in mingw.
 +
</td>
 +
<td style="background-color: red; text-align: center; font-weight: bold; font-size: bigger">High</td>
 +
<td>Have only got libtool to compile a simple sockets program. We have now port it to distcc
 +
* probably use #ifdef MINGW ... #else ... #endif in source files that use socket.h
 +
* another possibility is to have one header file handle this, and the source files include that header file </td>
 +
</tr>
 +
 +
<tr>
 +
<td>Find more applications that use the Makefile system and can use CL</td>
 +
<td style="background-color: #ff671e; text-align: center; font-weight: bold; font-size: bigger">Medium</td>
 +
<td>Difficult to find. Not many open source applications use Makefile and MSVC, many choosing to opt for NMake</td>
 +
</tr>
 +
 +
<tr>
 +
<td>Compile NSPR using distcc</td>
 +
<td style="background-color: #ff671e; text-align: center; font-weight: bold; font-size: bigger">Medium</td>
 +
<td>Dave suggested only building NSPR because its smaller and won't take as long to compile. Cesar is responsible for this</td>
 +
</tr>
 +
 +
<tr>
 +
<td>boy there sure are lot of memory leaks... would be nice to fix that</td>
 +
<td style="background-color: yellow; text-align: center; font-weight: bold; font-size: bigger">Low</td>
 +
<td>We can use tools like valgrind and mtrace to help. Or we can use  Boehm-GC to free the memory automatically. But this is a lot of work for such low priority.</td>
 +
</tr>
 +
 +
<tr>
 +
<td>Make sure that there are no orphan distcc processes left after make maintainer-check</td>
 +
<td style="background-color: yellow; text-align: center; font-weight: bold; font-size: bigger">Low</td>
 +
<td>This is a problem when tests are killed, and can leave to false-negatives when you try to rerun the tests again. But that can be fixed with god ol' kill-y mc. kill, agent 009</td>
 +
</tr>
 +
 +
</table>
 +
 +
<!--
 +
 +
{|class="standard-table" border='1'
 +
|-
 +
| Need to port unit tests to ensure same functionality for cl as for gcc.
 +
| medium
 +
|-
 +
| <strike>Get distcc working with Moz </strike>
 +
{| class="standard-table" border='1'
 +
  |-
 +
  | <strike>check why MSVC Preprocessor seems to be dying with firefox 2.0 release source.
 +
Possible Solution: We're doing unix to windows path conversion on the daemon, move that code to the client and everything might be good.</strike>
 +
  | <strike>high (currently Cesar & Tom)</strike>
 +
  |-
 +
  | <strike>Figure out why distributed builds die but nondistributed builds don't. This issue lies in the Moz Build System not in distcc code.</strike>
 +
  | <strike>High</strike>
 +
  |}
 +
| <strike>HIGH</strike>
 +
|-
 +
| Get a list of programs that work with our distcc
 +
{| class="standard-table" border='1'
 +
| Application
 +
| Reporter
 +
|-
 +
| [http://cdot.senecac.on.ca/software/nled/ NLED]
 +
| Cesar
 +
|-
 +
| [http://www.mozilla.org mozilla]
 +
| Tom + Cesar
 +
|}
 +
|Medium
 +
|-
 +
|boy there sure are lot of memory leaks... would be nice to fix that
 +
| low
 +
|-
 +
|Make sure that there are no orphan distcc processes left after make maintainer-check
 +
| low
 +
|-
 +
|}
 +
-->
 +
 +
== How to get distcc to compile use cl ==
 +
 +
Tom and Cesar created this patch keeping backwards compatibility in mind. You should be able to still use distcc to compile your c programs if you wish. But if you applied the patch, you probably want to do more than that.
 +
 +
Firstly, the environment variables for distributing with cl is different. '''DISTCC_HOSTS''' is still valid for gcc, but cl will be looking for '''DISCC_CL_HOSTS''' instead. In fact, that is the only thing it looks for. Unlike gcc, cl will not look for ~/.distcc directory. This might be implemented later on.
 +
 +
Secondly, masquerade mode works only for gcc. If you want want to use cl, you must explicitly say so (distcc cl hello.c). If you are using gnu make, you must also explictly tell it to use cl as well (eg using CC, CXX, etc).
 +
 +
== Maintainer-check ==
 +
 +
 +
''NOTE :: If you run the checks, please check your processes (ps aux, or the task manager) for leaked distcc/distccd programs. We will fix this ASAP''
 +
 +
You may ignore this section if you installed Microsoft's CL in the default C:\ directory and your tests work fine.
 +
 +
In order to properly run the tests, you <u>may</u> need to make a few changes. The '''PATH''' variable is reinitalized when the test cases are run. Normally, the tests ''will'' fail when they try the cl tests. This is because PATH does not look for the cl executable and the libraries. Tom and Cesar have made certain assemptions when extending the include path, and included these following paths :
 +
 +
/cygdrive/c/Program Files/Microsoft Visual Studio 8/VC/bin<br>
 +
/cygdrive/c/Program Files/Microsoft Visual Studio 8/Common7/IDE
 +
 +
If it is different for your machine, you need to find these in the Makefile.in or Makefile and change them to your settings.
 +
 +
== Extending the program ==
 +
This will be postponed until we get distcc working properly (not fully) with cl.
  
 
= Project Details =
 
= Project Details =
Line 31: Line 184:
  
 
We have kept the two vtable branch because we actually got a lot of the cl specific stuff done (before we realized we missed some functions).
 
We have kept the two vtable branch because we actually got a lot of the cl specific stuff done (before we realized we missed some functions).
 +
 +
== Help Us Out ==
 +
 +
Here's what you can do to help us out...:
 +
 +
* <strike>[[Distcc with MSVC:CL Options]]</strike>
 +
* [http://www.foobartastic.com/multicompiler-support-for-distcc/ help us test!]
 +
* [[Distcc_With_MSVC:build_testing | Windows Apps for Testing Distcc With MSVC]]
  
 
== The First Patch (Oops) ==
 
== The First Patch (Oops) ==
Line 37: Line 198:
  
 
= Project News =
 
= Project News =
 +
See what's up with this project! Older news can be found [[Distcc With MSVC:Older News|here]]
 +
 +
== May 1, 2007 ==
 +
 +
I've been doing a lot of #ifdef conditions, just to get it to compile (although we will be doing windows threads instead of fork, this is just a step). But now I'm stuck on the function fcntl(). It's a POSIX function, and it has POSIX flags, but some of the flags don't seem to exist in Windows. I don't have enough information to know how important the function is, and if we can just skip it.
 +
 +
And I forgot my power cable in a certain building, and laptop is running out of battery!
 +
 +
== April 30, 2007 ==
 +
 +
There are some header files in distcc that have the same name as some of the standard header files in /usr/include. One file is rpc.h, which is being #included by "sockets.h". Since we don't need sockets.h, this shouldn't be a problem in ''this case''. But that is still something we have to watch out for.
 +
 +
Bah! windows.h #includes rpc.h, dammit.
 +
 +
Resource : [http://www.microsoft.com/technet/interopMigration/taskstools/migrate/unix/ucamg/ch05uav3.mspx#EDHAC Microsofts Unix migration guide]
 +
 +
== April 28, 2007 ==
 +
 +
PDB files :
 +
vc80.pdb - source file debug symbols
 +
'''project'''.pdb - project debug symbols
 +
 +
vc80.pdb can be overwritten by/Fd'''debug'''
 +
 +
Link between '''source'''.c[pp] and '''debug''' is hard coded in the object file. You can find it by running the ''strings'' command on the object file (Thanks shaver!).
 +
 +
Leaving debug symbols aside. We should probably start porting distcc to mingw and replacing incompatible functions. Since that would be more time consuming and more difficult to get right.
 +
 +
== April 23, 2007 ==
 +
 +
Semester is over. Because I honestly have no life, and there is work to do here, I started mapping out the distcc protocol, so we can start making any changes, if necessary. It was a wasted afternoon because the protocol is already documented in the trunk, under docs directory (who would have guessed?)
 +
 +
Preliminary examination indicates that this won't be so much work. The functions seem to already be there.
 +
 +
== March 19, 2007 ==
  
−
For the most up to date info look up [http://www.foobartastic.com foobartastic]
+
There was a subtle bug in Mozilla's code base that [https://bugzilla.mozilla.org/show_bug.cgi?id=374563 bug 274563] deals with that made it impossible for distcc to build Moz. If you apply the patch attached to the [https://bugzilla.mozilla.org/show_bug.cgi?id=374563 bug] then you'll be able to do distcc builds of Mozilla. We will soon be releasing our first beta to reflect our confidence in the code.
  
−
==Nov 6. 2006==
+
==Feb 27, 2007==
−
There is a scheduled 'hack day' being set up. Were putting it at every tuesday at around 5:20 in the TEL open lab.
 
  
−
==Nov 2. 2006==
+
We are able to do distributed builds of Mozilla but it dies when trying to link stuff together (issues surrounding ATL?). So in order to take advantage of Ben Smedberg's wonderful [http://benjamin.smedbergs.us/blog/2007-01-12/mozillabuild-rc1/ Mozilla Build] and be able to get help figuring out weird linkage bugs and dependancies without having to deal with cygwin issues (which are galore), we've started a branch with a mingw port: svn://cdot.senecac.on.ca/distcc/branches/mingwcompatible/
−
After a haitus from touching distcc, we battled the serpent of the C (haha) once again and retreated with our lives, barely, but lost our ego and motivation (at least Cesar did). We cleared up a path setting bug that caused CL to die and return a bad return value. Then we (eventually) passed ParseHost case, which parses an environment variable for ip addresses and domain names. A inproperly done svn merge wasted a good few hours on that.
 
  
−
We also created safeguards, so that any unimplemented vtable function that is called will throw an error message instead of having a segmentation fault.  
+
==Jan 4, 2007==
 +
The post made to the build newgroup seems to have gathered dust. If mozilla is limited to only 4 jobs, than distcc is somewhat limited. But that's not our problem.
  
−
==Oct 20. 2006==
+
Tom has found one problem with using cygwin. Using absolute pathnames. For example :
−
We have a usable windows box for doing our msvc testing (finally!). Thanks to [[User:Bhearsum]] for the boxes.  
+
<code><pre>
 +
$ cl hello.c
 +
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86
 +
Copyright (C) Microsoft Corporation. All rights reserved.
  
−
==Oct 12. 2006==
+
hello.c
−
Argh. I have been trying to build firefox using distcc for about 2 weeks, with many small details getting in the way. At one point things were working, but than it stopped and I don't know why. I finally, at one point, got things to build on andrew's machine. But the build failed complaining about (this is one of the many, many things) ''undefined reference operator delete(void*)''. I think it might be because distcc is calling gcc rather than c++. I changed the MOZCONFIG to use CC='distcc gcc' and CXX='distcc g++'. This threw me this ugliness :
+
Microsoft (R) Incremental Linker Version 8.00.50727.42
−
make[1]: warning: jobserver unavailable: using -j1. Add `+' to parent make rule.
+
Copyright (C) Microsoft Corporation.  All rights reserved.
−
Which I told reed about, but has never came across this problem. One nasty solution is to use -j without specifing a number to allow unlimited jobs. At least it's working.
 
  
−
In other news, Tom sent a [http://lists.samba.org/archive/distcc/2006q4/003466.html patch].
+
/out:hello.exe
 +
hello.obj
 +
</pre></code>
 +
Works. But this doesn't :
 +
<code><pre>
 +
$ cl /cygdrive/c/home/temp/hello.c
 +
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86
 +
Copyright (C) Microsoft Corporation. All rights reserved.
  
−
== Oct. 2 2006 ==
+
cl : Command line warning D9002 : ignoring unknown option '/cygdrive/c/home/temp
−
A preliminary test of running distcc in cygwin revealed we were failing 4 tests; one or two more than our first patch. We tried to fix it, but never had success and didn't issue a commit. A while later we try running distcc again on another machine. This time we were only failing one test, which is less than our first patch. Here is the known tests to fail
+
/hello.c'
−
# EmptySource_Case --- This is expected to fail since this is a bug in the earlier gcc compiler.
+
cl : Command line error D8003 : missing source filename
−
The explination for the extra three preliminary tests failing is because we interupted one of the tests, which we believed caused the daemon to be orphaned and exacting it's revenge on the murder's of it's parent processes.
+
</pre></code>
 +
There are a few things we can do about this :
 +
* replace /cygdrive/c/ with C:\
 +
* user must replace the enviroment variable TMPDIR which defaults to /tmp
  
−
== Sept. 26 2006 ==
+
Cesar found another problem. Both C/C++ preprocessed file ends in .i. cl has no way of knowning if it's a C/C++ preprocessed file. So it refuses to compile it (it treats it as an object file). So when we preprocess, we have to redirect to the same extension as the source file.
−
Tried to do parallel builds with mozilla in linux (maybe we should have done this first). Here is something usefull to know :
+
<code><pre>
−
To parallel build in mozilla, you cannot pass the -jN option to make and expect it to work. It didn't for me, and make was killing itself without an explination. I later learned that you have to add the following line to mozconfig
+
$ cl hello.i
−
  mk_add_options MOZ_MAKE_FLAGS=-j8
+
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86
 +
Copyright (C) Microsoft Corporation.  All rights reserved.
  
−
== Sept. 24 2006 ==
+
cl : Command line warning D9024 : unrecognized source file type 'hello.i', objec
 +
t file assumed
 +
Microsoft (R) Incremental Linker Version 8.00.50727.42
 +
Copyright (C) Microsoft Corporation.  All rights reserved.
  
−
We think we've got the architecture right now. Ofcourse the only way to know is to start porting cl :). We're now going to try to port the tests to work with cl.exe.
+
/out:hello.exe
 +
hello.i
 +
hello.i : fatal error LNK1107: invalid or corrupt file: cannot read at 0xEE7F
 +
</pre></code>

Latest revision as of 11:50, 29 May 2007

Project Name

distcc, distributed compilations for C/C++ compilers.

Project Description

The goal of this project is to speed up Windows Mozilla Builds by spreading object compilations over several machines. This can be done by using an open source product called distcc created by Martin Pool. Unfortunately this is only available on *nix machines using gcc. Our project involves porting this software to Windows and use Microsoft's Visual C++ compiler using cygwin and msys.

Project Leader(s)

Project Contributor(s)

We have stuff for you to do!

  • Dave Liebreich - Thanks for checkin whether cygwin's ln will create working links to cl.exe (e.g. a link name clfoo.exe to cl.exe).
  • Ben Hearsum - for the windows box and support! Thank you!
  • Jason Spiro - For helping figure out why cl wouldn't execute in the unit testing framework and assisting in finding the right path to solve this issue.
  • Phil - helped with changing the page on helping people do research into cl options for us.
  • Moe
  • Mark P. (aka RealMarkP)
  • David Humphrey
  • Jeff
  • Melz

Product Documentation

The project has been focused on getting distcc to work with Mozilla, mainly because Mozilla uses GNU Makefiles which can run concurrent jobs. We have started a tutorial in order to help testers/developers. We have also written documentation for others who might wish to add support to distcc for other compilers (e.g., Java, C#, etc.).

Current Status

  • Some of the Test Cases may be incorrect. Neither Tom nor Cesar have extensive experience in CL options. So it may not distribute when it should, or it may distribute when it shouldn't.

What's still to do

Task Priority Status
Fix documentation, including website, manpage, and tutorials Medium Neglected. I think Tom has worked on the man page, Cesar has worked on the tutorial
  • find a suitable replacement for fork()
    • using windows threads. Need to use #ifdef directives to switch between unix fork and threads. But perhaps where is a better way to do this.
    • Perhaps talk to Martian Pool to see if he has any ideas? He hasn't been in the mailing list for a while.
 High Have not started
  • Send .pdb files to allow debug builds. Currently blocking mozilla from ever using distcc.
    • looking at the protocol. Hopefully we don't have to changed a lot.
 High Have not started
  • Port distcc to use mingw
    • Properly link in winsock library (fixed using libtool)
    • use windows threads (another high priority item)
    • locate and replace unix functionality that existed in cygwin, but doesn't exist in mingw.
 High Have only got libtool to compile a simple sockets program. We have now port it to distcc
  • probably use #ifdef MINGW ... #else ... #endif in source files that use socket.h
  • another possibility is to have one header file handle this, and the source files include that header file
Find more applications that use the Makefile system and can use CL Medium Difficult to find. Not many open source applications use Makefile and MSVC, many choosing to opt for NMake
Compile NSPR using distcc Medium Dave suggested only building NSPR because its smaller and won't take as long to compile. Cesar is responsible for this
boy there sure are lot of memory leaks... would be nice to fix that Low We can use tools like valgrind and mtrace to help. Or we can use Boehm-GC to free the memory automatically. But this is a lot of work for such low priority.
Make sure that there are no orphan distcc processes left after make maintainer-check Low This is a problem when tests are killed, and can leave to false-negatives when you try to rerun the tests again. But that can be fixed with god ol' kill-y mc. kill, agent 009


How to get distcc to compile use cl

Tom and Cesar created this patch keeping backwards compatibility in mind. You should be able to still use distcc to compile your c programs if you wish. But if you applied the patch, you probably want to do more than that.

Firstly, the environment variables for distributing with cl is different. DISTCC_HOSTS is still valid for gcc, but cl will be looking for DISCC_CL_HOSTS instead. In fact, that is the only thing it looks for. Unlike gcc, cl will not look for ~/.distcc directory. This might be implemented later on.

Secondly, masquerade mode works only for gcc. If you want want to use cl, you must explicitly say so (distcc cl hello.c). If you are using gnu make, you must also explictly tell it to use cl as well (eg using CC, CXX, etc).

Maintainer-check

NOTE :: If you run the checks, please check your processes (ps aux, or the task manager) for leaked distcc/distccd programs. We will fix this ASAP

You may ignore this section if you installed Microsoft's CL in the default C:\ directory and your tests work fine.

In order to properly run the tests, you may need to make a few changes. The PATH variable is reinitalized when the test cases are run. Normally, the tests will fail when they try the cl tests. This is because PATH does not look for the cl executable and the libraries. Tom and Cesar have made certain assemptions when extending the include path, and included these following paths :

/cygdrive/c/Program Files/Microsoft Visual Studio 8/VC/bin
/cygdrive/c/Program Files/Microsoft Visual Studio 8/Common7/IDE

If it is different for your machine, you need to find these in the Makefile.in or Makefile and change them to your settings.

Extending the program

This will be postponed until we get distcc working properly (not fully) with cl.

Project Details

Mozilla (and others :) ) is interested in speeding up their builds by using a distributed compiler (distcc). The problem is that distcc only works with gcc but not with the msvc compiler (cl).

The plan has two stages:

  1. change distcc to support multiple compilers
  2. add cl support

Initially, we planned to use two vtables (one for distcc and another for distccd) to keep track of which compiler specific operation to use when. However that bombed out so we're currently using a single vtable.

We have kept the two vtable branch because we actually got a lot of the cl specific stuff done (before we realized we missed some functions).

Help Us Out

Here's what you can do to help us out...:

The First Patch (Oops)

This was released to the distcc mailing list. This has a few functions missing. It was meant to provide the two vtable support without cl. It should be ignored and we'll be re-releasing a working version of this soon.

Project News

See what's up with this project! Older news can be found here

May 1, 2007

I've been doing a lot of #ifdef conditions, just to get it to compile (although we will be doing windows threads instead of fork, this is just a step). But now I'm stuck on the function fcntl(). It's a POSIX function, and it has POSIX flags, but some of the flags don't seem to exist in Windows. I don't have enough information to know how important the function is, and if we can just skip it.

And I forgot my power cable in a certain building, and laptop is running out of battery!

April 30, 2007

There are some header files in distcc that have the same name as some of the standard header files in /usr/include. One file is rpc.h, which is being #included by "sockets.h". Since we don't need sockets.h, this shouldn't be a problem in this case. But that is still something we have to watch out for.

Bah! windows.h #includes rpc.h, dammit.

Resource : Microsofts Unix migration guide

April 28, 2007

PDB files : 

vc80.pdb - source file debug symbols
project.pdb - project debug symbols

vc80.pdb can be overwritten by/Fddebug

Link between source.c[pp] and debug is hard coded in the object file. You can find it by running the strings command on the object file (Thanks shaver!).

Leaving debug symbols aside. We should probably start porting distcc to mingw and replacing incompatible functions. Since that would be more time consuming and more difficult to get right.

April 23, 2007

Semester is over. Because I honestly have no life, and there is work to do here, I started mapping out the distcc protocol, so we can start making any changes, if necessary. It was a wasted afternoon because the protocol is already documented in the trunk, under docs directory (who would have guessed?)

Preliminary examination indicates that this won't be so much work. The functions seem to already be there.

March 19, 2007

There was a subtle bug in Mozilla's code base that bug 274563 deals with that made it impossible for distcc to build Moz. If you apply the patch attached to the bug then you'll be able to do distcc builds of Mozilla. We will soon be releasing our first beta to reflect our confidence in the code.

Feb 27, 2007

We are able to do distributed builds of Mozilla but it dies when trying to link stuff together (issues surrounding ATL?). So in order to take advantage of Ben Smedberg's wonderful Mozilla Build and be able to get help figuring out weird linkage bugs and dependancies without having to deal with cygwin issues (which are galore), we've started a branch with a mingw port: svn://cdot.senecac.on.ca/distcc/branches/mingwcompatible/

Jan 4, 2007

The post made to the build newgroup seems to have gathered dust. If mozilla is limited to only 4 jobs, than distcc is somewhat limited. But that's not our problem.

Tom has found one problem with using cygwin. Using absolute pathnames. For example : 

$ cl hello.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

hello.c
Microsoft (R) Incremental Linker Version 8.00.50727.42
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:hello.exe
hello.obj

Works. But this doesn't : 

$ cl /cygdrive/c/home/temp/hello.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9002 : ignoring unknown option '/cygdrive/c/home/temp
/hello.c'
cl : Command line error D8003 : missing source filename

There are a few things we can do about this :

  • replace /cygdrive/c/ with C:\
  • user must replace the enviroment variable TMPDIR which defaults to /tmp

Cesar found another problem. Both C/C++ preprocessed file ends in .i. cl has no way of knowning if it's a C/C++ preprocessed file. So it refuses to compile it (it treats it as an object file). So when we preprocess, we have to redirect to the same extension as the source file. 

$ cl hello.i
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9024 : unrecognized source file type 'hello.i', objec
t file assumed
Microsoft (R) Incremental Linker Version 8.00.50727.42
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:hello.exe
hello.i
hello.i : fatal error LNK1107: invalid or corrupt file: cannot read at 0xEE7F
Retrieved from "https://wiki.cdot.senecapolytechnic.ca/w/index.php?title=Distcc_With_MSVC&oldid=12038"