View source for D-Bus and other Linux desktop integration improvements

'''Back To [http://zenit.senecac.on.ca/wiki/index.php/Project_List#D-Bus_and_other_Linux_desktop_integration_improvements  Project List]'''

== Project Name ==

D-Bus and other Linux desktop integration improvements.

== Project Description ==

Various Linux distributors have patches in their Firefox packages that add bits and pieces of Linux integration, and Mozilla would like to see even more available. One particular area of interest is controlling the browser via d-bus, and exposing dbus events to the application and extensions.

===Introduction to D-Bus:===
Currently on the Linux desktop, there is a void in desktop application integration. Many solutions are used such as CORBA, DCOP, etc…, but neither satisfies the new vision for an integrated Linux desktop.

D-Bus is relatively new; however, it has grown in popularity very quickly. The KDE project has announced it will replace DCOP with D-BUS in KDE version 4. Gnome has also made developments to include the D-Bus technology. Many corporations including Redhat and Novell are interested in this new technology.

D-Bus is meant to replace CORBA and DCOP, and provides a RPC communication technology. It is developed in the C language as an IPC process, but has high level language bindings for C++, QT, Java, Python, etc… Most Linux distributions already include a running D-Bus library. There is also a D-Bus port for Windows under development.

===Possible Ideas===
D-Bus can be integrated in two ways. One is to add a patch in code to use expose D-Bus events from the system or services (such as Network Connectivity or Services Status) to Mozilla. The other is to develop an API framework that allows Mozilla to communicate with other applications.

There is currently a project called DBuzzilla which has done a good job implementing a module which allows Mozilla to communicate between applications or the system. The author has mentioned he had some personal issues, and has not touched the project since the summer of 2006. He said he will continue to maintain his project. There might be an interest working with this project. However, it seems the author is not willing to work in a manner that is consistent with open development. It is maintained by one person, with some or none users. It is not complete, but is a good start. If this project is to be continued and maintained by Seneca students, it is probably a good idea to start a new project that is updated to the latest version of D-Bus and is developed in an open development manner.

===Project Objective and Features===

The goal of this project is to create a module that will expose the D-Bus technology to Mozilla technologies. This will work by coding a set of libraries and processes in XPCOM and D-Bus. The learning curve in this project is long (assuming you do not know XPCOM). You should also be or become familiar with Linux.

You will be coding in the low level C library of D-Bus as per the recommendation of Havoc Pennington (D-Bus project leader). The D-Bus library C functions will be wrapped around your C++ objects. Depending on the development route chosen, you will either using the Glib library or the NSPR library to handle threads. NSPR must be used for the data type bindings. You will also code using XPCOM to expose the API to other Mozilla components.

The module must include the basic objects, data types, parsers, client/server processes, etc….

Features:

*'''Phase 1''' - Basic D-Bus Communication Classes (using XPCOM and NSPR)
**'''DBus''' - Singleton class which starts and ends D-Bus client and server processes in Mozilla
**'''Connection'''
**'''Message''' - Abstract base class used by the following:
***'''Signal''' - Intent is to send a signal to notify listening applications of some status
***'''MethodCall''' - Intent is to call a method (like RPC) with the option of arguments
***'''MethodResponse''' - Intent is to send a response to a method call
***'''ErrorResponse''' - Intent is to send a error response to a method call
**'''Argument''' - Attached to a method call
**'''ArgumentIterator''' - Used to read each ArgumentList
**'''ArgumentList''' - Used to contain all arguments from a method call or response
*'''Phase 2''' - Enrich Framework
**'''Multi-threading'''
***'''NSPR'''
***'''Glib'''
**'''Proxy''' classes
**'''Mozilla''' process
***'''Client''' (handles requests from internal sources, other XPCOM modules)
***'''Server''' (handles requests from external sources)
***'''Monitor''' (listens to system and applications events to kill internal dependencies, ex. an XPCOM module waiting for a response from an external source which was just killed, dbus sent us a signal telling us this source went offline)
***'''Dispatcher''' (notifies the system and applications Mozilla events, ex. Firefox closing)
*'''Phase 3''' - Advanced Features
**'''Introspect'''
**'''Complete D-Bus spec''' - there are some features in the D-Bus spec for future development, such as D-Bus via Sockets

The project design is not fixed and it is very open to suggestions.

===Considerations===

D-Bus has recently frozen the API for version 1.0. 
It is unclear if the DBuzilla project has had any updates with this new version or if of the design will be affected. 
Documentation doesn't always exist for the technologies being used. When learning D-Bus, you may have to play with a function to understand its behaviour.

Read the mailing list discussion we started. A good description of problems we will encounter is in a few of them.

==Important Links==
====Mailing Lists====
*[http://lists.freedesktop.org/archives/dbus/2006-November/006387.html Our Post to Freedesktop Dbus mailing list - Need some help and direction]
*[http://lists.freedesktop.org/archives/dbus/2006-November/006388.html Havoc Response - Direction we should go]
*[http://lists.freedesktop.org/archives/dbus/2006-November/006389.html Gary Response - Problems we will encounter]
*[http://lists.freedesktop.org/archives/dbus/2006-November/006394.html Havoc's Response to Gary]
*[http://lists.freedesktop.org/archives/dbus/2006-November/006404.html Discussion Continues]
*[http://lists.freedesktop.org/archives/dbus/2006-November/006420.html Discussion Continues]
*[http://lists.freedesktop.org/archives/dbus/2006-November/006396.html Discussion Continues]

====Articles====
*http://www.linuxjournal.com/article/7744.
*[http://www.redhat.com/magazine/003jan05/features/dbus/ Get on D-BUS]

====Tutorials & Specification====
*http://www.freedesktop.org/wiki/Software/dbus
*http://dbus.freedesktop.org/doc/api/html/group__DBusConnection.html
*http://en.wikipedia.org/wiki/D-BUS
*[http://www.matthew.ath.cx/misc/dbus Using the D-Bus C API]
*[http://www-128.ibm.com/developerworks/linux/library/l-dbus.html Connect desktop apps using D-BUS]

====DBuzilla====
*[http://code.google.com/soc/mofo/appinfo.html?csaid=13DDE9DE6AF50C64 DBuzilla - Google summer of code]
*[http://zenit.senecac.on.ca/wiki/index.php/Dbuzilla_report DBuzilla Report]

====Patches====
*[https://bugzilla.mozilla.org/show_bug.cgi?id=312793 Support DBUS and listen to NetworkManager events]

====Mozilla====
*[http://books.mozdev.org/html/mozilla-chp-8.html Creating Applications with Mozilla]
*[http://www.mozilla.org/projects/xpcom/book/cxc/ Creating XPCOM Components]
*IBM developerWorks - XPCOM Overview [http://www-128.ibm.com/developerworks/webservices/library/co-xpcom.html part 1] [http://www-128.ibm.com/developerworks/webservices/library/co-xpcom2.html part 2] [http://www-128.ibm.com/developerworks/webservices/library/co-xpcom3.html part 3] [http://www-128.ibm.com/developerworks/webservices/library/co-xpcom4/index.html part 4] [http://www-128.ibm.com/developerworks/webservices/library/co-xpcom5.html part 5]
*[http://www.mozilla.org/scriptable/xpidl/idl-authors-guide/index.html IDL Author's Guide]
*[http://developer.mozilla.org/en/docs/XPIDL:xpidl XPIDL]

====Linux Desktop====
*[http://developer.gnome.org/ Gnome Developers]
*[http://en.wikipedia.org/wiki/DCOP KDE DCOP]

== Project Leader(s) ==

Project Leader: '''[[User:mattar1|Mohamed Attar]]''', '''[[User:mckwan|Man Choi Kwan]]'''

Please contact '''[mailto:mattar1@learn.senecac.on.ca Mohamed]''' if you are interested in joining this project. Make sure you have done the following first:

# Be sure to pick out your favourite coffee mug. 
# Find some music that will keep you up all night (trance, happy hardcore, etc... check out www.di.fm).
# TBA

== Project Contributor(s) ==

[[User:Themystic|Tom Aratyn]]  - Gave us a lecture on XPCOM and helped us figure our where we should be looking for D-Bus/XPCOM integration.

[[User:David.humphrey|Dave]] - Has given us direction and ideas to start the project.

[[User:Andrew|Andrew Smith]] - Setup SVN for us, but helped fix our environment with Anjuta.

== Project ==

The following is our code, and some tutorials on XPCOM and D-Bus.

=== Project Code ===

This is our current basic class files and tutorials. Download it [http://matrix.senecac.on.ca/~mckwan3/dbus/dbus_tutorials.tar.gz here].

We were converting the C dbus examples into C++ objects.

We used Anjuta (an IDE), but you can just go into the src dir and type 'make' (in the command line).

This is the [http://matrix.senecac.on.ca/~mckwan3/dbus/dbuzilla.tar.gz DBuzilla project file] which will be valuable later. Currently the author has taken down the code or the server just is down for some reason.

=== XPCOM Tutorial and info ===
XPCOM (Cross Platform Component Object Model) is a cross platform component model from Mozilla. 

It is similar to CORBA or Microsoft COM. It has multiple language bindings and Interface Description Language let the XPCOM components be used and implemented in JavaScript, Java, and Python in addition to C++. 

Interfaces in XPCOM are defined in a dialect of IDL called XPIDL.  

XPCOM itself provides a set of core components and classes, e.g. file and memory management, threads, basic data structures (strings, arrays, variants), etc. 

The majority of XPCOM components is not part of this core set and is provided by other parts of the platform (e.g. Gecko or Necko) or by an application or even by an extension.

For D-Bus to work in Mozilla environment we need to create C++ XPCOM components to access the methods with in the D-Bus framework. 

To create a C++ XPCOM component we need to create the implementation file and component modules for the required methods. 

Once the components are done we can create interface definition file to connect with the XPCOM components.  

Follow the steps how to create C++ XPCOM component are listed below.

====Packages needed:==== 
* C++ Compiler
* egcs
* gcc
* GNU make
* GTK/GLib
* Perl 5
* tar

====Create the Component====
1.	Download the Gecko SDK.
* You can find it at ftp://ftp.mozilla.org/pub/mozilla.org/mozilla/releases/mozilla1.7.13
* You can choose a different Mozilla release if you like.
* Extract the SDK to a local directory.

2.	Create a GUID for the main interface.
* You can use the uuidgen utility to generate the unique 128 bit number. Or
* You can use one of the special “bots” on IRC at the irc.mozilla.org server (irc://irc.mozilla.org/#mozilla).

3.	Create the interface definition file – ImyComponent.idl
* Use the following template add methods and attributes to the interface.
* Fill in the GUID you have generated.

  #include “nsISupports.idl”
  [scriptable, uuid(YOUR_INTERFACE_GUID)]
  interface IMyComponent:nsISuports
  {	
     int MyMethod(in int a, in int b);
  };
4.	Generate the interface header and typelib files out of the interface definition file.
* Use the xpidl utility that comes with Gecko SDK.
* Substitute the “_DIR_” in the following commands with the full path to the xpcom/idl directory found under the Gecko SDK main directory.
* xpidl -m hearer  -I_DIR_ IMyComponent.idl will created the IMyComponent.h hearder file.
* xpidl  -m typelib -I_DIR_ IMyComponent.idl will create the IMyComponent.xpt typelib file.

5.	The interface header file IMyComponent.h contains templates for building the component header and implementation files. You can copy and paste the templates to create these files, changing only the component name.

6.	Create the component header file “MyComponent.h”.
* Start by inserting double includsion protection code (#ifndef _My_COMPONENT_H_ ….).
* Add #include “IMyComponent.h” to include the interface definition.
* Create a GUID for the component.
* Add the following lines, which define the component name, contract ID, and GUID:

   #define MY_COMPONENT_CONTRACTID “@mydomain.com/XPCOMSample/MyComponent;1”
   #define MY_COMPONENT_CLASSNAME “A Simple XPCOM Sample”
   #define MY_COMPONENT_CID  _YOU_COMPONENT_GUID_

* Copy the header template from IMyComponent.h (starting with /*Header file */) into the MyComponent.h file.
* Replace all the instances of _MYCLASS_ with the name of the component.

7.	Create the component implementation file “MyComponent.cpp”.
* Add #include “MyComponent.h” to include the component definitions.
* Copy the implementation template from IMyComponent.h (starting with /*Implementation file */) into the MyComponent.cpp file.
* Replace all the instances of _MYCLASS_ with the name of the component.
* Add method implementation code.

8.	Create the module definitions file “MyComponentModule.cpp”

* Add #include “nsIGenericFactory.h” to include Mozilla GenericFactory definitions.
* Add #include “MyComponent.h” to include the component definitions.
* Add NS_GENERIC_FACTORY_CONSTRUCTIOR(MyComponent)  to define the constructor for the component.
* Add
   static nsModuleComponentInfo components[] = 
   {
       {
             MY_COMPONENT_CLASSNAME, MY_COMPONENT_CID,
            MY_COMPONENT_CONTRACTID, MyComponentConstructor,
       }
   };

 to define the class name, contract ID and GUID of the component.

* Add NS_IMPL_NSGETMODULE(“MyComponentsMoudle”, components) to export the above information to Mozilla.

9.	Create a custom makefiles.

====Building the Component====

1.	Build the component.

* Navigate to the src directory.
* Issues the make command. MyComponent.so file is created.

2.	Register the new component with Mozilla.

*Copy MyComponent.so and IMyComponent.xpt file to the Mozilla components directory.
* This directory is typically located at ~/firefox/components (or ~/mozilla/components).
* Run the regxpcom -x COMPONENTS_DIR command supplied with the Gecko SDK to register the new component where COMPONENT_DIR is the component directory.
* Delete the xpti.dat and compreg.dat file form the Mozilla profile directory. These files will be automatically rebuilt by Mozilla the next time it is restarted.
* The profile directory is typically located at:~/mozilla/firefox/default.????

3.	Test the new component.

=== DBUS Tutorial and info ===

I'll walk through a simple example of sending a signal to give you an example of what its like.

It would be good if you know how to connect to databases. The concepts are similar, but rather than connecting to a database you connect to a bus.

First, you either need to use an IDE like Anjuta or learn how Make and its syntax.

Make sure you linux distro has Dbus installed. 

Quick and dirty way of checking is typing dbus-viewer in teh command line or locate dbus. =P

You will need to include the libdbus libraries in your project or make files. They are not in the default path.

The first header files you will need:

 #include <stdio.h>
 #include "glib.h"
 #ifndef DBUS_API_SUBJECT_TO_CHANGE
 #define DBUS_API_SUBJECT_TO_CHANGE
 #include <dbus/dbus.h>
 #endif

You need the define DBUS_API... if you not using the dbus version 1.0+.

You need a connection to the DBus server running, and an error object for the error message.

In C, you the function return is the state usually 0 for null then you check the error object for the message.

 int main()
 {
  DBusConnection *conn;
  DBusError error;
  dbus_error_init(&error);

Use the dbus native functions to connect to the System bus. You can also connect to teh session bus which is usually meant for application to application communication. System is meant for events, but you can use either one. There is also a starter bus, but not meant to be used by us.

  conn = dbus_bus_get(DBUS_BUS_SYSTEM, &error);
  if(conn == NULL)
  {
    fprintf(stderr, "Failed to open connection to bus: %s\n",error.message);
    dbus_error_free(&error);
    exit(1);
   }
   else printf("Connected to System Bus\n");

If you succeed at that point, you will have to create you signal.

This involves creating a DBusMessage (represents the signal) and DBusMessageIter (which used to store arguments).

You need a unique number to associate replies with requests. Will be used to manage communication later.

Signal value is a string.

 dbus_uint32_t serial = 0;
 DBusMessage* msg;
 DBusMessageIter args;
 const char* sigvalue = "Hello World";

You do the following to make teh Message object a signal. The internals adds the flags it a signal. There are other functions that make a message into a methodcall, method response, or errors.

 printf("Attempting to create a new Signal.\n");
 msg = dbus_message_new_signal("/com/halcyoncomplex/test",// object name of the signal
                               "com.halcyoncomplex.test", // interface name of the signal
                               "test");                   // name of the signal
 if (NULL == msg)
 { 
  fprintf(stderr, "Message Null\n"); 
  exit(1); 
 }
 else printf("Signal Created.\n");

Adding arguments. Type string for our Hello World.

 printf("Attempting to add arguments to signal.\n");
 dbus_message_iter_init_append(msg, &args);
 
 if (!dbus_message_iter_append_basic(&args, DBUS_TYPE_STRING, &sigvalue))
 { 
  fprintf(stderr, "Out Of Memory!\n"); 
  exit(1);
 }
 else printf("Added argument of type(String) with value(Hello World)\n");

Now we send the signal.

  printf("Attempting to send signal.\n");
  // send the message and flush the connection
  if (!dbus_connection_send(conn, msg, &serial)) 
  { 
   fprintf(stderr, "Out Of Memory!\n"); 
   exit(1);
  }
  else printf("Signal Sent. =)\n");

Clean up.

  printf("Serial value for my signal is: %u\n",&serial);
  //printf("Flushing connection.\n");
  dbus_connection_flush(conn);
  printf("Unreferencing message object.\n");
  // free the message 
  dbus_message_unref(msg);
  printf("Closing connection.\n");
  dbus_connection_close(conn);
 }

The concepts aren't hard, getting things working is a little bit.

If you've taken Mehmet's JDBC Connectivity or Jordan's Web Services you might be able to relate some concepts and patterns.

I don't think Seneca has taught any RPC in their BTP series classes, but reviewing BTP600 patterns will help.

D-Bus is a RPC-like technology and when making your code run in threads or asynchorous you'll need to understand proxies (along with more patterns) very well.

== Experiences ==

=== Mohamed Attar ===

The first problem I had was trying to understand where D-Bus is suppose to fit into Mozilla.

It wasn't very clear to me who to talk to in Mozilla, so we looked elsewhere and got some great feedback from the Freedesktop.org community.

I got some information from the author of DBuzilla, but nothing I didn't know already from his blog.

Once the project objectives was in perspective, getting started was more motivating.

Man and I realized working together on D-Bus wasn't working out too well, so we split the work where I would stick with D-Bus and he would work on Mozilla technologies and try to meet somewhere in the middle.

We had more trouble getting Mozilla stuff to work. It needs a few long hours to get through the tutorials and get a running example, but we only had bits of time here and there so we got stuck there. Tom helped us figure a few things out, but it was pretty late in the project to capitalize.

Either way it was a worth while struggle. We learned more than we developed. If we get marked on deliverables rather than what we learned then I am fearful. lol

Maybe with the time off I can get back into it. I enjoyed this project probably more than most others, except for the feeling of failure and being overwhelmed all the time.

What would I recommend to the next group of students interested in this project. Work in small steps. Don't think of how the framework should be designed and what the best ways in coding something until you are comfortable in both or all technologies. Just enough experience to develop and debug your code. Divide the work based on technologies. Even if you want to learn it all, if each of you can make something work with each technology only then can you work with each other and use each others experience to make something useful. You'll have to learn the other side anyways, its just easier to struggle in one technology rather than both. Split the pain, then share it.

== Suggestions & Ideas ==

* Shaver passed along this link to [http://trac.csquad.org/wiki/AboutDBuzilla#CurrentStatus DBuzilla], which would be good to investigate.

== Project News ==

===Status===
<u>As of December 15th:</u>

Our project objectives and vision is defined.

We got responses from the community about problems we will run into, and got some status news from the author of DBuzilla.

We have some C code running using D-Bus. Working on the phase 1 classes.

Code is still very immature.

We need to get XPCOM tutorials running, and try exposing some basic classes and functions.

After we have a running dbus object(s) with XPCOM, try using DBuzilla.

Man and Mohamed are still interested in continuing over the break.

===News Posts===

<u>December 15th, 2006</u>

Sorry we haven't updated for a while.

Tom, Man, and I (Mohamed) had a discussion trying to understand how to use Mozilla technologies with D-Bus. Tom has helped us find some important information.

We're just updating the wiki now and documenting our work for a lower entry barrier for other students.

<u>November 28th, 2006</u>

I found an email from the author of DBuzilla in my spam folder. Basically telling us he hasn't work on it for a few months due to some personal issues, but he is planning to maintain it again.

<u>November 13, 2006</u>

Got some info from freedesktop.org mailing list, can be found [http://lists.freedesktop.org/archives/dbus/2006-November/006387.html here].

<u>November 9th, 2006</u>

Presented project to class.[http://zenit.senecac.on.ca/wiki/index.php/DBus_status_reprt DBus Status Report]

<u>November 7th, 2006</u>

Per request of [http://zenit.senecac.on.ca/wiki/index.php/User:David.humphrey David Humphrey] the [http://zenit.senecac.on.ca/wiki/index.php/Dbuzilla_report DBuzilla Report].

We just got a response from the author of DBuzilla. I have sent another email asking him what the status of DBuzilla is and what he plans to do with it. Asked if he is maintaining it. Hope to get a response soon. =)

<u>November 1st, 2006</u>

Man and I (Mohamed) are kind of lost. We wanted to write an extension to expose a HAL signal via DBus for network connectivity updates to Mozilla. We found somebody from Novell already did that and its in the trunk. We've talked to Dave about our problem. He recommended we write up a report about what DBuzilla does, supports, whats missing, etc... and he'll talk to the Mozilla guys to see if that can be used for any purposes. We'll also ask around the GNOME and Freedesktop groups and see if they have some ideas about DBus integration with Mozilla.

<u>October 30, 2006</u>

We're waiting for a response from Christophe Nowicki about his project. Man thinks its best to write a patch and help improve the DBuzilla project.

We're still working on our extension to become familiar with the technologies, and we'll setup and try DBuzilla.

<u>October 29, 2006</u>

A few things to update us on.

Over the break I took that XUL and Firefox Extensions workshop, and learned how to make a basic extension. 

Man and I have continued to play with extensions, and are trying to learn how to make XPCOM objects.

We've both decided to use Anjuta to start. We got our systems setup, built a few dbus examples and continued to read the specification.

We've decided to start our project with an extension that will give firefox some better awareness of the network connectivity. The progress has been slower than we expected by a long shot, but we figured if we can get something small working then we can build something more usable off that.

We've checked out the DBuzilla project and that looks great. I hope to steal some code from there.
I've contacted Christophe Nowicki, the DBuzilla author, about his project. The project hasn't been update for about 2 months, so I was wondering if he is currently working on that project or if he has left it for now. 

<u>October 8, 2006</u>

I finally got my laptop working with Ubuntu and was able to fix my resolution issue. 
Thanks to the Linux club and my friend Stewart from work for helping me fix that.

Since then I've been trying to compile some examples of dbus.

I had a lot of trouble because gcc coudn't find where dbus.h and other headers files were located, and no tutorial I found actually showed the command and flags used to compile it.

Thanks to [http://lists.freedesktop.org/archives/dbus/2006-June/005032.html this] mailing list question I was able to figure out the paths and flags (with a few changes for my system).

Its not much progress, but its a step.

<u>October 6, 2006</u>

Man continued looking through codes in [http://trac.csquad.org/wiki/AboutDBuzilla#CurrentStatus DBuzilla] to further understand
the existing code.

<u>October 4, 2006</u>

Try to install XULRunner with no successes. Ask [[User:Mylau|Michael Lau]] for help, well try again later.

<u>September 27th, 2006</u>

Man and Mohamed are setting up a server to compile our work on. We're reading and playing around with some D-Bus client/server stuff before we try to find where it fits in Mozilla. Slow start so far. =(

<u>September 15th, 2006 </u>

Today [[User:mckwan|Man Choi Kwan]] and [[User:mattar1|Mohamed Attar]] joined this project. We're just getting introduced to this topic and will be doing some research over the weekend.

==To Do==
*Code some XPCOM tutorial (not successful yet)
*Code some base classes of Phase 1 (work in progress)
*Get help with XPCOM (started, work in progress)
*Code some examples of D-Bus (done)
*Contact other communities that might help us the project idea, goal to contact Freedesktop (done)
*Ensure all group members have a machine they can run linux on with X11. (done)
*Research on D-Bus. Get some code and research to share with the group. (done, work in progress)
*Find some Mozilla/Linux Gurus to bother. =P (not successful)