Winter 2010 Posters/Koji Builder
Title
Koji Builder by: David Cabral (dpcabra1@learn.senecac.on.ca)
Introduction
The goal was to create machines/builders that work with the koji-hub to build packages on the ARM (armv5tel) architecture. Fedora Project doesn't have the resources to maintain the ARM architecture for all its packages and with the increase of actual ARM systems being used, the need of builders have too increased. Kojid is the build daemon that runs on the build machines. It's primary responsibility is polling for incoming build requests and handling them accordingly. Kojid is written in Python and communicates with koji-hub via XML-RPC.
Description
Setting up the builders according to the koji-hub configuration. The builder relys heavily on the koji-hub to be configured before kojid could functional. Kojid and koji-hub need to communicate with each other, so we use SSL Certifications to authenticate the requests. Each builder needs it's own certificate and needs to be put in the PostgreSQL database. Kojid uses Mock to create a chroot of a minimal installation to test the packages and make sure they install correctly and if the SRPM work it creates all the RPMs. We used one actual ARM machine and nine virtual machines to familiarize the steps involved in getting builders to work with a Koji Hub. The final goal is to make a Koji farm dedicated to building ARM packages for Fedora since the architecture is getting more and more popular everyday. Since there is only one other ARM Koji Farm which doesn't seem to be too current, we hope to create a current and ever going package building environment. Creating this farm will help with the Fedora Project Community development and Seneca to create interest in software building and release.
Challenges
- Builder had to be added to the PostgreSQL database before starting Kojid on the builders.
- After installing and configuring the builders they didn't seem to authenticate do to some complications with the certifications. Once new certifications where made the issues ceased.
- Koji-hub was seeing the builder and getting updates. However, the builder wasn't returning a ready signal. After digging threw the logs, we saw it was because the machines didn't have enough space on the systems to do actual builds. The virtual machines were re-sized to include some extra space and koji-hub finally got a ready signal from the builders.
- Documentation was lacking on the web, but it didn't create much problem for the builders since the Fedora Project wiki had the basic configurations required to setup kojid. After just a few weeks, the blogs we wrote became the top search items on Google.
- Understanding the relationship of package creation and Koji's method of organizing software packages in the database. This is still a learning process as Koji has just become fully functional.
Conclusion
Kojid is written in Python and uses XML-RPC to send requests between the builder and the koji-hub. Koji uses Mock to build its packages and then returns a status signal back to the koji-hub for easy accessibility of the logs. Koji-hub requires PostgreSQL to create a database to contain information of the builders and status of package building. Kojid works directly with the koji-hub. They use SSL Certifications to authenticate the requests and the packages so no rogue files/machines can interfere. Once the builders where configured and koji-hub was finally finished updating the repository information, the building of packages seemed to work without much other trouble. Koji has been setup and seems to be working.
Links
- David Cabral's Blog - http://pokerface3.blogspot.com/
- Fedora Project - http://fedoraproject.org/wiki/Koji/ServerHowTo
- Software Build and Release 600 wiki - http://zenit.senecac.on.ca/wiki/index.php/SBR600
- Fedora Arm Secondary Architecture wiki - http://zenit.senecac.on.ca/wiki/index.php/Fedora_ARM_Secondary_Architecture
Special Thanks
- Chris Tyler
- Dennis Gillmore
- The students of the SBR600 class
A special thanks to all of you. Without your help, I wouldn't have understood any of this Koji stuff.
Images
Deadline
- Your poster must be submitted to your professor by 2010-04-16 in order to leave sufficient time for printing. It would be a good idea to submit it early and get feedback from your prof in case revisions are necessary.
Presentation Day
Please plan to join us for the entire afternoon on April 23 in T1014 in case there are questions about your poster. The event starts at 1:30; please plan to be there by 1:00 pm. If you have other exams, discuss the exam schedule with your professor in advance. Dress appropriately for speaking to the audience, which will include industry partners, funding agencies, and Seneca administration.