Winter 2010 Posters/Koji Hub Poster

From CDOT Wiki
Jump to: navigation, search


Project Name

Koji Build System for Fedora ARM

Name

Paul Whalen

The Koji Build System

The goal was to set out to create a Koji build system for the ARM architecture. Koji is the software that builds and tests packages for the Fedora Project. The software package used by Fedora is called an RPM and provides a means for easy installation, removal and updating. The process of building software on a platform ensures that all the necessary files are available for that software package to function in the given hardware environment. Currently we are aware of one other Koji Build System for the ARM architecture which is located in Germany.

The plan was to initially set up all components of the Koji build system on one server, with the Koji builder Daemons running on Virtual ARM Machines and the physical hardware as it arrives. Planned hardware includes various applications of the ARM processor. After installation and configuration of the PostgreSQL, the Koji Hub would be configured, followed by the other components of Koji including the Koji Web, Kojira and finally the set up of the actual builders themselves.

Koji Components

Koji Hub - an XML-RPC server that receives and relies upon calls from the Koji Builder Daemons and other Koji components to initiate communications.

Koji Web - an interactive web interface that runs on mod_python modules in the Apache Web Server. The Koji Web provides an interface for Koji users to see the status of queued builds and information for trouble shooting unsuccessful build attempts.

Koji Builder Daemon - the build daemon that runs on each of the Koji build machines and is responsible for polling the Koji Hub for build requests. Once a request is received for the Koji Hub the Koji Builder will initiate a test build of the software package ensuring there are no errors, if errors are detected they are logged for further analysis.

Components Required By Koji

Apache Web Server - The most popular web server on the internet Apache is responsible for serving the pages provided by the Koji Web scripts. Apache is configured to use a variety of modules including SSL and mod_python. The SSL modules facilitates authentication of users and the different components of the Koji Build System.

PostgreSQL - PostgreSQL is the back-end database used by the Koji Build System. It stores tables of information including users, packages and build categories.

Mock - mock is a tool that creates a protected and minimal Linux file system that is used to test build software packages.

Problems Encountered

  • Initially the Virtual Machines had no network connectivity, network settings and devices had to be added manually before becoming functional.
  • Problems using the PostgreSQL database - installation for use with Koji required a standard installation. After the first attempt PostgreSQL was reinstalled successfully and Koji was successfully able to interact with the database.
  • After what appeared to be a successful installation and configuration, Koji builders were listed as not being fully functional, this appeared to be an issue with the Koji Hub but was later determined to be a lack of disk space on the Koji builders. Disk space was reallocated and distributed amongst the ARM VM's.
  • Authentication using SSL certificates - The SSL certificates posed some authentication issues due to conflicting configurations when generated. The certificates were recreated and authentication was successful.
  • Various configuration issues - throughout the process the different configuration files that required editing were mis-configured, or needed to be edited multiple times to work correctly with the Koji configuration.
  • The creation of repositories by Kojira was failing after 2919/12788 software packages (RPMS), additional memory was added to the Virtual ARM Machines an the repository creation continued.
  • Lack of documentation - because this is the second Koji Build System for the ARM architecture, there are limited resources. Within weeks of starting work and blogging on Koji, our blogs became the top results when searching the internet.
  • Understanding the relationship of package creation and Koji's method of organizing software packages in the database. This is still a learning process as Koji has just become fully functional.

Results

Currently we have a functional Koji Build system. The Koji Hub is successfully dispatching packages to the Koji builder Daemons and Kojira is creating new repositories for use with the ARM architecture. Eight VM's and the Open-RD Client are online and are listed as Ready and Enabled using the Koji command line tool and the web interface through Koji Web. Koji Web can be used to monitor all Koji operations and provide real time log information that can be used to troubleshoot problems as they arise.

Acknowledgements

  • Dennis Gilmore
  • Chris Tyler

Links

Logos

  • Fedora
  • Seneca
  • Apache
  • PostgreSQL

Images

Retrieved from "https://wiki.cdot.senecapolytechnic.ca/w/index.php?title=Winter_2010_Posters/Koji_Hub_Poster&oldid=38360"