Difference between revisions of "Winter 2010 Posters/Koji Hub Poster"
(→Acknowledgements) |
(→Components Required By Koji) |
||
| (32 intermediate revisions by the same user not shown) | |||
| Line 3: | Line 3: | ||
= Project Name = | = Project Name = | ||
| − | Koji | + | Koji Build System for Fedora ARM |
| − | = | + | = Name = |
Paul Whalen | Paul Whalen | ||
| Line 11: | Line 11: | ||
= The Koji Build System = | = The Koji Build System = | ||
| − | + | The goal was to set out to create a Koji build system for the ARM architecture. Koji is the software that builds and tests packages for the Fedora Project. The software package used by Fedora is called an RPM and provides a means for easy installation, removal and updating. The process of building software on a platform ensures that all the necessary files are available for that software package to function in the given hardware environment. Currently we are aware of one other Koji Build System for the ARM architecture which is located in Germany. | |
| − | The plan was to initially set up all components of the Koji build system on | + | |
| + | The plan was to initially set up all components of the Koji build system on one server, with the Koji builder Daemons running on Virtual ARM Machines and the physical hardware as it arrives. Planned hardware includes various applications of the ARM processor. After installation and configuration of the PostgreSQL, the Koji Hub would be configured, followed by the other components of Koji including the Koji Web, Kojira and finally the set up of the actual builders themselves. | ||
| + | |||
= Koji Components = | = Koji Components = | ||
| Line 21: | Line 23: | ||
Koji Builder Daemon - the build daemon that runs on each of the Koji build machines and is responsible for polling the Koji Hub for build requests. Once a request is received for the Koji Hub the Koji Builder will initiate a test build of the software package ensuring there are no errors, if errors are detected they are logged for further analysis. | Koji Builder Daemon - the build daemon that runs on each of the Koji build machines and is responsible for polling the Koji Hub for build requests. Once a request is received for the Koji Hub the Koji Builder will initiate a test build of the software package ensuring there are no errors, if errors are detected they are logged for further analysis. | ||
| − | = | + | = Components Required By Koji = |
| − | + | Apache Web Server - The most popular web server on the internet Apache is responsible for serving the pages provided by the Koji Web scripts. Apache is configured to use a variety of modules including SSL and mod_python. The SSL modules facilitates authentication of users and the different components of the Koji Build System. | |
| + | |||
| + | PostgreSQL - PostgreSQL is the back-end database used by the Koji Build System. It stores tables of information including users, packages and build categories. | ||
| + | |||
| + | Mock - mock is a tool that creates a protected and minimal Linux file system that is used to test build software packages. | ||
| + | |||
| + | = Problems Encountered = | ||
| + | * Initially the Virtual Machines had no network connectivity, network settings and devices had to be added manually before becoming functional. | ||
| + | |||
| + | * Problems using the PostgreSQL database - installation for use with Koji required a standard installation. After the first attempt PostgreSQL was reinstalled successfully and Koji was successfully able to interact with the database. | ||
| − | + | * After what appeared to be a successful installation and configuration, Koji builders were listed as not being fully functional, this appeared to be an issue with the Koji Hub but was later determined to be a lack of disk space on the Koji builders. Disk space was reallocated and distributed amongst the ARM VM's. | |
| − | - | + | * Authentication using SSL certificates - The SSL certificates posed some authentication issues due to conflicting configurations when generated. The certificates were recreated and authentication was successful. |
| − | - | + | * Various configuration issues - throughout the process the different configuration files that required editing were mis-configured, or needed to be edited multiple times to work correctly with the Koji configuration. |
| − | - | ||
| − | + | * The creation of repositories by Kojira was failing after 2919/12788 software packages (RPMS), additional memory was added to the Virtual ARM Machines an the repository creation continued. | |
| + | * Lack of documentation - because this is the second Koji Build System for the ARM architecture, there are limited resources. Within weeks of starting work and blogging on Koji, our blogs became the top results when searching the internet. | ||
| + | * Understanding the relationship of package creation and Koji's method of organizing software packages in the database. This is still a learning process as Koji has just become fully functional. | ||
| + | |||
| + | = Results = | ||
| + | Currently we have a functional Koji Build system. The Koji Hub is successfully dispatching packages to the Koji builder Daemons and Kojira is creating new repositories for use with the ARM architecture. Eight VM's and the Open-RD Client are online and are listed as Ready and Enabled using the Koji command line tool and the web interface through Koji Web. Koji Web can be used to monitor all Koji operations and provide real time log information that can be used to troubleshoot problems as they arise. | ||
= Acknowledgements = | = Acknowledgements = | ||
| − | Dennis Gilmore | + | *Dennis Gilmore |
| − | Chris Tyler | + | |
| + | *Chris Tyler | ||
= Links = | = Links = | ||
| − | *Fedora Project - http://fedoraproject.org/wiki/Koji/ServerHowTo | + | * Fedora Project - http://fedoraproject.org/wiki/Koji/ServerHowTo |
| + | * Paul Whalen's SBR600 Blog - http://paulfedora.wordpress.com/ | ||
| + | * Software Build and Release 600 wiki - http://zenit.senecac.on.ca/wiki/index.php/SBR600 | ||
| + | |||
| + | * Fedora Arm Secondary Architecture wiki - http://zenit.senecac.on.ca/wiki/index.php/Fedora_ARM_Secondary_Architecture | ||
= Logos = | = Logos = | ||
| Line 45: | Line 65: | ||
* Fedora | * Fedora | ||
* Seneca | * Seneca | ||
| + | * Apache | ||
| + | * PostgreSQL | ||
| + | |||
| + | = Images = | ||
| + | * Open-RD Client Board - http://www.einfochips.com/marvell/images/OpenRD-Client-Zoom.png | ||
| + | * RPM Package Logo - http://www.linux-support.com/cms/images/stories/illutrations/rpm-package.png | ||
Latest revision as of 23:36, 16 April 2010
Contents
Project Name
Koji Build System for Fedora ARM
Name
Paul Whalen
The Koji Build System
The goal was to set out to create a Koji build system for the ARM architecture. Koji is the software that builds and tests packages for the Fedora Project. The software package used by Fedora is called an RPM and provides a means for easy installation, removal and updating. The process of building software on a platform ensures that all the necessary files are available for that software package to function in the given hardware environment. Currently we are aware of one other Koji Build System for the ARM architecture which is located in Germany.
The plan was to initially set up all components of the Koji build system on one server, with the Koji builder Daemons running on Virtual ARM Machines and the physical hardware as it arrives. Planned hardware includes various applications of the ARM processor. After installation and configuration of the PostgreSQL, the Koji Hub would be configured, followed by the other components of Koji including the Koji Web, Kojira and finally the set up of the actual builders themselves.
Koji Components
Koji Hub - an XML-RPC server that receives and relies upon calls from the Koji Builder Daemons and other Koji components to initiate communications.
Koji Web - an interactive web interface that runs on mod_python modules in the Apache Web Server. The Koji Web provides an interface for Koji users to see the status of queued builds and information for trouble shooting unsuccessful build attempts.
Koji Builder Daemon - the build daemon that runs on each of the Koji build machines and is responsible for polling the Koji Hub for build requests. Once a request is received for the Koji Hub the Koji Builder will initiate a test build of the software package ensuring there are no errors, if errors are detected they are logged for further analysis.
Components Required By Koji
Apache Web Server - The most popular web server on the internet Apache is responsible for serving the pages provided by the Koji Web scripts. Apache is configured to use a variety of modules including SSL and mod_python. The SSL modules facilitates authentication of users and the different components of the Koji Build System.
PostgreSQL - PostgreSQL is the back-end database used by the Koji Build System. It stores tables of information including users, packages and build categories.
Mock - mock is a tool that creates a protected and minimal Linux file system that is used to test build software packages.
Problems Encountered
- Initially the Virtual Machines had no network connectivity, network settings and devices had to be added manually before becoming functional.
- Problems using the PostgreSQL database - installation for use with Koji required a standard installation. After the first attempt PostgreSQL was reinstalled successfully and Koji was successfully able to interact with the database.
- After what appeared to be a successful installation and configuration, Koji builders were listed as not being fully functional, this appeared to be an issue with the Koji Hub but was later determined to be a lack of disk space on the Koji builders. Disk space was reallocated and distributed amongst the ARM VM's.
- Authentication using SSL certificates - The SSL certificates posed some authentication issues due to conflicting configurations when generated. The certificates were recreated and authentication was successful.
- Various configuration issues - throughout the process the different configuration files that required editing were mis-configured, or needed to be edited multiple times to work correctly with the Koji configuration.
- The creation of repositories by Kojira was failing after 2919/12788 software packages (RPMS), additional memory was added to the Virtual ARM Machines an the repository creation continued.
- Lack of documentation - because this is the second Koji Build System for the ARM architecture, there are limited resources. Within weeks of starting work and blogging on Koji, our blogs became the top results when searching the internet.
- Understanding the relationship of package creation and Koji's method of organizing software packages in the database. This is still a learning process as Koji has just become fully functional.
Results
Currently we have a functional Koji Build system. The Koji Hub is successfully dispatching packages to the Koji builder Daemons and Kojira is creating new repositories for use with the ARM architecture. Eight VM's and the Open-RD Client are online and are listed as Ready and Enabled using the Koji command line tool and the web interface through Koji Web. Koji Web can be used to monitor all Koji operations and provide real time log information that can be used to troubleshoot problems as they arise.
Acknowledgements
- Dennis Gilmore
- Chris Tyler
Links
- Fedora Project - http://fedoraproject.org/wiki/Koji/ServerHowTo
- Paul Whalen's SBR600 Blog - http://paulfedora.wordpress.com/
- Software Build and Release 600 wiki - http://zenit.senecac.on.ca/wiki/index.php/SBR600
- Fedora Arm Secondary Architecture wiki - http://zenit.senecac.on.ca/wiki/index.php/Fedora_ARM_Secondary_Architecture
Logos
- Fedora
- Seneca
- Apache
- PostgreSQL
Images
- Open-RD Client Board - http://www.einfochips.com/marvell/images/OpenRD-Client-Zoom.png
- RPM Package Logo - http://www.linux-support.com/cms/images/stories/illutrations/rpm-package.png
