Difference between revisions of "Fedora-ARM,Dogfood - koji Hub"

From CDOT Wiki
Jump to: navigation, search
 
(71 intermediate revisions by the same user not shown)
Line 5: Line 5:
 
== Project Description ==
 
== Project Description ==
  
−
The Fedora-ARM koji system uese HongKong,an x86_64 system, as the Koji hub.
+
The Fedora-ARM koji system uese IRAQ,an x86_64 system, as the Koji hub.
  
 
The Fedora-ARM project to use ARM system as the Koji hub(this is called "Eating own dogfood" in the industry)
 
The Fedora-ARM project to use ARM system as the Koji hub(this is called "Eating own dogfood" in the industry)
Line 23: Line 23:
  
 
== Project Details ==
 
== Project Details ==
−
'''First stage'''
 
−
* '''Koji Authentication Selection'''
 
  
−
    Koji primarily supports Kerberos and SSL Certificate authentication. For basic koji command line access,   
+
Setting Up a Koji Build System
−
    plain user/pass combinations are possible.  However, kojiweb does not support plain user/pass authentication.
 
−
    Furthermore, once either Kerberos or SSL Certificate authentication is enabled so that kojiweb will work,
 
−
    the plain user/pass method will stop working entirely.
 
−
    As such plain user/pass authentication is a stop gap measure at best unless you intend to never setup
 
−
    a fully functional kojiweb instance.
 
  
−
    The Kerberos credentials of the initial admin user will be necessary to bootstrap the user database.
+
The Koji components may live on separate resources as long as all resources are able to communicate.
 +
To understand koji configuration is an important in project.
  
−
    For SSL authentication, SSL certificates for the xmlrpc server, for the various koji components,
+
'''KOJI Configuration'''
−
    and one for the admin user will need to be setup
 
  
−
    1.[http://fedoraproject.org/wiki/Koji/ServerHowTo#Koji_Authentication_Selection  Setting up SSL Certificates for authentication]
+
* SSL and authentication via certificates
 +
* Creating a database in PostgreSQL and importing a schema
 +
* Working with psql
 +
* Apache configuration
 +
* Koji web and Koji hub
  
−
    2.[http://fedoraproject.org/wiki/Koji/ServerHowTo#Setting_up_Kerberos_for_authentication Setting up Kerberos for authentication]
 
  
−
So,Release 0.1 results here [http://ycshon.blogspot.com/2010/11/sbr-600-project-release-01.html  in my blog]
 
  
−
'''Second stage'''
+
'''Packages'''
  
−
* Koji Database
+
On the server (koji-hub/koji-web)
  
−
1.Create Database
+
* httpd
 +
* mod_ssl
 +
* postgresql-server
 +
* mod_python (>= 3.3.1 for Kerberos authentication)
  
−
yum install postgresql-server
+
On the builder (koji-builder)
  
−
2. Once installed you will then need to initialize the Database with the following command:
+
* mock
 +
* rpm-build
 +
* createrepo
  
−
service postgresql initdb
+
On the yum repository creation and maintenance (kojira)
  
−
3.Start the Postgresql service with:
+
On the Bootrapping the Koji build environment
  
−
service postgresql start
+
* Importing packages and preparing Koji to run builds
 +
* External Repos and preparing Koji to run builds
  
−
4. At this point the Postgresql server is installed and operational provided no errors were reported with the database initialized.
+
== Project Plan ==
  
−
The next steps is to create a user named “koji”, set up Postgresql and populate the schema:
+
Goals for each release:
  
−
useradd koji
+
'''release 0.1 - Koji Certificates '''
−
passwd -d koji
 
−
su - postgres
 
−
createuser koji
 
−
Shall the new role be a superuser? (y/n) n
 
−
Shall the new role be allowed to create databases? (y/n) n
 
−
Shall the new role be allowed to create more new roles?(y/n) n
 
  
−
createdb -O koji koji
+
'''setting up SSL Certificates for Authentication'''
−
logout
 
−
su - koji
 
−
psql koji koji < /usr/share/doc/koji*/docs/schema.sql
 
−
exit
 
  
−
* Koji Hub Configuration
+
* Certificate generation
 +
* Generate CA
 +
* Generate the koji component certificates and the admin certificate
 +
* Copy certificates into ~/.koji for kojiadmin
  
−
1. introduction
 
  
−
It was agreed that all parts of our Koji build system would reside on IRAQ.
 
  
−
After successfully installing and configuring Postgresql the next step is the Koji Hub.
+
'''release 0.2 - PostgreSQL and koji hub'''
  
−
In order for the Koji Hub to work Apache should be installed as well as a few additional modules.
+
'''release 0.2-1'''
  
−
Run the following command as root:
+
'''1.PostgreSQL Server'''
  
 +
* Install PostgreSQL
 +
* Initialize PostgreSQL DB
 +
* Setup User Accounts
 +
* Setup PostgreSQL and populate schema
 +
* Authorize Koji-web and Koji-hub resources
 +
* Make auth changes live
 +
* SSL Certificate authentication
 +
* Give yourself admin permissions
  
−
{| class="wikitable" border="1"
+
 
−
| yum install koji-hub httpd mod_ssl mod_python
+
 
−
|-
+
'''release 0.2-2'''
−
|
+
 
−
|-
+
'''2.Koji hub'''
−
|
+
 
−
|-
+
* Install koji-hub
−
| Then edit the Apache conf file – ‘/etc/httpd/conf/httpd.conf’ and change the “MaxRequestsPerChild” to 100.
+
* Required Configuration
−
|-
+
* Optional Configuration
−
| On IRAQ these setting were already in place as Apache was running and configured.
+
* SELinux Configuration
−
|-
+
* Koji filesystem skeleton
−
|
+
 
−
|-
+
 
−
| Next edit the ‘/etc/koji-hub/hub.conf’ file and add the following lines:
+
* 0.3 - '''Koji Web'''
−
|-
+
 
−
| DBName = koji
+
'''release 0.3-1'''
−
|-
+
 
−
| DBUser = koji
+
'''1. kojiweb '''
−
|-
+
 
−
| DBHost = localhost
+
* Install Koji-Web
−
|-
+
* Required Configuration
−
| KojiDir = /mnt/koji
+
* Optional Configuration
−
|-
+
 
−
| LoginCreatesUser = On
+
'''2. Koji Builder'''
−
|-
+
 
−
| KojiWebURL = http://iraq.proximity.on.ca/koji
+
* Install kojid
−
|-
+
* Required Configuration
−
| Since we are using SSL for authentication, also add
+
* Optional Configuration (SSL certificates)
−
|-
+
* Add the host entry for the koji builder to the database
−
|
+
* Add the host to the createrepo channel
−
|-
+
* A note on capacity
−
| DNUsernameComponent = CN
+
* Start Kojid
−
|-
+
 
−
| ProxyDNs = "/C=CA/ST=Ontario/O=Seneca CDOT/OU=/CN=kojiweb/emailAddress="
+
'''3.kojira'''
−
|-
+
 
−
|
+
* Install kojira
−
|-
+
* Required Configuration
−
| And in the ‘/etc/httpd/conf.d/kojihub.conf’ uncomment the following lines:
+
* Optional Configuration
−
|-
+
* Add the user entry for the kojira user
−
|
+
* Start Kojira
−
|-
+
 
−
| <Location /kojihub>
+
 
−
|-
+
 
−
| SSLOptions +StdEnvVars
+
'''release 0.3-2'''  
−
|-
+
 
−
| </Location>
+
'''2. Test kojiweb'''
−
|-
+
 
−
|
+
* User account
−
|-
+
* Build packages
−
| Using the Koji certificates, we need to add the following lines to ‘/etc/httpd/conf.d/ssl.conf’, under the section ‘VirtualHost _default_:443′:
+
 
−
|-
+
 
−
|
+
 
−
|-
+
== Project News ==
−
| SSLCertificateFile      /etc/pki/koji/certs/kojihub.crt
+
 
−
|-
+
1,November I will do Project Plan 0.1- I set up Koji Certificates
−
| SSLCertificateKeyFile  /etc/pki/koji/certs/kojihub.key
+
 
−
|-
+
5,November I did build and create Koji Certifacation on IRAQ server.
−
| SSLCertificateChainFile /etc/pki/koji/koji_ca_cert.crt
+
 
−
|-
+
19,November I set up Koji Database.
−
| SSLCACertificateFile    /etc/pki/koji/koji_ca_cert.crt
+
 
−
|-
+
26,November Setting up Postgresql server for Koji part 1
−
| SSLVerifyClient        require
+
 
−
|-
+
6,Dec Setting up Postgresql server for Koji part two
−
| SSLVerifyDepth          10
+
 
−
|-
+
16,Dev setting up Koji hub and Koji web,
−
| Even though SE Linux is not currently in use on IRAQ,
+
== Resources ==
−
|-
+
 
−
| it may be in the future. In order to allow Apache to connect to the Postgresql database run the following command as root:
+
[http://zenit.senecac.on.ca/wiki/index.php/Fedora_Arm_Secondary_Architecture/Koji_Certificates How to get koji certificate?]
−
|-
+
 
−
|
+
[http://zenit.senecac.on.ca/wiki/index.php/CDOT_Development_Systems Configuration of CDOT_Development system]
−
|-
+
 
−
| setsebool -P httpd_can_network_connect_db 1
+
[http://fedoraproject.org/wiki/Koji How To Setting up and Using Koji on Fedora]
−
|-
+
 
−
| To allow Koji to work, a skeleton filesystem needs to be created and the ownership
+
[http://developer.postgresql.org/pgdocs/postgres/index.html PostgreSQL9.1 devel Document]
−
|-
+
 
−
| changed so Apache can write to it as required. The following commands were executed:
+
[http://fedoraproject.org/wiki/Koji/ServerHowTo#PostgreSQL_Server PostgreSQL_Server]
−
|-
+
 
−
|
+
[http://fedoraproject.org/wiki/Koji/ServerBootstrap importing packages and preparing Koji to run builds]
−
|-
+
 
−
| mkdir -p /mnt/koji/{packages,repos,work,scratch}
+
[http://fedoraproject.org/wiki/Koji/ExternalRepoServerBootstrap External Repos and preparing Koji to run builds]
−
|-
 
−
| chown -R apache.apache /mnt/koji
 
−
|-
 
−
| Then edited the '/etc/koji.conf' file and changed the following lines:
 
−
|-
 
−
|
 
−
|-
 
−
| ;url of XMLRPC server
 
−
|-
 
−
| server = http://iraqong.proximity.on.ca/koji
 
−
|-
 
−
| ;url of package download site
 
−
|-
 
−
| pkgurl = http://iraq.proximity.on.ca/packages
 
−
|-
 
−
| ;path to the koji top directory
 
−
|-
 
−
| topdir = /mnt/koji
 
−
|-
 
−
| ;configuration for SSL athentication
 
−
|-
 
−
| ;client certificate
 
−
|-
 
−
| cert = ~/.koji/client.crt
 
−
|-
 
−
| ;certificate of the CA that issued the client certificate
 
−
|-
 
−
| ca = ~/.koji/clientca.crt
 
−
|-
 
−
| ;certificate of the CA that issued the HTTP server certificate
 
−
|-
 
−
| serverca = ~/.koji/serverca.crt
 
−
|-
 
−
| After this is competed, the final step is the addition of the user and builder accounts. First add the kojira account and grant repo privileges with the following command( this should be done before running kojira for the first time) :
 
−
|-
 
−
|
 
−
|-
 
−
| su - kojiadmin
 
−
|-
 
−
| koji add-user kojira
 
−
|-
 
−
| koji grant-permission repo kojira
 
−
|-
 
−
|
 
−
|-
 
−
| Then add as many builders as required using the following commands editing where required (this should also be done prior to running kojid on each host):
 
−
|-
 
−
|
 
−
|-
 
−
| koji add-host arm-001-001 arm
 
−
|-
 
−
| koji add-host arm-001-002 arm
 
−
|-
 
−
| koji add-host arm-001-003 arm
 
−
|-
 
−
|
 
−
|-
 
−
| == Project Plan ==
 
−
|-
 
−
|
 
−
|-
 
−
| Goals for each release:
 
−
|-
 
−
| * 0.1 - '''Koji Certificates '''
 
−
|-
 
−
| Koji Hub setup - Certificates/security
 
−
|-
 
−
| * 0.2 - '''Koji Hub setup and Koji Database'''
 
−
|-
 
−
| Koji Hub Setup- Configuration
 
−
|-
 
−
| To setup PostgreSQL for use with Koji
 
−
|-
 
−
| * 0.3 - '''Koji Web'''
 
−
|-
 
−
| Build software with koji hub
 
−
|-
 
−
| == Project News ==
 
−
|-
 
−
|
 
−
|-
 
−
| 1,November I will do Project Plan 0.1- I set up Koji Certificates
 
−
|-
 
−
|
 
−
|-
 
−
| 5,November I did build and create Koji Certifacation on IRAQ server.
 
−
|-
 
−
|
 
−
|-
 
−
| 19,November I set up Koji Database.
 
−
|-
 
−
|
 
−
|-
 
−
| 26,November I set up Koji hub Configuration
 
−
|-
 
−
|
 
−
|-
 
−
| == Resources ==
 
−
|-
 
−
|
 
−
|-
 
−
| [http://zenit.senecac.on.ca/wiki/index.php/Fedora_Arm_Secondary_Architecture/Koji_Certificates How to get koji certificate?]
 
−
|-
 
−
|
 
−
|-
 
−
| [http://zenit.senecac.on.ca/wiki/index.php/CDOT_Development_Systems Configuration of CDOT_Development system]
 
−
|-
 
−
|
 
−
|-
 
−
| [http://fedoraproject.org/wiki/Koji How To Setting up and Using Koji on Fedora]
 
−
|-
 
−
|
 
−
|-
 
−
| [http://developer.postgresql.org/pgdocs/postgres/index.html PostgreSQL9.1 devel Document]
 
−
|-
 
−
|
 
−
|-
 
−
| [http://fedoraproject.org/wiki/Koji/ServerHowTo#PostgreSQL_Server PostgreSQL_Server]
 
−
|-
 
−
|
 
−
|}
 

Latest revision as of 08:17, 13 January 2011

Project Name

Fedora-ARM Dogfood -Koji Hub

Project Description

The Fedora-ARM koji system uese IRAQ,an x86_64 system, as the Koji hub.

The Fedora-ARM project to use ARM system as the Koji hub(this is called "Eating own dogfood" in the industry)

The project involves configuring the OpenRD-Client system as koji hub.

The Fedora ARM Koji system is running. It is currently building F13 under the supervision of Whalen and Chris Tyler. Koji have 22 hardware ARM builders. I am in the progress of building Fedora 13 under IRAQ.

Project Leader(s)

Young Chol Shon

Project Contributor(s)

No one contributed to this project

Project Details

Setting Up a Koji Build System

The Koji components may live on separate resources as long as all resources are able to communicate. To understand koji configuration is an important in project.

KOJI Configuration

  • SSL and authentication via certificates
  • Creating a database in PostgreSQL and importing a schema
  • Working with psql
  • Apache configuration
  • Koji web and Koji hub


Packages

On the server (koji-hub/koji-web)

  • httpd
  • mod_ssl
  • postgresql-server
  • mod_python (>= 3.3.1 for Kerberos authentication)

On the builder (koji-builder)

  • mock
  • rpm-build
  • createrepo

On the yum repository creation and maintenance (kojira)

On the Bootrapping the Koji build environment

  • Importing packages and preparing Koji to run builds
  • External Repos and preparing Koji to run builds

Project Plan

Goals for each release:

release 0.1 - Koji Certificates

setting up SSL Certificates for Authentication

  • Certificate generation
  • Generate CA
  • Generate the koji component certificates and the admin certificate
  • Copy certificates into ~/.koji for kojiadmin


release 0.2 - PostgreSQL and koji hub

release 0.2-1

1.PostgreSQL Server

  • Install PostgreSQL
  • Initialize PostgreSQL DB
  • Setup User Accounts
  • Setup PostgreSQL and populate schema
  • Authorize Koji-web and Koji-hub resources
  • Make auth changes live
  • SSL Certificate authentication
  • Give yourself admin permissions


release 0.2-2

2.Koji hub

  • Install koji-hub
  • Required Configuration
  • Optional Configuration
  • SELinux Configuration
  • Koji filesystem skeleton


  • 0.3 - Koji Web

release 0.3-1

1. kojiweb

  • Install Koji-Web
  • Required Configuration
  • Optional Configuration

2. Koji Builder

  • Install kojid
  • Required Configuration
  • Optional Configuration (SSL certificates)
  • Add the host entry for the koji builder to the database
  • Add the host to the createrepo channel
  • A note on capacity
  • Start Kojid

3.kojira

  • Install kojira
  • Required Configuration
  • Optional Configuration
  • Add the user entry for the kojira user
  • Start Kojira


release 0.3-2

2. Test kojiweb

  • User account
  • Build packages


Project News

1,November I will do Project Plan 0.1- I set up Koji Certificates

5,November I did build and create Koji Certifacation on IRAQ server.

19,November I set up Koji Database.

26,November Setting up Postgresql server for Koji part 1

6,Dec Setting up Postgresql server for Koji part two

16,Dev setting up Koji hub and Koji web,

Resources

How to get koji certificate?

Configuration of CDOT_Development system

How To Setting up and Using Koji on Fedora

PostgreSQL9.1 devel Document

PostgreSQL_Server

importing packages and preparing Koji to run builds

External Repos and preparing Koji to run builds

Retrieved from "https://wiki.cdot.senecapolytechnic.ca/w/index.php?title=Fedora-ARM,Dogfood_-_koji_Hub&oldid=54579"