|
|
| (39 intermediate revisions by the same user not shown) |
| Line 5: |
Line 5: |
| | == Project Description == | | == Project Description == |
| | | | |
| − | The Fedora-ARM koji system uese HongKong,an x86_64 system, as the Koji hub. | + | The Fedora-ARM koji system uese IRAQ,an x86_64 system, as the Koji hub. |
| | | | |
| | The Fedora-ARM project to use ARM system as the Koji hub(this is called "Eating own dogfood" in the industry) | | The Fedora-ARM project to use ARM system as the Koji hub(this is called "Eating own dogfood" in the industry) |
| Line 23: |
Line 23: |
| | | | |
| | == Project Details == | | == Project Details == |
| − | '''First stage'''
| |
| − | * '''Koji Authentication Selection'''
| |
| | | | |
| − | Koji primarily supports Kerberos and SSL Certificate authentication. For basic koji command line access,
| + | Setting Up a Koji Build System |
| − | plain user/pass combinations are possible. However, kojiweb does not support plain user/pass authentication.
| |
| − | Furthermore, once either Kerberos or SSL Certificate authentication is enabled so that kojiweb will work,
| |
| − | the plain user/pass method will stop working entirely.
| |
| − | As such plain user/pass authentication is a stop gap measure at best unless you intend to never setup
| |
| − | a fully functional kojiweb instance.
| |
| | | | |
| − | The Kerberos credentials of the initial admin user will be necessary to bootstrap the user database.
| + | The Koji components may live on separate resources as long as all resources are able to communicate. |
| | + | To understand koji configuration is an important in project. |
| | | | |
| − | For SSL authentication, SSL certificates for the xmlrpc server, for the various koji components,
| + | '''KOJI Configuration''' |
| − | and one for the admin user will need to be setup
| |
| | | | |
| − | * 1.[http://fedoraproject.org/wiki/Koji/ServerHowTo#Koji_Authentication_Selection Setting up SSL Certificates for authentication] | + | * SSL and authentication via certificates |
| | + | * Creating a database in PostgreSQL and importing a schema |
| | + | * Working with psql |
| | + | * Apache configuration |
| | + | * Koji web and Koji hub |
| | | | |
| − | * 2.[http://fedoraproject.org/wiki/Koji/ServerHowTo#Setting_up_Kerberos_for_authentication Setting up Kerberos for authentication]
| |
| | | | |
| − | * 3.[http://fedoraproject.org/wiki/Koji/ServerHowTo#PostgreSQL_Server Setting up PostgreSQL and Create Database]
| |
| | | | |
| − | So,Release 0.1 results here [http://ycshon.blogspot.com/2010/11/sbr-600-project-release-01.html in my blog]
| + | '''Packages''' |
| | | | |
| − | '''Second stage'''
| + | On the server (koji-hub/koji-web) |
| | | | |
| − | * Koji Database | + | * httpd |
| | + | * mod_ssl |
| | + | * postgresql-server |
| | + | * mod_python (>= 3.3.1 for Kerberos authentication) |
| | | | |
| − | 1.Create Database
| + | On the builder (koji-builder) |
| | | | |
| − | yum install postgresql-server
| + | * mock |
| | + | * rpm-build |
| | + | * createrepo |
| | | | |
| − | 2. Once installed you will then need to initialize the Database with the following command:
| + | On the yum repository creation and maintenance (kojira) |
| | | | |
| − | service postgresql initdb
| + | On the Bootrapping the Koji build environment |
| | | | |
| − | 3.Start the Postgresql service with:
| + | * Importing packages and preparing Koji to run builds |
| | + | * External Repos and preparing Koji to run builds |
| | | | |
| − | service postgresql start
| + | == Project Plan == |
| | | | |
| − | 4. At this point the Postgresql server is installed and operational provided no errors were reported with the database initialized.
| + | Goals for each release: |
| | | | |
| − | The next steps is to create a user named “koji”, set up Postgresql and populate the schema:
| + | '''release 0.1 - Koji Certificates ''' |
| | | | |
| − | useradd koji
| + | '''setting up SSL Certificates for Authentication''' |
| − | passwd -d koji
| |
| − | su - postgres
| |
| − | createuser koji
| |
| − | Shall the new role be a superuser? (y/n) n
| |
| − | Shall the new role be allowed to create databases? (y/n) n
| |
| − | Shall the new role be allowed to create more new roles?(y/n) n
| |
| | | | |
| − | createdb -O koji koji
| + | * Certificate generation |
| − | logout
| + | * Generate CA |
| − | su - koji
| + | * Generate the koji component certificates and the admin certificate |
| − | psql koji koji < /usr/share/doc/koji*/docs/schema.sql
| + | * Copy certificates into ~/.koji for kojiadmin |
| − | exit
| |
| | | | |
| − | Authorize Koji-web and Koji-hub resources: In this example, Koji-web and Koji-hub are running on localhost.
| |
| | | | |
| − | /var/lib/pgsql/data/pg_hba.conf: These settings need to be valid and inline with other services configurations.
| |
| | | | |
| − | Please note, the first matching auth line is used so this line must be above any other potential matches. Add:
| + | '''release 0.2 - PostgreSQL and koji hub''' |
| | | | |
| − | host koji koji 127.0.0.1/32 trust
| + | '''release 0.2-1''' |
| − | local koji apache trust
| |
| − | local koji koji trust
| |
| | | | |
| − | To edit /var/lib/pgsql/data/pg_hba.conf
| + | '''1.PostgreSQL Server''' |
| | | | |
| − | <code># TYPE DATABASE USER CIDR-ADDRESS METHOD
| + | * Install PostgreSQL |
| − | host koji koji 127.0.0.1/32 trust
| + | * Initialize PostgreSQL DB |
| − | host koji apache 127.0.0.1/32 trust
| + | * Setup User Accounts |
| − | host koji koji ::1/128 trust
| + | * Setup PostgreSQL and populate schema |
| − | host koji apache ::1/128 trust
| + | * Authorize Koji-web and Koji-hub resources |
| − | local koji apache trust
| + | * Make auth changes live |
| − | local koji koji trust</code>
| + | * SSL Certificate authentication |
| | + | * Give yourself admin permissions |
| | | | |
| − | '''TO enable auth changes live:'''
| |
| | | | |
| − | root@localhost$ su - postgres
| |
| − | postgres@localhost$ pg_ctl reload
| |
| − | postgres@localhost$ exit
| |
| | | | |
| − | Bootstrapping the initial koji admin user into the PostgreSQL database: The initial admin user must be manually added to the user database using sql commands. Once they are added and given admin privilege, they may add additional users and change privileges of those users via the koji command line tool's administrative commands. However, if you choose to use the simple user/pass method of authentication, then any password setting/changing must be done manually via sql commands as there is no password manipulation support exposed through the koji tools.
| + | '''release 0.2-2''' |
| | | | |
| − | The sql commands you need to use vary by authentication mechanism.
| + | '''2.Koji hub''' |
| | | | |
| − | SSL Certificate authentication: there is no need for either a password or a Kerberos principal, so this will suffice:
| + | * Install koji-hub |
| | + | * Required Configuration |
| | + | * Optional Configuration |
| | + | * SELinux Configuration |
| | + | * Koji filesystem skeleton |
| | | | |
| − | koji=> insert into users (name, status, usertype) values ('admin-user-name', 0, 0);
| |
| | | | |
| − | Give yourself admin permissions
| + | * 0.3 - '''Koji Web''' |
| − | | |
| − | koji=> insert into user_perms (user_id, perm_id, creator_id) values (<id of user inserted above>, 1, <id of user inserted above>);
| |
| − | | |
| − | you can get the ID of the new user by running the query:
| |
| − | | |
| − | koji=> select * from users;
| |
| − | | |
| − | | |
| − | So,Release 0.2-1 results here [http://ycshon.blogspot.com/2010/11/sbr-600-project-release-02-1.html in my blog]
| |
| − | | |
| − | So,Release 0.3-1 results here [http://ycshon.blogspot.com/2010/11/sbr-600-project-release-02-1.html in my blog]
| |
| − | | |
| − | '''Last stage'''
| |
| − | | |
| − | '''Setup Koji Web and web''' | |
| − | | |
| − | | |
| − | | |
| − | * Koji Hub Configuration
| |
| − | | |
| − | 1. introduction
| |
| − | | |
| − | It was agreed that all parts of our Koji build system would reside on IRAQ.
| |
| − | | |
| − | After successfully installing and configuring Postgresql the next step is the Koji Hub.
| |
| − | | |
| − | In order for the Koji Hub to work Apache should be installed as well as a few additional modules.
| |
| − | | |
| − | Run the following command as root:
| |
| − | | |
| − | yum install koji-hub httpd mod_ssl mod_python
| |
| − | | |
| − | | |
| − | 2. Edit the Apache conf file
| |
| − | | |
| − | ~/etc/httpd/conf/httpd.conf and change the “MaxRequestsPerChild” to 100.
| |
| − | | |
| − | On IRAQ these setting were already in place as Apache was running and configured.
| |
| − | | |
| − | Next edit the ‘/etc/koji-hub/hub.conf’ file and add the following lines:
| |
| − | | |
| − | DBName = koji
| |
| − | DBUser = koji
| |
| − | DBHost = localhost
| |
| − | KojiDir = /mnt/koji
| |
| − | LoginCreatesUser = On
| |
| − | KojiWebURL = http://iraq.proximity.on.ca/koji
| |
| − | | |
| − | | |
| − | 3. Using SSL for authentication
| |
| − | | |
| − | Add this line.
| |
| − | | |
| − | DNUsernameComponent = CN
| |
| − | ProxyDNs = "/C=CA/ST=Ontario/O=Seneca CDOT/OU=/CN=kojiweb/emailAddress="
| |
| − | | |
| − | And in the /etc/httpd/conf.d/kojihub.conf uncomment the following lines:
| |
| − | | |
| − | <Location /kojihub>
| |
| − | SSLOptions +StdEnvVars
| |
| − | </Location>
| |
| − | | |
| − | 4.Using [http://zenit.senecac.on.ca/wiki/index.php/Fedora_Arm_Secondary_Architecture/Koji_Certificates the Koji certificates]
| |
| − | | |
| − | ,and add the following lines to ‘/etc/httpd/conf.d/ssl.conf’, under the section ‘VirtualHost _default_:443′:
| |
| − | | |
| − | SSLCertificateFile /etc/pki/koji/certs/kojihub.crt
| |
| − | SSLCertificateKeyFile /etc/pki/koji/certs/kojihub.key
| |
| − | SSLCertificateChainFile /etc/pki/koji/koji_ca_cert.crt
| |
| − | SSLCACertificateFile /etc/pki/koji/koji_ca_cert.crt
| |
| − | SSLVerifyClient require
| |
| − | SSLVerifyDepth 10
| |
| − | | |
| − | | |
| − | 5. SE Linux
| |
| − | In the IRAQ. In order to allow Apache to connect to the Postgresql database run the following command as root:
| |
| − | | |
| − | setsebool -P httpd_can_network_connect_db 1
| |
| − | | |
| − | 6.A skeleton filesystem
| |
| − | | |
| − | To allow Koji to work, a skeleton filesystem needs to be created and the ownership
| |
| − | | |
| − | changed so Apache can write to it as required. The following commands were executed:
| |
| − | | |
| − |
| |
| − | mkdir -p /mnt/koji/{packages,repos,work,scratch}
| |
| − | chown -R apache.apache /mnt/koji
| |
| − | | |
| − | Then edited the '/etc/koji.conf' file and changed the following lines:
| |
| − | | |
| − | ;url of XMLRPC server
| |
| − | server = http://iraq.proximity.on.ca/kojihub
| |
| − | ;url of web interface
| |
| − | weburl = http://iraq.proximity.on.ca/koji
| |
| − | ;url of package download site
| |
| − | pkgurl = http://iraq.proximity.on.ca/packages
| |
| − | ;path to the koji top directory
| |
| − | topdir = /mnt/koji
| |
| − | ;configuration for SSL athentication
| |
| − | ;client certificate
| |
| − | cert = ~/.koji/client.crt
| |
| − | ;certificate of the CA that issued the client certificate
| |
| − | ca = ~/.koji/clientca.crt
| |
| − | ;certificate of the CA that issued the HTTP server certificate
| |
| − | serverca = ~/.koji/serverca.crt
| |
| − |
| |
| − | | |
| − | 7. Build accounts
| |
| − | The final step is the addition of the user and builder accounts.
| |
| − | | |
| − | First add the kojira account and grant repo privileges with the following command( this should be done before running kojira for the first time) :
| |
| − | | |
| − | su - kojiadmin
| |
| − | koji add-user kojira
| |
| − | koji grant-permission repo kojira
| |
| − | | |
| − | Then add as many builders as required using the following commands editing where required (this should also be done prior to running kojid on each host):
| |
| − | | |
| − | koji add-host arm-001-001 arm
| |
| − | koji add-host arm-001-002 arm
| |
| − | koji add-host arm-001-003 arm
| |
| − | | |
| − | So,Release 0.2-2 results here [http://ycshon.blogspot.com/2010/11/sbr-600-project-release-02-2.html in my blog]
| |
| − | | |
| − | * Kojiweb configuration
| |
| − | | |
| − | 1.Introduction
| |
| − | Koji-web is a set of scripts that run in mod_python and use the Cheetah templating engine to provide an web interface to Koji.
| |
| − | koji-web exposes a lot of information and also provides a means for certain operations, such as cancelling builds.
| |
| | | | |
| − | Run the following command as root:
| + | '''release 0.3-1''' |
| | | | |
| − | root@localhost$ yum install koji-web mod_ssl
| + | '''1. kojiweb ''' |
| | | | |
| − | 2. Edit the file /etc/httpd/conf.d/kojiweb.conf:
| + | * Install Koji-Web |
| | + | * Required Configuration |
| | + | * Optional Configuration |
| | | | |
| − | PythonOption KojiHubURL http://hub.example.com/kojihub
| + | '''2. Koji Builder''' |
| − | PythonOption KojiWebURL http://www.example.com/koji
| |
| − | PythonOption KojiPackagesURL http://server.example.com/mnt/koji/packages
| |
| − | PythonOption WebCert /etc/pki/koji/kojiweb.pem
| |
| − | PythonOption ClientCA /etc/pki/koji/koji_ca_cert.crt
| |
| − | PythonOption KojiHubCA /etc/pki/koji/koji_ca_cert.crt
| |
| − | PythonOption LoginTimeout 72
| |
| − | PythonOption Secret CHANGE_ME
| |
| | | | |
| − | 3. Optional Configuration
| + | * Install kojid |
| | + | * Required Configuration |
| | + | * Optional Configuration (SSL certificates) |
| | + | * Add the host entry for the koji builder to the database |
| | + | * Add the host to the createrepo channel |
| | + | * A note on capacity |
| | + | * Start Kojid |
| | | | |
| − | /etc/httpd/conf.d/kojiweb.conf:
| + | '''3.kojira''' |
| | | | |
| − | If using Kerberos, these settings need to be valid and inline with other services configurations.
| + | * Install kojira |
| | + | * Required Configuration |
| | + | * Optional Configuration |
| | + | * Add the user entry for the kojira user |
| | + | * Start Kojira |
| | | | |
| − | <pre><Location /koji/login>
| |
| − | AuthType Kerberos
| |
| − | AuthName "Koji Web UI"
| |
| − | KrbMethodNegotiate on
| |
| − | KrbMethodK5Passwd off
| |
| − | KrbServiceName HTTP
| |
| − | KrbAuthRealm EXAMPLE.COM
| |
| − | Krb5Keytab /etc/httpd.keytab
| |
| − | KrbSaveCredentials off
| |
| − | Require valid-user
| |
| − | ErrorDocument 401 /koji-static/errors/unauthorized.html
| |
| − | </Location></pre>
| |
| | | | |
| − | /etc/httpd/conf.d/kojiweb.conf:
| |
| | | | |
| − | If using SSL auth, these settings need to be valid and inline with other services configurations.
| + | '''release 0.3-2''' |
| − | <pre><Location /koji/login>
| |
| − | SSLOptions +StdEnvVars
| |
| − | </Location></pre>
| |
| | | | |
| − | /etc/httpd/conf.d/ssl.conf: Add the needed SSL options for apache.
| + | '''2. Test kojiweb''' |
| | | | |
| − | <pre>SSLVerifyClient require
| + | * User account |
| − | SSLVerifyDepth 10</pre>
| + | * Build packages |
| | | | |
| − | If you are using SSL authentication, the "PythonOption WebCert" file above must contain both the public and private key.
| |
| | | | |
| − | Web interface now operational
| |
| | | | |
| − | At this point you should be able to point your web browser at the kojiweb URL and be presented with the koji interface. Many operations should work in read only mode at this point, and any configured users should be able to log in.
| |
| − |
| |
| − | == Project Plan ==
| |
| − |
| |
| − | Goals for each release:
| |
| − | * 0.1 - '''Koji Certificates '''
| |
| − | Koji Hub setup - Certificates/security
| |
| − | * 0.2 - '''Koji Database with PostgreSQL'''
| |
| − | To insert user permision to SQL server
| |
| − | To setup PostgreSQL for use with Koji
| |
| − | * 0.3 - '''Koji Web'''
| |
| − | To setup Koji web and Hub
| |
| | == Project News == | | == Project News == |
| | | | |
| Line 316: |
Line 151: |
| | 19,November I set up Koji Database. | | 19,November I set up Koji Database. |
| | | | |
| − | 26,November I set up Koji hub Configuration | + | 26,November Setting up Postgresql server for Koji part 1 |
| | | | |
| | 6,Dec Setting up Postgresql server for Koji part two | | 6,Dec Setting up Postgresql server for Koji part two |
| | + | |
| | + | 16,Dev setting up Koji hub and Koji web, |
| | == Resources == | | == Resources == |
| | | | |
| Line 330: |
Line 167: |
| | | | |
| | [http://fedoraproject.org/wiki/Koji/ServerHowTo#PostgreSQL_Server PostgreSQL_Server] | | [http://fedoraproject.org/wiki/Koji/ServerHowTo#PostgreSQL_Server PostgreSQL_Server] |
| | + | |
| | + | [http://fedoraproject.org/wiki/Koji/ServerBootstrap importing packages and preparing Koji to run builds] |
| | + | |
| | + | [http://fedoraproject.org/wiki/Koji/ExternalRepoServerBootstrap External Repos and preparing Koji to run builds] |
Project Name
Fedora-ARM Dogfood -Koji Hub
Project Description
The Fedora-ARM koji system uese IRAQ,an x86_64 system, as the Koji hub.
The Fedora-ARM project to use ARM system as the Koji hub(this is called "Eating own dogfood" in the industry)
The project involves configuring the OpenRD-Client system as koji hub.
The Fedora ARM Koji system is running. It is currently building F13 under the supervision of Whalen and Chris Tyler. Koji have 22 hardware ARM builders.
I am in the progress of building Fedora 13 under IRAQ.
Project Leader(s)
Young Chol Shon
Project Contributor(s)
No one contributed to this project
Project Details
Setting Up a Koji Build System
The Koji components may live on separate resources as long as all resources are able to communicate.
To understand koji configuration is an important in project.
KOJI Configuration
- SSL and authentication via certificates
- Creating a database in PostgreSQL and importing a schema
- Working with psql
- Apache configuration
- Koji web and Koji hub
Packages
On the server (koji-hub/koji-web)
- httpd
- mod_ssl
- postgresql-server
- mod_python (>= 3.3.1 for Kerberos authentication)
On the builder (koji-builder)
- mock
- rpm-build
- createrepo
On the yum repository creation and maintenance (kojira)
On the Bootrapping the Koji build environment
- Importing packages and preparing Koji to run builds
- External Repos and preparing Koji to run builds
Project Plan
Goals for each release:
release 0.1 - Koji Certificates
setting up SSL Certificates for Authentication
- Certificate generation
- Generate CA
- Generate the koji component certificates and the admin certificate
- Copy certificates into ~/.koji for kojiadmin
release 0.2 - PostgreSQL and koji hub
release 0.2-1
1.PostgreSQL Server
- Install PostgreSQL
- Initialize PostgreSQL DB
- Setup User Accounts
- Setup PostgreSQL and populate schema
- Authorize Koji-web and Koji-hub resources
- Make auth changes live
- SSL Certificate authentication
- Give yourself admin permissions
release 0.2-2
2.Koji hub
- Install koji-hub
- Required Configuration
- Optional Configuration
- SELinux Configuration
- Koji filesystem skeleton
release 0.3-1
1. kojiweb
- Install Koji-Web
- Required Configuration
- Optional Configuration
2. Koji Builder
- Install kojid
- Required Configuration
- Optional Configuration (SSL certificates)
- Add the host entry for the koji builder to the database
- Add the host to the createrepo channel
- A note on capacity
- Start Kojid
3.kojira
- Install kojira
- Required Configuration
- Optional Configuration
- Add the user entry for the kojira user
- Start Kojira
release 0.3-2
2. Test kojiweb
- User account
- Build packages
Project News
1,November I will do Project Plan 0.1- I set up Koji Certificates
5,November I did build and create Koji Certifacation on IRAQ server.
19,November I set up Koji Database.
26,November Setting up Postgresql server for Koji part 1
6,Dec Setting up Postgresql server for Koji part two
16,Dev setting up Koji hub and Koji web,
Resources
How to get koji certificate?
Configuration of CDOT_Development system
How To Setting up and Using Koji on Fedora
PostgreSQL9.1 devel Document
PostgreSQL_Server
importing packages and preparing Koji to run builds
External Repos and preparing Koji to run builds
