Difference between revisions of "Fedora Arm Secondary Architecture/Koji Builders"

From CDOT Wiki
Jump to: navigation, search
(Created page with '= Koji Daemon - Builder = Kojid is the build daemon that runs on each of the build machines. Its primary responsibility is polling for incoming build requests and handling them ...')
 
(→‎Start Kojid)
 
(29 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
= Koji Daemon - Builder =
+
[[Category:Fedora ARM Secondary Architecture]][[Category:Winter 2010 SBR600]]
 +
= Goal =
 +
 
 +
To configure a Koji builder system (either VM or physical hardware).
  
 
Kojid is the build daemon that runs on each of the build machines. Its primary responsibility is polling for incoming build requests and handling them accordingly. Koji also has support for tasks other than building. Creating install images is one example. kojid is responsible for handling these tasks as well. kojid uses mock for building. It also creates a fresh buildroot for every build. kojid is written in Python and communicates with koji-hub via XML-RPC.
 
Kojid is the build daemon that runs on each of the build machines. Its primary responsibility is polling for incoming build requests and handling them accordingly. Koji also has support for tasks other than building. Creating install images is one example. kojid is responsible for handling these tasks as well. kojid uses mock for building. It also creates a fresh buildroot for every build. kojid is written in Python and communicates with koji-hub via XML-RPC.
  
−
'''Configuration Files:'''
+
= Configuration Files =
  
 
* /etc/kojid/kojid.conf - Koji Daemon Configuration
 
* /etc/kojid/kojid.conf - Koji Daemon Configuration
 
* /etc/sysconfig/kojid - Koji Daemon Switches
 
* /etc/sysconfig/kojid - Koji Daemon Switches
  
−
'''Install kojid:'''
+
= Steps to Perform =
−
<pre>
+
 
−
root@localhost$ yum install koji-builder
+
== Storage Space ==
−
</pre>
+
 
 +
Ensure that there is a good amount of space on the ARM system for mock (<code>/var/lib/mock</code> and <code>/var/cache/mock</code>) before proceeding. You will want a minimum of about 8GB.
  
−
== Required Configuration ==
+
On the ARM VMs, you may need to increase the VM size. Stop the VM, add additional storage, and then restart the VM:
  
−
/etc/kojid/kojid.conf:
+
'''On the VM:'''
−
''This needs to point at your koji-hub.''
+
poweroff
−
<pre>
+
 
−
; The URL for the xmlrpc server
+
Allow 30 seconds for system shutdown.
−
server=http://hub.example.com/kojihub
+
 
 +
'''On HongKong:'''
 +
virsh destroy ''domain''
 +
dd if=/dev/zero bs=1M count=8192 >>/var/lib/libvirt/images/arm/''domain''.img
 +
fsck -f /var/lib/libvirt/images/arm/''domain''.img
 +
resize2fs /var/lib/libvirt/images/arm/''domain''.img
 +
virsh start ''domain''
  
−
; the username has to be the same as what you used with add-host
+
Where ''domain'' is the VM name, such as <code>arm-001-999</code>.
−
; in this example follow as below
 
−
user = kojibuilder1.example.com
 
−
</pre>
 
−
''This item may be changed, but may not be the same as KojiDir on the kojihub.conf file (although it can be something under KojiDir, just not the same as KojiDir)''
 
−
<pre>
 
−
; The directory root for temporary storage
 
−
workdir=/tmp/koji
 
−
</pre>
 
  
−
== SSL certificates For Authentication ==
+
If using a physical ARM system, ensure that <code>/var/lib/mock</code> and <code>/var/cache/mock</code> have sufficient space, perhaps by symlinking them to subdirectories of the mountpoint of an external disk, NFS share, or iSCSI block device.
  
−
/etc/kojid/kojid.conf:
+
== Install kojid ==
−
''If using SSL, these settings need to be valid.''
+
yum install koji-builder
−
<pre>
 
−
;client certificate
 
−
; This should reference the builder certificate we created above, for
 
−
; kojibuilder1.example.com
 
−
cert = /etc/kojid/arm-001-008.pem
 
  
−
;certificate of the CA that issued the client certificate
+
== NFS Configuration ==
−
ca = /etc/kojid/koji_ca_cert.crt
 
  
−
;certificate of the CA that issued the HTTP server certificate
+
Perform the steps outlined on the [[Fedora_Arm_Secondary_Architecture/NFS_Configuration|NFS Configuration]] page, including adding entries to the /etc/hosts file.
−
serverca = /etc/kojid/koji_ca_cert.crt
 
  
−
</pre>
+
== Koji Configuration ==
  
−
==== Add the host entry for the koji builder to the database ====
+
/etc/kojid/kojid.conf:
−
Make sure you do this before you start kojid for the first time,
+
''This needs to point at your koji-hub.''
−
or you'll need to manually remove entries from the sessions and users
 
−
table before it can be run successfully.
 
−
<pre>
 
−
kojiadmin@localhost$ koji add-host kojibuilder1.example.com i386 x86_64
 
−
</pre>
 
  
−
==== Add the host to the createrepo channel ====
+
; The URL for the xmlrpc server
−
Channels are a way to control which builders process which tasks. By default
+
server=http://hongkong/kojihub
−
hosts are added to the ''default'' channel. At least some build hosts also needs to be added
+
−
to the ''createrepo'' channel so there will be someone to process repo creation tasks initiated by kojira.
+
; the username has to be the same as what you used with add-host
 +
; in this example follow as below
 +
user = arm-001-001 ;Matches your VM name
 +
 +
; The URL for the packages tree
 +
pkgurl=http://hongkong/yum/arm_built/base/12/arm/
 +
   
 +
; ''This item may be changed, but may not be the same as KojiDir on the kojihub.conf file (although it can be something under KojiDir, just not the same as KojiDir)''
 +
  ; The directory root for temporary storage
 +
workdir=/tmp/koji
  
−
<pre>
+
==== SSL Certificates For Authentication ====
−
kojiadmin@localhost$ koji add-host-to-channel kojibuilder1.example.com createrepo
 
−
</pre>
 
  
−
==== A note on capacity ====
+
Copy the appropriate certificate(s) from <code>HongKong:/etc/pki/koji/certs/</code>
−
The default capacity of a host added to the host database is 2.  This means that once the load average on that machine exceeds 2, kojid will not accept any additional tasks.  This is separate from the maxjobs item in the configuration file.  Before kojid will accept a job, it must pass both the test to ensure the load average is below capacity and that the current number of jobs it is already processing is less than maxjobs.  However, in today's modern age of quad core and higher CPUs, a load average of 2 is generally insufficient to fully utilize hardware.  As there is not an option to set the capacity of the host via the command line tools, it must be done manually in psql.
 
  
−
<pre>
+
/etc/kojid/kojid.conf:
−
koji@localhost$ psql koji
 
−
koji=# select (id, name, capacity) from host;
 
−
              row             
 
−
--------------------------------
 
−
(1,kojibuilder1.example.com,2)
 
−
(2,kojibuilder2.example.com,2)
 
−
(2 rows)
 
  
−
koji=# update host set capacity = 16 where id = 1;
+
  ;client certificate
−
UPDATE 1
+
; This should reference the builder certificate we created above, for
−
koji=#
+
; kojibuilder1.example.com
−
</pre>
+
cert = /etc/kojid/arm-001-008.pem
 +
 +
;certificate of the CA that issued the client certificate
 +
ca = /etc/kojid/koji_ca_cert.crt
 +
 +
;certificate of the CA that issued the HTTP server certificate
 +
serverca = /etc/kojid/koji_ca_cert.crt
  
 
==== Start Kojid ====
 
==== Start Kojid ====
−
<pre>
+
{{Admon/important|Koji Host Entry|Before starting Kojid, make sure that the entry for your builder is in the database. The command to do this (as ''kojiadmin'' on HongKong) is <code>koji add-host ''hostname'' armv5tel</code>}}
−
root@localhost$ /sbin/service kojid start
+
 
−
</pre>
+
After, start Kojid and configure it to start at each boot:
−
Check /var/log/kojid.log to verify that kojid has started successfully.
+
service kojid start
 +
chkconfig kojid on
 +
 
 +
== Testing ==
 +
Check <code>/var/log/kojid.log</code> on the builder to verify that kojid has started successfully.
 +
 
 +
Finally, check the [http://hongkong.proximity.on.ca/koji/hosts Koji hosts web display] on HongKong to see that the host is working properly -- it should be enabled, ready, and checking it.

Latest revision as of 10:51, 28 April 2010

Goal

To configure a Koji builder system (either VM or physical hardware).

Kojid is the build daemon that runs on each of the build machines. Its primary responsibility is polling for incoming build requests and handling them accordingly. Koji also has support for tasks other than building. Creating install images is one example. kojid is responsible for handling these tasks as well. kojid uses mock for building. It also creates a fresh buildroot for every build. kojid is written in Python and communicates with koji-hub via XML-RPC.

Configuration Files

  • /etc/kojid/kojid.conf - Koji Daemon Configuration
  • /etc/sysconfig/kojid - Koji Daemon Switches

Steps to Perform

Storage Space

Ensure that there is a good amount of space on the ARM system for mock (/var/lib/mock and /var/cache/mock) before proceeding. You will want a minimum of about 8GB.

On the ARM VMs, you may need to increase the VM size. Stop the VM, add additional storage, and then restart the VM:

On the VM: 

poweroff

Allow 30 seconds for system shutdown.

On HongKong: 

virsh destroy domain
dd if=/dev/zero bs=1M count=8192 >>/var/lib/libvirt/images/arm/domain.img
fsck -f /var/lib/libvirt/images/arm/domain.img
resize2fs /var/lib/libvirt/images/arm/domain.img
virsh start domain

Where domain is the VM name, such as arm-001-999.

If using a physical ARM system, ensure that /var/lib/mock and /var/cache/mock have sufficient space, perhaps by symlinking them to subdirectories of the mountpoint of an external disk, NFS share, or iSCSI block device.

Install kojid

yum install koji-builder

NFS Configuration

Perform the steps outlined on the NFS Configuration page, including adding entries to the /etc/hosts file.

Koji Configuration

/etc/kojid/kojid.conf: This needs to point at your koji-hub. 

; The URL for the xmlrpc server
server=http://hongkong/kojihub

; the username has to be the same as what you used with add-host
; in this example follow as below
user = arm-001-001 ;Matches your VM name

; The URL for the packages tree
pkgurl=http://hongkong/yum/arm_built/base/12/arm/

; This item may be changed, but may not be the same as KojiDir on the kojihub.conf file (although it can be something under KojiDir, just not the same as KojiDir)
 ; The directory root for temporary storage
workdir=/tmp/koji

SSL Certificates For Authentication

Copy the appropriate certificate(s) from HongKong:/etc/pki/koji/certs/

/etc/kojid/kojid.conf: 

 ;client certificate
; This should reference the builder certificate we created above, for
; kojibuilder1.example.com
cert = /etc/kojid/arm-001-008.pem

;certificate of the CA that issued the client certificate
ca = /etc/kojid/koji_ca_cert.crt

;certificate of the CA that issued the HTTP server certificate
serverca = /etc/kojid/koji_ca_cert.crt

Start Kojid

Important.png
Koji Host Entry
Before starting Kojid, make sure that the entry for your builder is in the database. The command to do this (as kojiadmin on HongKong) is koji add-host hostname armv5tel

After, start Kojid and configure it to start at each boot: 

service kojid start
chkconfig kojid on

Testing

Check /var/log/kojid.log on the builder to verify that kojid has started successfully.

Finally, check the Koji hosts web display on HongKong to see that the host is working properly -- it should be enabled, ready, and checking it.

Retrieved from "https://wiki.cdot.senecapolytechnic.ca/w/index.php?title=Fedora_Arm_Secondary_Architecture/Koji_Builders&oldid=38942"