1,885
edits
Changes
Created page with '{{Admon/important|General Notes|These are general notes, not in any specific sequence.}} = 2010-08-25 = == Koji Architecture and Background == === Koji builds packages in a cons…'
{{Admon/important|General Notes|These are general notes, not in any specific sequence.}}
= 2010-08-25 =
== Koji Architecture and Background ==
=== Koji builds packages in a consistent and reproducable manner ===
* Layered on top of yum, mock (...)
* Keeps track of the buildroot used so that builds can be reproduced
* Uses inheritance to simplify building one release based on a previous release
* Replaces/next generation of "brew"
** Current "brew" is themed koji
** Previous system was "beehive" which used a static buildroot including the kitchen sink
*** Made it difficult to reproduce a build
* Hub is passive, responds only to XML-RPC (via http or https)
* Gaining new features:
** Policy controls -- "this builder will only build F12, this other one F13 & F14"
=== Kojira ===
* Monitors the tags that populate the buildroot
** Queries koji-hub for "changes since ..." every 10-15 seconds
* Queues newrepo tasks when a change is detected
=== Bodhi ===
* Manipulates tags on koji-hub
** distf12-updates-candidate > dist-f12-updates-testing > dist-f12-updates
=== Mash ===
* Creates a repo of the latest build of each package with a particular tag
== How to: Detect build dependency breakage ==
* run 'repoclosure' several times a day to detect dependency breakage (due to e.g. soname bumps)
** Now crontab'd to run on HongKong at 0600 and 1300 each day (user 'chris'), outputting to /tmp/depcheck.log
== Questions ==
* build-previous: primary arch tags become secondary arch targets, therefore the target set on primary and secondary archs is different
* setup for git builds
* building for higher arm architecture versions
* SBR600 topic sequence
== SBR project ideas ==
* Script the goes through build failures and groups them together with analysis (which deps were missing, which section of build (%prep, %build, %install) caused a build failure)
* Koji host display -- add "current tasks" and/or add "--builder $X" option to 'koji list-tasks'
= 2010.08.26 =
== Tasks ==
* Get repoclosure running
* Align web access to built packages with what is done on the primary repo
== Wiki Gardening ==
Needed on Fedora Wiki:
* Updates to Koji Server HOWTO
* For secondary archs (generally):
*# Developer's page
*#* Talking to secondary arch koji hubs
*#* Contact info for secondary arch groups
*# Users' page
*#* General information
*#* Index on arch-specific user pages
* For each secondary arch:
*# Developers & Team
*#* Communication
*#* Farm details
*#* Status
*#* Goals
*# Users
*#* What's available
*#* How to Install
*#* Contact
*# Package maintainers
*#* Common issues on $ARCH
== Supporting multiple arch levels ==
* ARM archs
** E - enhanced for DSP
** T - thumb
** L - little-endian
** B - big-endian
** J - Jazelle
** M - microcontroller (not typically used for user-level devices)
* Reasonable path:
** armv7+hardfp > armv5tel+softfp
== Setting up koji to build from git ==
# Change the allowed_scms line in /etc/kojid/kojid.conf on each builder: <code>allowed_scms=pkgs.fedoraproject.org:/*:false:fedpkg,sources</code>
# Add to the srpm-build group on the hub: koji arm-koji add-group-pkg dist-f13-build srpm-build fedora-packager
# Install updated fedora-packager in buildroot - '''Still need to do!'''
Other changes to the <code>/etc/kojid/kojid.conf</code> file:
; The vendor to use in rpm headers
vendor=Fedora Project
; The packager to use in rpm headers
packager=Fedora Project
; the distribution to use in rpm headers
distribution=Fedora Project
; The _host string to use in mock
mockhost=redhat-linux-gnu
; E-mail origination address
from_addr=ARM Koji Build System <buildsys@fedoraproject.org>
== Next steps ==
# finish running build-previous
# set up and run koji-shadow
## set up config file /etc/koji-shado/koji-shadow.conf
## run:
##* koji-shadow -c koji-shadow.conf.test --prefer-new --tag-build --import-noarch dist-f13
##* koji-shadow --tag-build dist-f13 # taking advantage of defaults
# koji-shadow example config file
# (still working out all the config options)
[main]
server=http://sparc.koji.fedoraproject.org/kojihub/
remote=http://koji.fedoraproject.org/kojihub
arches=sparcv9 sparc64
prefer_new=True
import_noarch=True
auth_cert=/home/dennis/.fedora.cert
auth_ca=/home/dennis/.fedora-server-ca.cert
serverca=/home/dennis/.fedora-server-ca.cert
remote_topurl=http://kojipkgs.fedoraproject.org/
[rules]
ignorelist=Glide3 Glide3-libGL alex amtu acpid apmd apmud athcool biosdevname camstream darcs cmospwd cmucl compat-gcc-296 cabal-install coredumper cpufrequtils cpuid crash hlint dmidecode cpphs dssi-vst edac-utils efibootmgr elilo fedora-ksplice firmware-addon-dell freeipmi ghc ghc-GLUT ghc-HUnit ghc-OpenGL ghc-X11 ghc-X11-xft ghc-editline ghc-fgl ghc-ghc-paths ghc-gtk2hs ghc-haskell-src ghc-html ghc-mmap ghc-mtl ghc-parallel ghc-parsec ghc-regex-base ghc-regex-compat ghc-regex-posix ghc-stm ghc-tar ghc-haskeline ghc-xhtml ghc-xmonad-contrib ghc-zlib ghc-haskell-src-exts ghc-uniplate gnu-efi gpart gprolog gsynaptics ghc-HTTP eclipse-cdt happy haskell-platform i8kutils ibmasm grub ghc-cgi ioport iprutils ipw2100-firmware ipw2200-firmware irqbalance flashrom ksplice latrace libbsr hscolour librtas libsmbios libspe2 libunwind libx86 lrmi mcelog memtest86+ msr-tools nspluginwrapper seabios ohm olpc-kbdshim olpc-powerd ghc-dataenc ghc-hashed-storage pcc xorg-x11-drv-openchrome ghc-binary picprog pmtools powerpc-utils powerpc-utils-papr ppc64-utils microcode_ctl procbench ps3-utils pvs-sbcl python-psyco eclipse-changelog pcmciautils openscada redhat-lsb s3switch eclipse-rpm-editor spicctrl stapitrace svgalib syslinux sysprof system-config-boot tbb ghc-QuickCheck tpb unetbootin valgrind vbetool ghc-network viaideinfo yaboot vrq xenner wine wraplinux x86info xen xmonad xorg-x11-drv-geode xorg-x11-drv-intel xorg-x11-drv-neomagic xorg-x11-drv-vmware xorg-x11-drv-vmmouse superiotool qpid-cpp
substitutions=nfs-utils-lib-1.1.1-3.fc9 nfs-utils-lib-1.1.1-5.fc9
glibc32-2.4.90-13 glibc32-2.6.90-14.2
glibc64-2.4.90-13 glibc64-2.6.90-14.2
includelist=fake-build-provides-1.0-3.fc12 xorg-x11-drv-ati-6.13.0-0.21.20100219gite68d3a389.fc12 poppler-data-0.4.0-1.fc12 source-highlight-2.10-4.fc12
protectlist=fedora-release
= 2010-08-25 =
== Koji Architecture and Background ==
=== Koji builds packages in a consistent and reproducable manner ===
* Layered on top of yum, mock (...)
* Keeps track of the buildroot used so that builds can be reproduced
* Uses inheritance to simplify building one release based on a previous release
* Replaces/next generation of "brew"
** Current "brew" is themed koji
** Previous system was "beehive" which used a static buildroot including the kitchen sink
*** Made it difficult to reproduce a build
* Hub is passive, responds only to XML-RPC (via http or https)
* Gaining new features:
** Policy controls -- "this builder will only build F12, this other one F13 & F14"
=== Kojira ===
* Monitors the tags that populate the buildroot
** Queries koji-hub for "changes since ..." every 10-15 seconds
* Queues newrepo tasks when a change is detected
=== Bodhi ===
* Manipulates tags on koji-hub
** distf12-updates-candidate > dist-f12-updates-testing > dist-f12-updates
=== Mash ===
* Creates a repo of the latest build of each package with a particular tag
== How to: Detect build dependency breakage ==
* run 'repoclosure' several times a day to detect dependency breakage (due to e.g. soname bumps)
** Now crontab'd to run on HongKong at 0600 and 1300 each day (user 'chris'), outputting to /tmp/depcheck.log
== Questions ==
* build-previous: primary arch tags become secondary arch targets, therefore the target set on primary and secondary archs is different
* setup for git builds
* building for higher arm architecture versions
* SBR600 topic sequence
== SBR project ideas ==
* Script the goes through build failures and groups them together with analysis (which deps were missing, which section of build (%prep, %build, %install) caused a build failure)
* Koji host display -- add "current tasks" and/or add "--builder $X" option to 'koji list-tasks'
= 2010.08.26 =
== Tasks ==
* Get repoclosure running
* Align web access to built packages with what is done on the primary repo
== Wiki Gardening ==
Needed on Fedora Wiki:
* Updates to Koji Server HOWTO
* For secondary archs (generally):
*# Developer's page
*#* Talking to secondary arch koji hubs
*#* Contact info for secondary arch groups
*# Users' page
*#* General information
*#* Index on arch-specific user pages
* For each secondary arch:
*# Developers & Team
*#* Communication
*#* Farm details
*#* Status
*#* Goals
*# Users
*#* What's available
*#* How to Install
*#* Contact
*# Package maintainers
*#* Common issues on $ARCH
== Supporting multiple arch levels ==
* ARM archs
** E - enhanced for DSP
** T - thumb
** L - little-endian
** B - big-endian
** J - Jazelle
** M - microcontroller (not typically used for user-level devices)
* Reasonable path:
** armv7+hardfp > armv5tel+softfp
== Setting up koji to build from git ==
# Change the allowed_scms line in /etc/kojid/kojid.conf on each builder: <code>allowed_scms=pkgs.fedoraproject.org:/*:false:fedpkg,sources</code>
# Add to the srpm-build group on the hub: koji arm-koji add-group-pkg dist-f13-build srpm-build fedora-packager
# Install updated fedora-packager in buildroot - '''Still need to do!'''
Other changes to the <code>/etc/kojid/kojid.conf</code> file:
; The vendor to use in rpm headers
vendor=Fedora Project
; The packager to use in rpm headers
packager=Fedora Project
; the distribution to use in rpm headers
distribution=Fedora Project
; The _host string to use in mock
mockhost=redhat-linux-gnu
; E-mail origination address
from_addr=ARM Koji Build System <buildsys@fedoraproject.org>
== Next steps ==
# finish running build-previous
# set up and run koji-shadow
## set up config file /etc/koji-shado/koji-shadow.conf
## run:
##* koji-shadow -c koji-shadow.conf.test --prefer-new --tag-build --import-noarch dist-f13
##* koji-shadow --tag-build dist-f13 # taking advantage of defaults
# koji-shadow example config file
# (still working out all the config options)
[main]
server=http://sparc.koji.fedoraproject.org/kojihub/
remote=http://koji.fedoraproject.org/kojihub
arches=sparcv9 sparc64
prefer_new=True
import_noarch=True
auth_cert=/home/dennis/.fedora.cert
auth_ca=/home/dennis/.fedora-server-ca.cert
serverca=/home/dennis/.fedora-server-ca.cert
remote_topurl=http://kojipkgs.fedoraproject.org/
[rules]
ignorelist=Glide3 Glide3-libGL alex amtu acpid apmd apmud athcool biosdevname camstream darcs cmospwd cmucl compat-gcc-296 cabal-install coredumper cpufrequtils cpuid crash hlint dmidecode cpphs dssi-vst edac-utils efibootmgr elilo fedora-ksplice firmware-addon-dell freeipmi ghc ghc-GLUT ghc-HUnit ghc-OpenGL ghc-X11 ghc-X11-xft ghc-editline ghc-fgl ghc-ghc-paths ghc-gtk2hs ghc-haskell-src ghc-html ghc-mmap ghc-mtl ghc-parallel ghc-parsec ghc-regex-base ghc-regex-compat ghc-regex-posix ghc-stm ghc-tar ghc-haskeline ghc-xhtml ghc-xmonad-contrib ghc-zlib ghc-haskell-src-exts ghc-uniplate gnu-efi gpart gprolog gsynaptics ghc-HTTP eclipse-cdt happy haskell-platform i8kutils ibmasm grub ghc-cgi ioport iprutils ipw2100-firmware ipw2200-firmware irqbalance flashrom ksplice latrace libbsr hscolour librtas libsmbios libspe2 libunwind libx86 lrmi mcelog memtest86+ msr-tools nspluginwrapper seabios ohm olpc-kbdshim olpc-powerd ghc-dataenc ghc-hashed-storage pcc xorg-x11-drv-openchrome ghc-binary picprog pmtools powerpc-utils powerpc-utils-papr ppc64-utils microcode_ctl procbench ps3-utils pvs-sbcl python-psyco eclipse-changelog pcmciautils openscada redhat-lsb s3switch eclipse-rpm-editor spicctrl stapitrace svgalib syslinux sysprof system-config-boot tbb ghc-QuickCheck tpb unetbootin valgrind vbetool ghc-network viaideinfo yaboot vrq xenner wine wraplinux x86info xen xmonad xorg-x11-drv-geode xorg-x11-drv-intel xorg-x11-drv-neomagic xorg-x11-drv-vmware xorg-x11-drv-vmmouse superiotool qpid-cpp
substitutions=nfs-utils-lib-1.1.1-3.fc9 nfs-utils-lib-1.1.1-5.fc9
glibc32-2.4.90-13 glibc32-2.6.90-14.2
glibc64-2.4.90-13 glibc64-2.6.90-14.2
includelist=fake-build-provides-1.0-3.fc12 xorg-x11-drv-ati-6.13.0-0.21.20100219gite68d3a389.fc12 poppler-data-0.4.0-1.fc12 source-highlight-2.10-4.fc12
protectlist=fedora-release
