User:Chris Tyler/Notes on meeting with Dennis Gilmore 2010-08-25

From CDOT Wiki
Jump to: navigation, search
Important.png
General Notes
These are general notes, not in any specific sequence.
Important.png
This is a draft only!
It is still under construction and content may change. Do not rely on this information.

2010-08-25

Koji Architecture and Background

Koji builds packages in a consistent and reproducable manner

  • Layered on top of yum, mock (...)
  • Keeps track of the buildroot used so that builds can be reproduced
  • Uses inheritance to simplify building one release based on a previous release
  • Replaces/next generation of "brew"
    • Current "brew" is themed koji
    • Previous system was "beehive" which used a static buildroot including the kitchen sink
      • Made it difficult to reproduce a build
  • Hub is passive, responds only to XML-RPC (via http or https)
  • Gaining new features:
    • Policy controls -- "this builder will only build F12, this other one F13 & F14"

Kojira

  • Monitors the tags that populate the buildroot
    • Queries koji-hub for "changes since ..." every 10-15 seconds
  • Queues newrepo tasks when a change is detected

Bodhi

  • Manipulates tags on koji-hub
    • distf12-updates-candidate > dist-f12-updates-testing > dist-f12-updates

Mash

  • Creates a repo of the latest build of each package with a particular tag

How to: Detect build dependency breakage

  • run 'repoclosure' several times a day to detect dependency breakage (due to e.g. soname bumps)
    • Now crontab'd to run on HongKong at 0600 and 1300 each day (user 'chris'), outputting to /tmp/depcheck.log

Questions

  • build-previous: primary arch tags become secondary arch targets, therefore the target set on primary and secondary archs is different
  • setup for git builds
  • building for higher arm architecture versions
  • SBR600 topic sequence

SBR project ideas

  • Script the goes through build failures and groups them together with analysis (which deps were missing, which section of build (%prep, %build, %install) caused a build failure)
  • Koji host display -- add "current tasks" and/or add "--builder $X" option to 'koji list-tasks'

2010.08.26

Tasks

  • Get repoclosure running
  • Align web access to built packages with what is done on the primary repo

Wiki Gardening

Needed on Fedora Wiki:

  • Updates to Koji Server HOWTO
  • For secondary archs (generally):
    1. Developer's page
      • Talking to secondary arch koji hubs
      • Contact info for secondary arch groups
    2. Users' page
      • General information
      • Index on arch-specific user pages
  • For each secondary arch:
    1. Developers & Team
      • Communication
      • Farm details
      • Status
      • Goals
    2. Users
      • What's available
      • How to Install
      • Contact
    3. Package maintainers
      • Common issues on $ARCH

Supporting multiple arch levels

  • ARM archs
    • E - enhanced for DSP
    • T - thumb
    • L - little-endian
    • B - big-endian
    • J - Jazelle
    • M - microcontroller (not typically used for user-level devices)
  • Reasonable path:
    • armv7+hardfp > armv5tel+softfp

Setting up koji to build from git

  1. Change the allowed_scms line in /etc/kojid/kojid.conf on each builder: allowed_scms=pkgs.fedoraproject.org:/*:false:fedpkg,sources
  2. Add to the srpm-build group on the hub: koji arm-koji add-group-pkg dist-f13-build srpm-build fedora-packager
  3. Install updated fedora-packager in buildroot - Still need to do!

Other changes to the /etc/kojid/kojid.conf file: 

; The vendor to use in rpm headers
vendor=Fedora Project

; The packager to use in rpm headers
packager=Fedora Project

; the distribution to use in rpm headers
distribution=Fedora Project

; The _host string to use in mock
mockhost=redhat-linux-gnu

; E-mail origination address
from_addr=ARM Koji Build System <buildsys@fedoraproject.org>

Next steps

  1. finish running build-previous
  2. set up and run koji-shadow
    1. run:
      • koji-shadow -c koji-shadow.conf.test --prefer-new --tag-build --import-noarch dist-f13
      • koji-shadow --tag-build dist-f13 # taking advantage of defaults
  3. Apply patch to koji-hub to permit koji-shadow builds: http://git.fedorahosted.org/git/?p=koji;a=commit;h=27b983573f8dd9fc3e0ef75acd81b0f3b2317e84
    1. set up config file /etc/koji-shado/koji-shadow.conf:
# koji-shadow example config file
# (still working out all the config options)

[main]
server=http://sparc.koji.fedoraproject.org/kojihub/
remote=http://koji.fedoraproject.org/kojihub
arches=sparcv9 sparc64
prefer_new=True
import_noarch=True
auth_cert=/home/dennis/.fedora.cert
auth_ca=/home/dennis/.fedora-server-ca.cert
serverca=/home/dennis/.fedora-server-ca.cert
remote_topurl=http://kojipkgs.fedoraproject.org/

[rules]
ignorelist=Glide3 Glide3-libGL alex amtu acpid apmd apmud athcool biosdevname camstream darcs cmospwd cmucl compat-gcc-296 cabal-install coredumper cpufrequtils cpuid crash hlint dmidecode cpphs dssi-vst edac-utils efibootmgr elilo fedora-ksplice firmware-addon-dell freeipmi ghc ghc-GLUT ghc-HUnit ghc-OpenGL ghc-X11 ghc-X11-xft ghc-editline ghc-fgl ghc-ghc-paths ghc-gtk2hs ghc-haskell-src ghc-html ghc-mmap ghc-mtl ghc-parallel ghc-parsec ghc-regex-base ghc-regex-compat ghc-regex-posix ghc-stm ghc-tar ghc-haskeline ghc-xhtml ghc-xmonad-contrib ghc-zlib ghc-haskell-src-exts ghc-uniplate gnu-efi gpart gprolog gsynaptics ghc-HTTP eclipse-cdt happy haskell-platform i8kutils ibmasm grub ghc-cgi ioport iprutils ipw2100-firmware ipw2200-firmware irqbalance flashrom ksplice latrace libbsr hscolour librtas libsmbios libspe2 libunwind libx86 lrmi mcelog memtest86+ msr-tools nspluginwrapper seabios ohm olpc-kbdshim olpc-powerd ghc-dataenc ghc-hashed-storage pcc xorg-x11-drv-openchrome ghc-binary picprog pmtools powerpc-utils powerpc-utils-papr ppc64-utils microcode_ctl procbench ps3-utils pvs-sbcl python-psyco eclipse-changelog pcmciautils openscada redhat-lsb s3switch eclipse-rpm-editor spicctrl stapitrace svgalib syslinux sysprof system-config-boot tbb ghc-QuickCheck tpb unetbootin valgrind vbetool ghc-network viaideinfo yaboot vrq xenner wine wraplinux x86info xen xmonad xorg-x11-drv-geode xorg-x11-drv-intel xorg-x11-drv-neomagic xorg-x11-drv-vmware  xorg-x11-drv-vmmouse superiotool qpid-cpp

substitutions=nfs-utils-lib-1.1.1-3.fc9 nfs-utils-lib-1.1.1-5.fc9
             glibc32-2.4.90-13 glibc32-2.6.90-14.2
             glibc64-2.4.90-13 glibc64-2.6.90-14.2

includelist=fake-build-provides-1.0-3.fc12 xorg-x11-drv-ati-6.13.0-0.21.20100219gite68d3a389.fc12 poppler-data-0.4.0-1.fc12 source-highlight-2.10-4.fc12

protectlist=fedora-release

2010-08-27

mash

  • Configuration files are in /etc/mash
    • tag.mash
      • One section for each tag to be composed
    • mash.conf
      • Master configuration options
  • Question: will deltarpms be useful on ARM?
    • Saves bandwidth
    • Costs a lot in CPU and storage

Example command line: 

 mash -o /mnt/koji/mask/f13-arm-20100827 f13-arm

Signing packages

  • sigul - signing server (new)
    • Signers have their own key/passphrase but no one needs to know the master passphrase for the key
  • Alternate is releng's sign.py/unsign.py script

glibc

  • ARM arch stuff in in a *ports tarball, needs to be combined with main code
  • Borrow bits from F14 glibc spec file to make it build
Retrieved from "https://wiki.cdot.senecapolytechnic.ca/w/index.php?title=User:Chris_Tyler/Notes_on_meeting_with_Dennis_Gilmore_2010-08-25&oldid=43126"