Open main menu

CDOT Wiki β

Koji Follow

Revision as of 18:13, 5 March 2013 by Fossjon (talk | contribs)

Koji Follow Que Script

Notes

  • This script does not automatically solve circular dependency issues (which there are a lot of, especially packages like sub perl modules)
  • This script does not automatically solve bootstrapping or library.so name bump requires issues (which happens often during a completely new tag rebuild)
  • This script does not attempt to solve any package build failure issues
  • This script can be run in 2 modes: "mass-rebuild" and "latest-follow" and ques builds in order based on their "creation_ts" value
    • mass-rebuild mode will attempt to compile a complete list of packages that were built for a given tag and then que them in the given order
    • latest-follow mode will perform a "list-tagged --latest" and que each unbuilt package in the given order

Possible Breakage Scenarios

Scenario 1

  1. libA-1.0-1.fc17 is built (Provides: libA.so.1)
  2. packageB-1.0-1.fc17 depends on packageC but is bootstrapped without it and built
  3. packageC-1.0-1.fc17 depends on libA,packageB and is built with libA-1.0-1.fc17,packageB-1.0-1.fc17 (Requires: libA.so.1)
  4. packageB-1.0-2.fc17 depends on packageC and is built with packageC-1.0-1.fc17
  5. libA-2.0-1.fc18 is built (Provides: libA.so.2)
    1. Attempting to build the full version of packageB-2.0-1.fc18 will depend on packageC-1.0-1.fc17 which requires libA.so.1 which only find libA.so.2 with the current repodata (this package needs to be bootstrapped again)
    2. Attempting to build packageC-2.0-1.fc18 will depend on packageB-1.0-2.fc17 which will depend on packageC-1.0-1.fc17 which brings us to the breakage error above

Scenario 2

  1. libA-1.0-1.fc17 is built (Provides: libA.so.1)
  2. packageB-1.0-1.fc17 depends on packageC but is bootstrapped without it and built
  3. packageC-1.0-1.fc17 depends on libA,packageB and is built with libA-1.0-1.fc17,packageB-1.0-1.fc17 (Requires: libA.so.1)
  4. packageB-1.0-2.fc17 depends on packageC and is built with packageC-1.0-1.fc17

Downloads

Example Config and Execution Scenarios

mass-rebuild mode

~/koji-follow.conf 

primary_url = "http://koji.fedoraproject.org"
primary_arch = "x86_64"
#secondary_url = "http://arm.koji.fedoraproject.org"
#secondary_arch = "armhfp"

#primary_url = "http://arm.koji.fedoraproject.org"
#primary_arch = "armhfp"
secondary_url = "http://japan.proximity.on.ca"
secondary_arch = "armv6hl"

client_cert = os.path.expanduser("~/.fedora.cert")
server_cert = os.path.expanduser("~/.fedora-server-ca.cert")

target_arch = "arm"
tag_name = "f18"
que_limit = 80

excl_list = ["glibc","kernel","koji","redhat-rpm-config","rpm","yum"]
check_tag = "all"
retry_build = "all"

python ~/koji-follow.py ~/koji-follow.conf > ~/logs/kfo.log 2> ~/logs/kfe.log

latest-follow mode

~/koji-follow.conf 

primary_url = "http://koji.fedoraproject.org"
primary_arch = "x86_64"
#secondary_url = "http://arm.koji.fedoraproject.org"
#secondary_arch = "armhfp"

#primary_url = "http://arm.koji.fedoraproject.org"
#primary_arch = "armhfp"
secondary_url = "http://japan.proximity.on.ca"
secondary_arch = "armv6hl"

client_cert = os.path.expanduser("~/.fedora.cert")
server_cert = os.path.expanduser("~/.fedora-server-ca.cert")

target_arch = "arm"
tag_name = "f18"
que_limit = 80

excl_list = ["glibc","kernel","koji","redhat-rpm-config","rpm","yum"]
check_tag = "latest"
retry_build = "failed"

python ~/koji-follow.py ~/koji-follow.conf > ~/logs/kfo.log 2> ~/logs/kfe.log

Example Output

STDOUT Messages

[info] pkg_process: [1/6604]: {'owner_name': 'jkeating', 'package_name': 'email2trac', 'task_id': 1503441, 'creation_event_id': 1793404, 'creation_time': '2009-07-25 14:59:59.646649', 'epoch': None, 'nvr': 'email2trac-0.13-6.fc12', 'name': 'email2trac', 'completion_time': '2009-07-25 15:35:37.427707', 'state': 1, 'version': '0.13', 'package_id': 6215, 'volume_name': 'DEFAULT', 'release': '6.fc12', 'creation_ts': 1248533999.64665, 'completion_ts': 1248536137.42771, 'id': 117394, 'volume_id': 0, 'owner_id': 2}
        [info] que_build: [email2trac-0.13-6.fc12.src.rpm] -> [cli-build/1362515245.29142.RuqZKocF]
[info] pkg_process: [2/6604]: {'owner_name': 'jkeating', 'package_name': 'powerpc-utils-papr', 'task_id': 1525540, 'creation_event_id': 1839140, 'creation_time': '2009-07-28 04:43:55.459368', 'epoch': None, 'nvr': 'powerpc-utils-papr-1.1.6-2.fc12', 'name': 'powerpc-utils-papr', 'completion_time': '2009-07-28 05:47:09.877729', 'state': 1, 'version': '1.1.6', 'package_id': 5369, 'volume_name': 'DEFAULT', 'release': '2.fc12', 'creation_ts': 1248756235.45937, 'completion_ts': 1248760029.87773, 'id': 122145, 'volume_id': 0, 'owner_id': 2}
        [info] que_build: [powerpc-utils-papr-1.1.6-2.fc12.src.rpm] -> [cli-build/1362515248.044621.imzInFfA]
[info] pkg_process: [3/6604]: {'owner_name': 'jkeating', 'package_name': 'ppc64-utils', 'task_id': 1525552, 'creation_event_id': 1839195, 'creation_time': '2009-07-28 04:46:54.025335', 'epoch': None, 'nvr': 'ppc64-utils-0.14-5.fc12', 'name': 'ppc64-utils', 'completion_time': '2009-07-28 05:51:18.860184', 'state': 1, 'version': '0.14', 'package_id': 131, 'volume_name': 'DEFAULT', 'release': '5.fc12', 'creation_ts': 1248756414.02534, 'completion_ts': 1248760278.86018, 'id': 122152, 'volume_id': 0, 'owner_id': 2}
        [info] que_build: [ppc64-utils-0.14-5.fc12.src.rpm] -> [cli-build/1362515249.622192.vcbTPtpj]
[info] pkg_process: [4/6604]: {'owner_name': 'rrakus', 'package_name': 'iprutils', 'task_id': 1787705, 'creation_event_id': 2291747, 'creation_time': '2009-11-04 05:55:36.885871', 'epoch': None, 'nvr': 'iprutils-2.2.18-1.fc12', 'name': 'iprutils', 'completion_time': '2009-11-04 05:58:25.827132', 'state': 1, 'version': '2.2.18', 'package_id': 722, 'volume_name': 'DEFAULT', 'release': '1.fc12', 'creation_ts': 1257314136.88587, 'completion_ts': 1257314305.82713, 'id': 139717, 'volume_id': 0, 'owner_id': 613}
        [info] que_build: [iprutils-2.2.18-1.fc12.src.rpm] -> [cli-build/1362515252.116506.kQNgDlWL]
[info] pkg_process: [5/6604]: {'owner_name': 'sharkcz', 'package_name': 'gtksourceview2-sharp', 'task_id': 3035160, 'creation_event_id': 3682022, 'creation_time': '2011-04-29 07:07:04.359159', 'epoch': None, 'nvr': 'gtksourceview2-sharp-1.0-10.svn89788.fc16', 'name': 'gtksourceview2-sharp', 'completion_time': '2011-04-29 07:09:55.860531', 'state': 1, 'version': '1.0', 'package_id': 6142, 'volume_name': 'DEFAULT', 'release': '10.svn89788.fc16', 'creation_ts': 1304060824.35916, 'completion_ts': 1304060995.86053, 'id': 241357, 'volume_id': 0, 'owner_id': 244}
        [info] que_build: [gtksourceview2-sharp-1.0-10.svn89788.fc16.src.rpm] -> [cli-build/1362515254.528944.WTNbsaqJ]
...

STDERR Messages

[info] primary_tag: [1188/1188/1584]: rwho
[info] primary_tag: [396/396/792]: jlatexmath
[info] primary_tag: [0/0/396]: fprintd
[info] primary_tag: [1584/1584/1980]: objectweb-asm4
[info] primary_tag: [1980/1980/2376]: ktp-approver
[info] primary_tag: [792/792/1188]: nginx
[info] primary_tag: [3168/3168/3564]: audex
[info] primary_tag: [2376/2376/2772]: python-sh
[info] primary_tag: [3564/3564/3960]: python-cmd2
[info] primary_tag: [2772/2772/3168]: perl-IPC-Run-SafeHandles
...
        [info] file_delete: aplus-fsf-4.22.4-21.fc16.src.rpm
        [info] file_download: [http://koji.fedoraproject.org/packages/aplus-fsf/4.22.4/21.fc16/src/aplus-fsf-4.22.4-21.fc16.src.rpm] -> [aplus-fsf-4.22.4-21.fc16.src.rpm]
        [info] file_delete: tritonus-0.3.7-0.13.20101108cvs.fc17.src.rpm
        [info] file_download: [http://koji.fedoraproject.org/packages/tritonus/0.3.7/0.13.20101108cvs.fc17/src/tritonus-0.3.7-0.13.20101108cvs.fc17.src.rpm] -> [tritonus-0.3.7-0.13.20101108cvs.fc17.src.rpm]
        [info] file_delete: jffi-1.0.10-1.fc17.src.rpm
        [info] file_download: [http://koji.fedoraproject.org/packages/jffi/1.0.10/1.fc17/src/jffi-1.0.10-1.fc17.src.rpm] -> [jffi-1.0.10-1.fc17.src.rpm]
        [info] file_delete: nautilus-sendto-trac-0.4-2.fc17.src.rpm
        [info] file_download: [http://koji.fedoraproject.org/packages/nautilus-sendto-trac/0.4/2.fc17/src/nautilus-sendto-trac-0.4-2.fc17.src.rpm] -> [nautilus-sendto-trac-0.4-2.fc17.src.rpm]
        [info] file_delete: sweep-0.9.3-9.fc17.src.rpm
        [info] file_download: [http://koji.fedoraproject.org/packages/sweep/0.9.3/9.fc17/src/sweep-0.9.3-9.fc17.src.rpm] -> [sweep-0.9.3-9.fc17.src.rpm]
...
Retrieved from "https://wiki.cdot.senecapolytechnic.ca/w/index.php?title=Koji_Follow&oldid=94366"