Difference between revisions of "Pidora SOP"

From CDOT Wiki
Jump to: navigation, search
(→‎Examples runs)
(→‎Running pidora-update.py)
Line 51: Line 51:
 
   --auto                enables logging and emails logs
 
   --auto                enables logging and emails logs
 
   --koji-tag=KOJITAG    specify the koji tag to sign
 
   --koji-tag=KOJITAG    specify the koji tag to sign
−
   --email=andrew.oatley-willis@senecacollege.ca
+
   --email=user@email.com
 
                         specify the email to send logs to
 
                         specify the email to send logs to
−
   --sigul-user=agreene  specify the user for sigul
+
   --sigul-user=user    specify the user for sigul
 
   --sigul-host=england.proximity.on.ca
 
   --sigul-host=england.proximity.on.ca
 
                         specify the host for sigul
 
                         specify the host for sigul
−
   --mash-user=root     specify the user for mash
+
   --mash-user=user     specify the user for mash
 
   --mash-host=japan.proximity.on.ca
 
   --mash-host=japan.proximity.on.ca
 
                         specify the host for mash
 
                         specify the host for mash
−
   --rsync-user=pidorapr
+
   --rsync-user=user
 
                         specify the user for rsync
 
                         specify the user for rsync
 
   --rsync-host=pidora.proximity.on.ca
 
   --rsync-host=pidora.proximity.on.ca

Revision as of 15:38, 30 September 2013

Pidora-horizontal.png

Intro

This page covers how one can manage the various needed parts of a complete release process. This includes the other SOP pages containing file configurations and/or quick command line references for the following procedures:

  1. Building packages
  2. Signing packages
  3. Creating repositories
  4. Composing images

Building Packages

Koji-Follow Source Code and Configuration SOPs

How-To-Run

  • ssh <your_username>@japan
  • sudo kojiadmin
  • screen -xr follow
  • python ~/koji-follow.py ~/koji-follow.conf > ~/logs/kfo.log 2> ~/logs/kfe.log

Post Commands

  • Repo analysis: 
    repoclosure --arch=armv6hl --arch=noarch --repofrompath=v6,http://japan.proximity.on.ca/repos/f18-build/latest/armv6hl/ -r v6 | tee /tmp/reporeport.log

Sign, Mash, and Rsync Together!

Prerequisites

  • Must setup passwordless logins from root@japan to all users in the script
  • Must have a working mash configuration: See mash below
  • Must have a configured sigul user with access to the signing key
  • Make sure that any externally called scripts are in the correct locations


Running pidora-update.py

  • ssh japan
  • run as root: pidora-smr
  • script location /root/pidora-update/pidora-update.py
Usage: pidora-smr [options]

Options:
  -h, --help            show this help message and exit
  -i, --info            check machine status and configuration
  -a, --all             sign, mash, rsync
  -s, --sign            sign all packages in listed tag
  -m, --mash            start a mash run
  -r, --rsync           perform a rsync of the mash repos
  -l, --list-unsigned   list unsigned rpms
  --auto                enables logging and emails logs
  --koji-tag=KOJITAG    specify the koji tag to sign
  --email=user@email.com
                        specify the email to send logs to
  --sigul-user=user     specify the user for sigul
  --sigul-host=england.proximity.on.ca
                        specify the host for sigul
  --mash-user=user      specify the user for mash
  --mash-host=japan.proximity.on.ca
                        specify the host for mash
  --rsync-user=user
                        specify the user for rsync
  --rsync-host=pidora.proximity.on.ca
                        specify the host for rsync
  --log-dir=/var/log/pidora-smr/
                        specify a logging directory
  --log-file=output     specify a log file name
  • All the configurations in the script can be changed with the command line options
    • Can change the user to access: sigul, mash, and rsync
    • Can change the host which runs: sigul, mash, and rsync
    • Can change which koji tag will be used for signing

Examples runs

  • Check which hosts can connect successfully
    • Also shows default configurations

pidora-smr --info

[Connection]
sigulhost = england.proximity.on.ca
siguluser = user
mashhost = japan.proximity.on.ca
mashuser = user
rsynchost = pidora.proximity.on.ca
rsyncuser = user

[General]
auto = False
mashdir = /usr/local/bin/mash-pidora
kojitags = ['f18-updates', 'f18-rpfr-updates', 'f18-updates-testing', 'f18-rpfr-updates-testing']
email = email@senecacollege.ca

logdir = /var/log/pidora-smr/
logfile = /var/log/pidora-smr/output

[Hosts]
working hosts: ['japan.proximity.on.ca', 'england.proximity.on.ca', 'pidora.proximity.on.ca']
failed hosts: []
  • Get a list of unsigned packages
    • Can check a single tag with --koji-tag=<tag-name>

pidora-smr --list-unsigned
  • Using the default configuration, start a sign, mash, rsync

pidora-smr --all
  • Change sigul user

pidora-smr --all --sigul-user=oatley
  • Run sign, mash, or sync separately, instead of together with the --all option

pidora-smr --sign
pidora-smr --mash
pidora-smr --rsync


  • Changing the configurations with command line options
    • Sign using a different user, on a single koji tag

pidora-smr --sign --koji-tag=f18-updates-testing

Changing Default Configurations

  • Can view all default configurations with the following command:

pidora-smr --info
  • Modify the script with a text editor
    • Change any of the variables listed below:
sigulhost = england.proximity.on.ca
siguluser = user
mashhost = japan.proximity.on.ca
mashuser = user
rsynchost = pidora.proximity.on.ca
rsyncuser = user

[General]
auto = False
mashdir = /usr/local/bin/mash-pidora
kojitags = ['f18-updates', 'f18-rpfr-updates', 'f18-updates-testing', 'f18-rpfr-updates-testing']
email = email@senecacollege.ca

Source Code

Pidora-smr source code

Signing Packages

Sigul Configuration SOPs

Creating Repositories

Mash File Configuration Details

/etc/mash/pidora-18.mash

# mash config file

[pidora-18]
rpm_path = %(arch)s/os/Packages
repodata_path = %(arch)s/os/
source_path = source/SRPMS
debuginfo = True
multilib = False
multilib_method = devel
tag = f18-rpfr
inherit = True

# Unsigned is temporary
strict_keys = True
#keys = 34E166FA, F8DF67E6, A82BA4B7, 069C8460, 97A1071F, E8E40FDE, 57BBCCBA, D22E77F2, 4EBFC273, 0B86274E, 6DF2196F, DF9B0AE9
#keys = CE8D31E3
keys = f1590cd5

repoviewurl = http://download.fedoraproject.org/pub/fedora-secondary/development/18/%(arch)s/os/
repoviewtitle = "Fedora Branched - %(arch)s"
arches = armhfp
# armv6 doesn't have horsepower to make delta useful
delta = False
# Change distro_tags as fedora-release version gets bumped
# The cpe URI should be fixed up
distro_tags = cpe:/o:fedoraproject:fedora:18 rawhide
hash_packages = True

/etc/mash/pidora-18-updates.mash

[pidora-18-updates]
rpm_path = %(arch)s/
repodata_path = %(arch)s/
source_path = SRPMS
debuginfo = True
multilib = False
multilib_method = devel
tag = f18-updates
inherit = False

# Unsigned is temporary
strict_keys = True
#keys = 34E166FA, F8DF67E6, A82BA4B7, 069C8460, 97A1071F, E8E40FDE, 57BBCCBA, D22E77F2, 4EBFC273, 0B86274E, 6DF2196F, DF9B0AE9
#keys = CE8D31E3
keys = f1590cd5

repoviewurl = http://download.fedoraproject.org/pub/fedora-secondary/development/18/%(arch)s/os/
repoviewtitle = "Fedora Branched - %(arch)s"
arches = armhfp
# armv6 doesn't have horsepower to make delta useful
delta = False
# Change distro_tags as fedora-release version gets bumped
# The cpe URI should be fixed up
distro_tags = cpe:/o:fedoraproject:fedora:18 update
hash_packages = True

/etc/mash/pidora-18-updates-testing.mash

[pidora-18-updates-testing]
rpm_path = %(arch)s/
repodata_path = %(arch)s/
source_path = SRPMS
debuginfo = True
multilib = False
multilib_method = devel
tag = f18-updates-testing
inherit = False

# Unsigned is temporary
strict_keys = True
#keys = 34E166FA, F8DF67E6, A82BA4B7, 069C8460, 97A1071F, E8E40FDE, 57BBCCBA, D22E77F2, 4EBFC273, 0B86274E, 6DF2196F, DF9B0AE9
#keys = CE8D31E3
keys = f1590cd5

repoviewurl = http://download.fedoraproject.org/pub/fedora-secondary/development/18/%(arch)s/os/
repoviewtitle = "Fedora Branched - %(arch)s"
arches = armhfp
# armv6 doesn't have horsepower to make delta useful
delta = False
# Change distro_tags as fedora-release version gets bumped
# The cpe URI should be fixed up
distro_tags = cpe:/o:fedoraproject:fedora:18 update
hash_packages = True

/etc/mash/pidora-18-rpfr-updates.mash

[pidora-18-rpfr-updates]
rpm_path = %(arch)s/
repodata_path = %(arch)s/
source_path = SRPMS
debuginfo = True
multilib = False
multilib_method = devel
tag = f18-rpfr-updates
inherit = False

# Unsigned is temporary
strict_keys = True
#keys = 34E166FA, F8DF67E6, A82BA4B7, 069C8460, 97A1071F, E8E40FDE, 57BBCCBA, D22E77F2, 4EBFC273, 0B86274E, 6DF2196F, DF9B0AE9
#keys = CE8D31E3
keys = f1590cd5

repoviewurl = http://download.fedoraproject.org/pub/fedora-secondary/development/18/%(arch)s/os/
repoviewtitle = "Fedora Branched - %(arch)s"
arches = armhfp
# armv6 doesn't have horsepower to make delta useful
delta = False
# Change distro_tags as fedora-release version gets bumped
# The cpe URI should be fixed up
distro_tags = cpe:/o:fedoraproject:fedora:18 update
hash_packages = True

/etc/mash/pidora-18-rpfr-updates-testing.mash

[pidora-18-rpfr-updates-testing]
rpm_path = %(arch)s/
repodata_path = %(arch)s/
source_path = SRPMS
debuginfo = True
multilib = False
multilib_method = devel
tag = f18-rpfr-updates-testing
inherit = False

# Unsigned is temporary
strict_keys = True
#keys = 34E166FA, F8DF67E6, A82BA4B7, 069C8460, 97A1071F, E8E40FDE, 57BBCCBA, D22E77F2, 4EBFC273, 0B86274E, 6DF2196F, DF9B0AE9
#keys = CE8D31E3
keys = f1590cd5

repoviewurl = http://download.fedoraproject.org/pub/fedora-secondary/development/18/%(arch)s/os/
repoviewtitle = "Fedora Branched - %(arch)s"
arches = armhfp
# armv6 doesn't have horsepower to make delta useful
delta = False
# Change distro_tags as fedora-release version gets bumped
# The cpe URI should be fixed up
distro_tags = cpe:/o:fedoraproject:fedora:18 update
hash_packages = True

/etc/mash/mash.conf

[defaults]
configdir = /etc/mash
buildhost = http://japan.proximity.on.ca/kojihub
repodir = file:///mnt/koji
use_sqlite = True
use_repoview = False

/usr/local/bin/mashrun-pidora-18

#!/bin/bash
#
# mashrun-pidora18 :: Compose a Pidora 18 repository
#

DISTNAME=pidora
RELEASE=18

# Default mash output dir is unique by dates (YYYYMMDD)
BASEDIR=/mnt/koji/mash
MASHDIR=${BASEDIR}/${DISTNAME}-${RELEASE}-$(date +%Y%m%d)

# If there is already a compose output directory for today,
# append -HHMMSS
if [ -d ${MASHDIR} ] # If YYYYMMDD exists, add -HHMMSS
then
    MASHDIR=${MASHDIR}-$(date +%H%M%S)
fi
mkdir -p ${MASHDIR}

{

exec 2>&1

echo "Composing to ${MASHDIR}..."

time mash -o ${MASHDIR} ${DISTNAME}-${RELEASE} -f /usr/local/mash/comps-${DISTNAME}-${RELEASE}.xml.gz

rm ${BASEDIR}/${DISTNAME}-${RELEASE}-latest
ln -s ${MASHDIR} ${BASEDIR}/${DISTNAME}-${RELEASE}-latest

echo
echo "Warnings about unsigned packages are temporarily suppressed from email."
echo "Refer to ${MASHDIR}/mash.log for warnings."

}| cat -v | tee ${MASHDIR}/mash.log | fgrep -v '(signed with no key)' | mail -s "${DISTNAME}-${RELEASE} Mash Run" mashrun-alert

# The 'cat -v' above is there to prevent an unprintable code from throwing off the
# mail command.

Mash Command Line Execution

  • /usr/local/bin/mashrun-pidora-18

Rsync to Pidora 18 Repositories

  • ssh pidorapr@pidora.ca
  • ~/bin/rsync-japan

Composing Images

Before you can attempt to run a compose you should check to make sure that the following packages are installed:

  • anaconda
  • lorax

SSH to the arm device you want to run the compose on. chroot into the armv6hl directory on the arm device.

  • chroot /root/f17v6

cd into the livemedia directory or where ever your kickstart file is located.

  • cd /livemedia

Run the provided Livemedia-Creator command provided below to start the compose process.


Example Kickstart File

http://scotland.proximity.on.ca/raspberrypi/test-releases/pidora-18/pidora.ks

Example Livemedia-Creator Command

  • livemedia-creator --make-disk --no-virt --image-only --keep-image --ks=rpfr-18.ks

Estimated time for compilation (27mins)

  • command details

-h, --help For more info on commands
--make-disk Build a disk image
--no-virt Use Anaconda's image install instead of virt-install
--image-only Exit after creating disk image.
--ks KS Kickstart file defining the install.

Example Post Script Execution

You may have more than one %post section, which can be useful for cases where some post-installation scripts need to be run in the chroot and others that need access outside the chroot.

%post section is required to be closed with a corresponding %end

Post Section Options

  • command details

--nochroot
Allows you to specify commands that you would like to run outside of the chroot environment.

--interpreter /usr/bin/python
Allows you to specify a different scripting language, such as Python. Replace /usr/bin/python with the scripting language of your choice.

--erroronfail
If the post-installation script fails, this option will cause an error dialog to be displayed and will halt installation. The error message will direct you to where the cause of the failure is logged.

--log=


Example Post Installation Script "Creating/Modifying the fstab"

cat << EOF > /etc/fstab
LABEL="rootfs" / ext4 defaults,noatime 1 1
LABEL="boot" /boot vfat noauto,comment=systemd.automount 1 2
EOF

Retrieved from "https://wiki.cdot.senecapolytechnic.ca/w/index.php?title=Pidora_SOP&oldid=99600"