Winter 2009 SBR600 Weekly Schedule

Please note:

• The schedule here is tentative.
• Week-by-week details will be filled in as the course progresses.

Week 1 (January 12) - Introduction

Tuesday

• Welcome
• Introductions
• Intro to Build & Release
  • Brief overview of the process
    • Versioning & repository systems
    • Compilation
    • Testing
    • Packaging
    • Compositing
    • Release
    • Distribution
    • Mirroring
  • These steps vary according to the particular project/product. For example, when distributing software physically, "Release" means performing a RTM, where the final "gold disk" is sent to the duplicating house to be mass-produced; but when distributing software electronically, "Release" means sending the software to the online distribution system. The sequence of steps also varies between projects/products.
• Course Layout
  • Project-based course
  • Working with Open Source
  • Working with the Fedora Project
• Communication Tools
• Course Outline
• Visit the CDOT Area

Thursday

• Makefile Basics
  • Targets, Dependencies, and Commands
  • Implied rules (e.g., .o files)
  • Examples

Readings/Resources

• Two simple makefile examples

ToDo:

Communication Lab: By Thursday, January 15, Set up your accounts (wiki, IRC, FAS2).

• Create a blog post which will appear on the OpenSource@Seneca Planet, containing:
  • A portion of an IRC conversation you've had with someone on a Fedora or Seneca IRC channel.
  • A link to your User page on the Seneca wiki
  • A link to your User page on the Fedora wiki
• Add an entry to the Winter 2009 SBR600 Participants page

Lab 1: By Wednesday, January 21:

• Build 2 packages from Source
  • The NLED editor from http://cdot.senecac.on.ca
  • Any package that uses autoconf -- SourceForge might be a good place to look for such packages.
• Blog about the experience.

Week 2 (January 19) - Overview of the Build and Release Processs

(Classes cancelled due to professor's illness)

• Complete ToDo items from week 1 if not already done.

Week 3 (January 26) - Creating RPM Packages

RPM Packages

• Purpose
• What's in an RPM package file
  • Metadata
    • What the package provides
    • Dependencies
    • Packager, date, license, summary, description, ...
  • Digital signature
  • Software
  • Data
    • Fonts
    • Icons
    • Sample data
  • Documentation
  • Configuration files
  • Setup scripts
    • Pre-install
    • Post-install
    • Pre-uninstall
    • Post-uninstall
    • Triggers

The RPM Database

• Purpose of the database
• Querying the RPM database
  • rpm -q

Creating Packages

• Packaging scenarios
• Setting up a Packaging Environment
  • Needed packages
    • rpm-build
    • rpmdevtools
    • rpmlint
  • Setting up the RPM tree
    • run rpmdev-setuptree
• Taking a look at existing source RPMS (useful as examples)
  • Installing
    • yumdownloader --source nameofpackage
    • rpm -i nameofpackage.src.rpm
    • Source will be in ~/rpmbuild/SOURCES and specfile will be in ~/rpmbuild/SPECS
  • Examine the specfile
  • Rebuild on the local machine
    • rpmbuild --rebuild nameofpackage.src.rpm
  • Building from the spec file
    • cd ~/rpmbuild/SPECS; rpmbuild -ba nameofpackage.spec

Writing a specfile

• Run rpmdev-newspec packagename in ~/rpmbuild/SPECS
• Edit the skeleton specfile.
• Test it: rpmbuild -ba packagename.spec

Layout of a specfile

• Basic Sections

1. preamble - basic metadata
2.  %prep - commands to prepare the package for building
3.  %build - commands to build the package
4.  %install - commands to install the built files
5.  %check - commands to check/test the built files (optional, often not included)
6.  %clean - commands to clean up the disk space
7.  %files - list of files to be included in the pacakge
8.  %changelog - record of the package's change-history

• Scriptlets
  •  %pre
  •  %post
  •  %preun
  •  %postun
• Macros
  •  %{_tmppath}
  •  %{buildroot}
  •  %{_bindir}
  •  %{_datadir}
  •  %{_mandir}
  •  %{_smp_flags}
  •  %setup
  •  %configure
  •  %makeinstall

Creating a Simple Package

• NLED
• Writing the specfile
• Testing the specfile
• Using rpmlint

Resources

• Fedora Package Maintainers page
  • Packaging How-To

See also "Fedora Linux" chapter 5 (see Seneca Library website > eBooks > View All > Safari > Fedora Linux).

TODO:

• Take the software you compiled last week and package it (not Nled!). Blog about the experience. Include a link to your source RPM (and optionally your binary RPM) from your blog.

Week 4 (February 2) - Working with Open Source Communities and Project Release 0.0

• Free Software and Open Source
  • Origins
    • Early computing
    • Public domain software
    • Richard Stallman and the Free Software Foundation
    • The Open Source Initiative
  • Current state
    • Becoming a dominant model in the industry
    • Prominent projects: Linux, Apache, OpenOffice.org, Eclipse, ...
  • What Open Source Means
    • Working in Public
    • Collaborating
    • Thinking and working globally
    • Reusing code, thinking smart, being nimble
  • Open Source business models
    • Support
    • Customization
    • Sponsorship

• Release and Build on Open Source
  • Looks like release and build in proprietary projects
  • More developers
  • More casual contributors
  • Internet instead of private network
    • Security issues

• The Fedora Project
  • Red Hat Linux
  • Fedora Core and Fedora Extras
  • Fedora
  • "Dedicated to the rapid advancement of Open Source"
  • The 4 F's
    • Features
    • Friends
    • Freedom
    • First

• The Structure of Fedora
  • Communication tools
  • Funding
  • How software gets into Fedora
    • Packagers
    • Sponsors
    • Infrastructure
  • Face to face
    • FUDCons
    • FADs

• Example from class: md5deep-3.2-1.fc10.src.rpm

ToDo:

1. Create an RPM.
2. Test it with rpmlint (spec, source RPM, binary RPM)
3. Test it with mock

Week 6 (February 16) - Basic Build II

• We analyzed the mass rebuild results so far to find the slowest 15 packages: Winter 2009 SBR600 Packages of Interest
• Questions:
  1. Why are 2 books in the top-4 slowest packages to build?
  2. Why are there so many Java packages in the top 15? Are they big, or slow to compile, or ?
  3. How much can we speed up packages in the top 15 using distcc?

ToDo:

• Prepare one of the CDOT machines for a non-mock build of a slow package. See Preparing to test distcc

Week 8 (March 9) - Server Farms & Distributed Processing I

Machine codes:

• [c] indicates client machine (where you are starting the build)
• [s] indicates server machine (running distccd)

Prerequisites for building with distcc:

• [c] distcc plus any requirements for a normal build (e.g., rpmbuild, cc, make, BuildRequires as specified in the spec file, etc)
• [s] distcc-server plus compilers (libraries etc. are not required)

Basic instructions on building with distcc:

1. [c] create ~/bin and populate it with symbolic links pointing to /usr/bin/distcc for each of the compilers your package might use -- gcc, cc, g++, c++
2. [c] add that directory to the start of your search path: PATH=${HOME}/bin:$PATH
3. [c] edit /etc/distcc/hosts to list the names of the hosts which will be servers (Recommendation: if your server and client are both on the GigE network -- scotland, ireland, china, india, or australia -- then append a "2" to the hostname to use the GigE interface (e.g., "scotland2")).
4. [s] on each server, allow access through the firewall for the distcc port (firewalls may be disabled on some of the machines alread)
5. [s] on each server, start distccd: distccd --daemon --allow IP-addresses-of-client-system
6. [c] start the build: time rpmbuild --rebuild package.src.rpm

ToDo:

• Blog about the build times with and without distcc.
• Blog an analysis of the /var/f10source/buildall.log file on Scotland.

Week 9 (March 16) - Server Farms & Distributed Processing II

Signing RPM packages

An RPM signature, like the digital signature used on many other software-signing systems, is a private key encryption of a checksum. RPM uses the GPG libraries for signing.

1. Create a GPG key: gpg --gen-key
2. Add the e-mail address associated with your gpg key to the %_gpg_name macro in ~/.rpmmacros -- the line will look like this: %_gpg_name "e-mail-address
3. Find (or make) some packages to put in your repository. Make sure that the epoch-version-release is higher than that of any package with the same name in the Fedora repositories.
4. Sign those packages with: rpm --addsign packagefile

Creating a YUM repository

A yum repository is just a directory of packages and some metadata.

1. Create a directory that can be served. The protocol used to serve that directory could be http, ftp, nfs, or something else (the files can be served by putting them on a DVD too!). For http, create the directory within /var/www/html
2. Put your signed packages in that directory.
3. Create the repository metadata for that directory: createrepo /name/of/directory

Notice that the repository metadata will be placed in a directory named repodata

Testing

1. Create a new repository file in /etc/yum.repos.d by copying and modifying an existing file in that directory. Keep gpgcheck=1 but comment out the gpgkey file.
2. Confirm that you cannot install from that repository using yum.
3. Uncomment the gpgkey line, and point it to a new file within /etc/pki/rpm-gpg/
4. Create that file by running (as your regular user): gpg --export --armour e-mail-address and saving the output
5. Confirm that you can now install from your repository. You should be asked whether you wish to import the key for your repo.

ToDo:

1. Create an RPM package that will install your repository configuration file and the key.
2. Test it.
3. Blog about this lab, and include a link to your repository RPM package.