Difference between revisions of "Winter 2011 SBR600 Weekly Schedule"

From CDOT Wiki
Jump to: navigation, search
(→‎rpmlint: Checking for a clean spec file, SRPM, and RPM)
(→‎Release Dates, Presentations, and Quizzes)
 
(47 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
{{Chris Tyler Draft}}[[Category:Winter 2011 SBR600]]
+
[[Category:Winter 2011 SBR600]]
−
{{Admon/important|Tentative Schedule - Winter 2011|Please note that the schedule here is tentative. Week-by-week details will be filled in as the course progresses.}}
+
{{Admon/important|Tentative Schedule - Winter 2011|Please note that the schedule here is tentative. Week-by-week details will be filled in as the course progresses. This content is also being refactored for easier navigation.}}
  
 
Previous semester: [[Fall 2010 SBR600 Weekly Schedule]]
 
Previous semester: [[Fall 2010 SBR600 Weekly Schedule]]
Line 10: Line 10:
 
* Introductions
 
* Introductions
  
−
=== Intro to Build & Release ===
+
=== Intro to SBR600 - Software Build & Release ===
−
* Brief overview of the process - These steps vary according to the particular project/product. For example, when distributing software physically, "Release" means performing a release-to-manufacturing (RTM), where the final "gold disk" is sent to the duplicating house to be mass-produced; but when distributing software electronically, "Release" means sending the software to the online distribution system. The sequence of steps also varies between projects/products.
+
* [[SBR600 Overview of the Build and Release Process|Brief overview of the Build & Release process]]
−
** Version Control
 
−
** Compilation
 
−
** Testing
 
−
** Packaging
 
−
** Compositing
 
−
** Release
 
−
** Distribution
 
−
** Mirroring
 
 
* Introduction to Free Software and Open Source
 
* Introduction to Free Software and Open Source
 
** The [http://fsf.org Free Software Foundation]
 
** The [http://fsf.org Free Software Foundation]
Line 32: Line 24:
 
** [[SBR600 Communication Tools|Communication Tools]]
 
** [[SBR600 Communication Tools|Communication Tools]]
 
** [[SBR600|Course Outline]]
 
** [[SBR600|Course Outline]]
−
* Visit to the [[CDOT]] Area
+
* How this Course Works
 +
** [[SBR600 Communication Tools]]
 +
** How coursework is submitted in SBR600
 +
* [[FUDCon Tempe 2011]] trip
 +
** Selection of attendees
 +
* Visit to the [[CDOT Area]]
  
 
=== To Do ===
 
=== To Do ===
Line 43: Line 40:
  
 
=== Using ''make'' ===
 
=== Using ''make'' ===
−
* Makefile Basics
+
 
−
** Targets, Dependencies, and Commands
+
* [[SBR600 make and Makefiles|make and Makefiles]]
−
** Implied rules (e.g., .o files)
 
−
** Examples
 
−
* Building software from a source tarball using a makefile
 
  
 
=== RPM Packages ===
 
=== RPM Packages ===
−
* Purpose
+
* Differences between managing RPMS and Installing from Source
−
* What's in an RPM package file
+
** RPMS provide a database of installed software
−
** Metadata
+
*** Let you determine what's installed
−
*** What the package provides
+
*** Automatic management of dependencies
−
*** Dependencies
+
*** Identify the origin of files
−
*** Packager, date, license, summary, description, ...
+
*** Permit easy update or removal
−
** Digital signature
+
*** Enable you to verify installation (useful for spotting file corruption and intrusions)
−
** Software
+
* [[RPM Package Contents|Contents of an RPM Package]]
−
** Data
 
−
*** Fonts
 
−
*** Icons
 
−
*** Sample data
 
−
** Documentation
 
−
** Configuration files
 
−
** Setup scripts
 
−
*** Pre-install
 
−
*** Post-install
 
−
*** Pre-uninstall
 
−
*** Post-uninstall
 
−
*** Triggers
 
  
 
=== The RPM Database ===
 
=== The RPM Database ===
−
* Purpose of the database
+
 
 
* [[RPM Queries|Querying the RPM database]]
 
* [[RPM Queries|Querying the RPM database]]
−
** rpm -q
 
  
−
=== Creating Packages ===
+
=== Layout of a specfile ===
−
* Packaging scenarios
 
−
* Setting up a Packaging Environment
 
−
** Needed packages
 
−
*** rpm-build
 
−
*** rpmdevtools
 
−
*** rpmlint
 
−
*** yum-utils
 
−
** Setting up the RPM tree
 
−
*** run <code>rpmdev-setuptree</code>
 
−
* Taking a look at existing source RPMS (useful as examples)
 
−
** Installing
 
−
*** <code>yumdownloader --source <i>nameofpackage</i></code>
 
−
*** <code>rpm -i <i>nameofpackage</i>.src.rpm</code>
 
−
*** Source will be in ~/rpmbuild/SOURCES and specfile will be in ~/rpmbuild/SPECS
 
−
** Examine the specfile
 
−
** Rebuild on the local machine -- takes a source RPM (SRPM) as input, produces binary and source RPMs as output
 
−
*** <code>rpmbuild --rebuild <i>nameofpackage</i>.src.rpm</code>
 
−
** Building from the spec file -- takes a specfile and source code as input, produces binary and source RPMs as output
 
−
*** <code>cd ~/rpmbuild/SPECS; rpmbuild -ba <i>nameofpackage</i>.spec</code>
 
  
−
=== Writing a specfile ===
+
* [[Contents of an RPM spec file]]
  
−
* Run <code>rpmdev-newspec ''packagename''</code> in ~/rpmbuild/SPECS
+
=== Creating Packages ===
−
* Edit the skeleton specfile.
 
−
* Test it: <code>rpmbuild -ba ''packagename''.spec</code>
 
−
** If successful, output will be binary RPM(s) in ~/rpmbuild/RPMS and source RPM in ~/rpmbuild/SRPMS
 
−
*** Can install binary RPM with: <code>rpm -i ''rpmname''</code>
 
−
** If unsuccessful, read the error messages carefully.
 
−
* Check it with rpmlint: <code>rpmlint ''packagename''*</code>
 
−
** Remember to check the spec file as well as the binary and source RPMs.
 
−
** Correct any errors found.
 
  
−
=== Layout of a specfile ===
+
* [[RPM Packaging Process]]
−
 
 
−
* Basic Sections
 
−
# preamble - basic metadata
 
−
# %prep - commands to prepare the package for building
 
−
# %build - commands to build the package
 
−
# %install - commands to install the built files
 
−
# %check - commands to check/test the built files (optional, often not included)
 
−
# %clean - commands to clean up the disk space
 
−
# %files - list of files to be included in the pacakge
 
−
# %changelog - record of the package's change-history
 
−
* Scriptlets
 
−
** %pre -- run before installation
 
−
** %post -- run after installation
 
−
** %preun -- run before uninstallation
 
−
** %postun -- run after uninstallation
 
−
*** Note that during upgrade, the installation of the new package is considered to happen before the removal of the old package.
 
−
* Macros
 
−
** %{_tmppath}
 
−
** %{buildroot}
 
−
** %{_bindir}
 
−
** %{_datadir}
 
−
** %{_mandir}
 
−
** %{_smp_flags}
 
−
** %setup
 
−
** %configure
 
−
** %makeinstall
 
  
 
=== Demo: Creating a Simple Package ===
 
=== Demo: Creating a Simple Package ===
Line 141: Line 69:
 
* Writing the specfile
 
* Writing the specfile
 
* Testing the specfile
 
* Testing the specfile
−
* Using rpmlint
+
** Using rpmlint
  
 
=== Resources ===
 
=== Resources ===
Line 149: Line 77:
 
** [[:fedora:PackageMaintainers/CreatingPackageHowTo|Packaging How-To]]
 
** [[:fedora:PackageMaintainers/CreatingPackageHowTo|Packaging How-To]]
 
* ''Fedora Linux'' chapter 5 (see Seneca Library website > eBooks > View All > Safari > Fedora Linux).
 
* ''Fedora Linux'' chapter 5 (see Seneca Library website > eBooks > View All > Safari > Fedora Linux).
 +
* rpmlint
 +
** [[:fedora:Common Rpmlint issues|Fedora Common Rpmlint issues]]
  
 
=== To Do ===
 
=== To Do ===
Line 154: Line 84:
 
By '''Monday, January 17''':
 
By '''Monday, January 17''':
 
# [[SBR600 RPM-Writing Lab|RPM-Writing Lab]]
 
# [[SBR600 RPM-Writing Lab|RPM-Writing Lab]]
−
# Send your [[SSH]] public key to [[User:Chris Tyler|your professor]]
+
# Send your [[SSH]] public key to [[User:Chris Tyler|your professor]] so he can create accounts for you on the [[CDOT Development Systems]].
  
 
= Week 2 (January 17) - Using Mock and Koji =
 
= Week 2 (January 17) - Using Mock and Koji =
Line 160: Line 90:
 
== Tuesday ==
 
== Tuesday ==
  
−
=== mock: Testing BuildRequires ===
+
=== Project Selection ===
 +
 
 +
This is a project-based course. These projects involve participation in an open-source community.
 +
* Projects are listed on the [[SBR600 Potential Projects]] page.
 +
* Select two or three projects that are of interest to you.
 +
** Do some initial research into what the project involves.
 +
*** Find out who to talk to in the community (start with the initial contacts listed on the project description)
 +
*** See what work has already been done related to that project. Check the Seneca wiki for work by previous SBR600 semesters, the upstream project's wiki and mailing list archives for information about the current state of the project, and the web for related information (similar projects being done by other groups).
 +
*** Join the mailing lists and IRC channels of the upstream community.
 +
** Update the [[Winter 2011 SBR600 Participants]] table with your project information, according to the instructions at the top of that page.
 +
* [[User:Chris Tyler|Your professor]] will approve your project selection via the [[Winter 2011 SBR600 Participants||participants page]].
 +
* Link your project title on the [[Winter 2011 SBR600 Participants|participants page]] to a page of the same name to create a project page. Copy the contents of the [[Sample Project]] page to your project page and fill in the details.
  
−
It's often difficult to get the BuildRequires in a spec file exactly right, because it's easy to overlook packages that are coincidentally installed on the machine. <code>mock</code> is a tool that uses the chroot system call to create a "cleanroom" environment for the build, so that any missing BuildRequires cause the build to fail.
+
Over the next 2 weeks, finalize your project plans and get started on your project:
 +
* The project page must be filled in, including your 0.1, 0.2, and 0.3 targets.
 +
** Release 0.1: Proof of concept (e.g., a first draft of a package, a basic script, infrastructure set up on a test system) - Note that this must include the release of ''something'', not just research, and must be done in consultation with the community.
 +
** Release 0.2: Initial working state - Whatever you are working on -- package, script, infrastructure configuration -- should be working, although it may not be feature-complete, fully deployed, or fully documented. Feedback from the community should be solicited. If there is a review process required to submit upstream, it should be started.
 +
** Release 0.3: Completed working state - The work is complete and documented. Any upstream review, whether formal or informal, has been completed, feedback has been incorporated into the project, and the work has been committed been
 +
* You must have a strategy in place for reaching your targets.
 +
You will make a brief (3-5 minute) presentation of your project plans on '''Thursday, February 3'''.
  
−
{{Admon/note|The '''mock''' Group|To use mock, you will need to add yourself to the '''mock''' group: <code>usermod -G mock ''yourUserName''</code>}}
+
=== ToDo ===
  
−
To build with mock:
+
* Finish [[SBR600 RPM-Writing Lab|building your two RPMs]] before Thursday's class (we'll use them in the lab)
 +
* Send your [[SSH]] key to [[User:Chris Tyler|your prof]] by Wednesday (will be needed for Thursday's lab)
 +
* Review the [[SBR600 Potential Projects]] and start researching the top 2-3 possibilities.
  
−
mock -r ''fedora-13-x86_64'' --rebuild ''foo*.src.rpm''
+
== Thursday ==
  
−
The value ''fedora-13-x86_64'' may be changed to any of the config files found in <code>/etc/mock</code> in order to test building based on the libraries and available packages for a specific architecture and Fedora release. If errors are reported, review the log files (see the mock output to determine the directory containing the log files).
+
=== mock: Testing BuildRequires ===
  
−
== Thursday ==
+
It's often difficult to get the BuildRequires in a spec file exactly right, because it's easy to overlook packages that are coincidentally installed on the machine. ''Mock'' is used to test that the BuildRequires for a package are complete and accurate, by creating a bare-bones [http://en.wikipedia.org/wiki/Chroot chroot] environment containing only the [[:fedora:Packaging/Guidelines#Exceptions_2|basic build packages]] plus any packages indicated by BuildRequires lines in the spec file.
  
 
=== koji: Testing multiple architectures ===
 
=== koji: Testing multiple architectures ===
Line 178: Line 127:
 
<code>koji</code> is a client-server system which allows you to queue builds within the Fedora build farm. This permits you to test whether your package builds on several different architectures, which is especially useful when you don't otherwise have access to the machines of that architecture.
 
<code>koji</code> is a client-server system which allows you to queue builds within the Fedora build farm. This permits you to test whether your package builds on several different architectures, which is especially useful when you don't otherwise have access to the machines of that architecture.
  
−
{{Admon/note|Koji Setup|Note that koji requires some setup, particularly for certificate-based authentication. See the [[:fedora:PackageMaintainers/UsingKoji|UsingKoji]] page on the Fedora wiki for setup instructions.}}
+
=== Resources ===
 +
* mock
 +
** [[:fedora:Projects/Mock|Mock Project page]]
 +
** [[:fedora:Using_Mock_to_test_package_builds|Using Mock to test package builds]]
 +
* koji
 +
** [[:fedora:PackageMaintainers/UsingKoji|Using Koji]]
  
−
To queue a build for all four Fedora-supported architectures (currently i386, x86_64, ppc, and ppc64):
+
=== ToDo ===
  
−
koji build ''dist-f12'' --scratch ''foo*.src.rpm''
+
* [[SBR600 Mock and Koji Lab|Mock and Koji Lab]]
  
−
In this example, ''dist-f12'' selects the package repository (similar to the <code>-r</code> option for mock, except that the build arch is not specified); <code>--scratch</code> specifies that this is a scratch build (the results are not kept or fed to Bodhi).
+
= Week 3 (January 24) - Solving Build Issues =
  
−
To perform a build that will be passed to Bodhi for distribution, substitute <code>--rebuild</code> for <code>--scratch</code> (do not do this until you have passed the package review and sponsor procedures).
+
= Week 4 (January 31) - Project Plan Presentations =
  
−
{{Admon/note|ARM Build|You can queue an ARM build by specifying the server "arm.koji.fedoraproject.org" on the Koji command line (or in <code>/etc/koji.conf</code>).}}
+
== Tuesday ==
  
−
== Resources ==
+
''Class is cancelled. Recover from FUDCon and work on your project plan.''
  
−
* rpmlint
+
== Thursday ==
−
** [[:fedora:Common Rpmlint issues]]
 
−
* mock
 
−
** [[:fedora:Projects/Mock|Mock Project page]]
 
−
** [[:fedora:Using_Mock_to_test_package_builds]]
 
−
* koji
 
−
** [[:fedora:PackageMaintainers/UsingKoji|Using Koji]]
 
−
 
 
−
== ToDo ==
 
  
−
* Test your RPM from last week with:
+
* Project pages are due.
−
** rpmlint
+
* Be prepared to give a detailed but brief (3- to 5-minute presentation) on your project plan.
−
** mock
 
−
** koji
 
  
−
* Blog about your experience.
+
= Week 5 (February 7) - Repositories/Distributing =
−
 
 
−
= Week 3 (January 24) - Solving Build Issues =
 
−
 
 
−
= Week 4 (January 31) - Repositories/Distributing =
 
  
 
== Signing RPM packages ==
 
== Signing RPM packages ==
Line 218: Line 158:
  
 
# Create a GPG key: <code>gpg --gen-key</code>
 
# Create a GPG key: <code>gpg --gen-key</code>
−
# Add the e-mail address associated with your gpg key to the <code>%_gpg_name</code> macro in <code>~/.rpmmacros</code> -- the line will look like this: <code>%_gpg_name "<i>e-mail-address</i></code>
+
# Add the e-mail address associated with your gpg key to the <code>%_gpg_name</code> macro in <code>~/.rpmmacros</code> -- the line will look like this: <code>%_gpg_name "<i>e-mail-address</i>"</code>
 
# Find (or make) some packages to put in your repository. Make sure that the epoch-version-release is higher than that of any package with the same name in the Fedora repositories.
 
# Find (or make) some packages to put in your repository. Make sure that the epoch-version-release is higher than that of any package with the same name in the Fedora repositories.
 
# Sign those packages with: <code>rpm --addsign <i>packagefile</i></code>
 
# Sign those packages with: <code>rpm --addsign <i>packagefile</i></code>
Line 249: Line 189:
  
 
== ToDo ==
 
== ToDo ==
−
 
−
Project
 
−
# Make sure your entry in the [[Fall 2010 SBR600 Project Table]] is complete, and that your project plan on your project page is filled in (including 0.1, 0.2, and 0.3 targets).
 
  
 
Lab
 
Lab
Line 257: Line 194:
 
# Test it.
 
# Test it.
 
# Blog about this lab, and include a link to your repository RPM package.
 
# Blog about this lab, and include a link to your repository RPM package.
−
 
−
= Week 5 (February 7) - Repositories/Distributing =
 
  
 
= Weeks 6 - 13  =
 
= Weeks 6 - 13  =
 
* Project work
 
* Project work
 +
 +
= Release Dates, Presentations, and Quizzes =
 +
* March 7 - '''Release 0.1'''
 +
* March 15/17 - Pre-0.2 presentations
 +
* March 24 - Written Quiz
 +
* April 4 - '''Release 0.2'''
 +
* April 5/7 - Pre-0.3 presentations
 +
* April 11/13 (To Be Confirmed) - OCE Presentations
 +
* April 14 - Practical Quiz
 +
* April 22 - '''Release 0.3''' - DO NOT BE LATE!
  
 
<!-- = Week 8 =
 
<!-- = Week 8 =

Latest revision as of 09:39, 12 April 2011

Important.png
Tentative Schedule - Winter 2011
Please note that the schedule here is tentative. Week-by-week details will be filled in as the course progresses. This content is also being refactored for easier navigation.

Previous semester: Fall 2010 SBR600 Weekly Schedule

Week 1 (Jan 10) - Introduction, Building from Source, and Creating RPM Packages

Tuesday

Welcome

  • About this course
  • Introductions

Intro to SBR600 - Software Build & Release

To Do

By Wednesday, January 12:

  1. Communication Lab
  2. Fedora Installation
  3. Build-from-Source Lab

Thursday

Using make

RPM Packages

  • Differences between managing RPMS and Installing from Source
    • RPMS provide a database of installed software
      • Let you determine what's installed
      • Automatic management of dependencies
      • Identify the origin of files
      • Permit easy update or removal
      • Enable you to verify installation (useful for spotting file corruption and intrusions)
  • Contents of an RPM Package

The RPM Database

Layout of a specfile

Creating Packages

Demo: Creating a Simple Package

  • Writing the specfile
  • Testing the specfile
    • Using rpmlint

Resources

To Do

By Monday, January 17:

  1. RPM-Writing Lab
  2. Send your SSH public key to your professor so he can create accounts for you on the CDOT Development Systems.

Week 2 (January 17) - Using Mock and Koji

Tuesday

Project Selection

This is a project-based course. These projects involve participation in an open-source community.

  • Projects are listed on the SBR600 Potential Projects page.
  • Select two or three projects that are of interest to you.
    • Do some initial research into what the project involves.
      • Find out who to talk to in the community (start with the initial contacts listed on the project description)
      • See what work has already been done related to that project. Check the Seneca wiki for work by previous SBR600 semesters, the upstream project's wiki and mailing list archives for information about the current state of the project, and the web for related information (similar projects being done by other groups).
      • Join the mailing lists and IRC channels of the upstream community.
    • Update the Winter 2011 SBR600 Participants table with your project information, according to the instructions at the top of that page.
  • Your professor will approve your project selection via the |participants page.
  • Link your project title on the participants page to a page of the same name to create a project page. Copy the contents of the Sample Project page to your project page and fill in the details.

Over the next 2 weeks, finalize your project plans and get started on your project:

  • The project page must be filled in, including your 0.1, 0.2, and 0.3 targets.
    • Release 0.1: Proof of concept (e.g., a first draft of a package, a basic script, infrastructure set up on a test system) - Note that this must include the release of something, not just research, and must be done in consultation with the community.
    • Release 0.2: Initial working state - Whatever you are working on -- package, script, infrastructure configuration -- should be working, although it may not be feature-complete, fully deployed, or fully documented. Feedback from the community should be solicited. If there is a review process required to submit upstream, it should be started.
    • Release 0.3: Completed working state - The work is complete and documented. Any upstream review, whether formal or informal, has been completed, feedback has been incorporated into the project, and the work has been committed been
  • You must have a strategy in place for reaching your targets.

You will make a brief (3-5 minute) presentation of your project plans on Thursday, February 3.

ToDo

Thursday

mock: Testing BuildRequires

It's often difficult to get the BuildRequires in a spec file exactly right, because it's easy to overlook packages that are coincidentally installed on the machine. Mock is used to test that the BuildRequires for a package are complete and accurate, by creating a bare-bones chroot environment containing only the basic build packages plus any packages indicated by BuildRequires lines in the spec file.

koji: Testing multiple architectures

koji is a client-server system which allows you to queue builds within the Fedora build farm. This permits you to test whether your package builds on several different architectures, which is especially useful when you don't otherwise have access to the machines of that architecture.

Resources

ToDo

Week 3 (January 24) - Solving Build Issues

Week 4 (January 31) - Project Plan Presentations

Tuesday

Class is cancelled. Recover from FUDCon and work on your project plan.

Thursday

  • Project pages are due.
  • Be prepared to give a detailed but brief (3- to 5-minute presentation) on your project plan.

Week 5 (February 7) - Repositories/Distributing

Signing RPM packages

An RPM signature, like the digital signature used on many other software-signing systems, is a private key encryption of a checksum. RPM uses the GPG libraries for signing.

  1. Create a GPG key: gpg --gen-key
  2. Add the e-mail address associated with your gpg key to the %_gpg_name macro in ~/.rpmmacros -- the line will look like this: %_gpg_name "e-mail-address"
  3. Find (or make) some packages to put in your repository. Make sure that the epoch-version-release is higher than that of any package with the same name in the Fedora repositories.
  4. Sign those packages with: rpm --addsign packagefile

Creating a YUM repository

A yum repository is just a directory of packages and some metadata.

  1. Create a directory that can be served. The protocol used to serve that directory could be http, ftp, nfs, or something else (the files can be served by putting them on a DVD too!). For http, create the directory within /var/www/html
  2. Put your signed packages in that directory.
  3. Create the repository metadata for that directory: createrepo /name/of/directory

Notice that the repository metadata will be placed in a directory named repodata

Testing

  1. Create a new repository file in /etc/yum.repos.d by copying and modifying an existing file in that directory. Keep gpgcheck=1 but comment out the gpgkey file.
  2. Confirm that you cannot install from that repository using yum.
  3. Uncomment the gpgkey line, and point it to a new file within /etc/pki/rpm-gpg/
  4. Create that file by running (as your regular user): gpg --export --armour e-mail-address and saving the output
  5. Confirm that you can now install from your repository. You should be asked whether you wish to import the key for your repo.

Repository-release RPM

To make it easier for users to access your repository, create a RPM containing:

  1. Your repo file
  2. Your GPG key

Take a look at the RPMFusion release RPM for an example.

ToDo

Lab

  1. Create an RPM package that will install your repository configuration file and the key.
  2. Test it.
  3. Blog about this lab, and include a link to your repository RPM package.

Weeks 6 - 13

  • Project work

Release Dates, Presentations, and Quizzes

  • March 7 - Release 0.1
  • March 15/17 - Pre-0.2 presentations
  • March 24 - Written Quiz
  • April 4 - Release 0.2
  • April 5/7 - Pre-0.3 presentations
  • April 11/13 (To Be Confirmed) - OCE Presentations
  • April 14 - Practical Quiz
  • April 22 - Release 0.3 - DO NOT BE LATE!


Exam Week

Retrieved from "https://wiki.cdot.senecapolytechnic.ca/w/index.php?title=Winter_2011_SBR600_Weekly_Schedule&oldid=62401"