Difference between revisions of "User:Paul.W/august2010"

From CDOT Wiki
Jump to: navigation, search
 
(16 intermediate revisions by the same user not shown)
Line 69: Line 69:
  
 
* checked builds from previous night, script stopped with same error, attempt to fix.
 
* checked builds from previous night, script stopped with same error, attempt to fix.
 +
* backup of sql database for Bacula not working, created a script to manually dump the bacula db, create folder with current date and insert sql backup.
 +
* problem discovered with arm6, filesystem seems to be corrupted. Taken offline.
 +
* troubleshooting kernel, tried new kernel, same issue with usb not being availible.
 +
 +
'''18 Wednesday'''
 +
 +
* arm6 filesystem corrupted, unreadable on another machine. Used another card to get the builder back online. Configured, back online and ready for tasks.
 +
* arm6 not processing tasks, after troubleshooting error was permissions on the filesystem. This change needs to be made each time rootfs used as perms are not correct.
 +
* test data connections in 1040, not connected to network, outlets dead.
 +
* cancel remaining jobs in queue to start build-previous-ct2.py (reasoning - rather then wait a second time for jobs that fail to build, cancel queue to requeue those packages.)
 +
 +
'''19 Thursday'''
 +
 +
* analyze build failures, looking for packages that could be imported to reduce the failures.
 +
* read blog created by maintainers of previous arm koji
 +
* wrote blog on current arm status.
 +
* started self review on mkrootfs rpm before submission.
 +
 +
'''20 Friday'''
 +
 +
* space issues on hongkong, cleared out all rpms that have already been uploaded into koji from /tmp directories. Also resized root partition adding an additional 40G.
 +
* troubleshooting issue on hongkong. some commands will not return a prompt, ie 'll /' , 'df -hT'. Issue turned out to be an nfs share that was no longer shared with HongKong.
 +
* Made changes to mkrootfs package, attempting to allow armv5tel rootfs to be created on any machine, not just another arm machine.
 +
* build of perl-version-0.82-1.fc12 has stopped all builds on koji, cannot init build root due to version conflict between perl and perl-version. I have untagged the offending package and taken all builders offline to allow new repos to be generated. Will then attempted to import perl-version-0.82-1.fc13 in an attempt to fix. Asked in the #koji channel for assistance, no response yet.
 +
 +
'''23 Monday'''
 +
 +
* troubleshooting build issues. Cannot initiate build root after builds of perl-version. Untagged perl version, create repo task finished, builds continued.
 +
* now issue with perl-Compress-Raw-Bzip2-2.020-2.fc13, untagged and waited for create repo. Issue still exists. All builders taken offline until resolved.
 +
* untagged perl-Compress-Raw-Zlib-2.024-2.fc12, this fixed the problem. Will have to revisit why this started.
 +
* all builders offline due to power outage, all booted fine when reset, except arm5, arm6. Issue seems to be with certs. Ran cert revocation script, still not coming online.
 +
 +
'''24 Monday'''
 +
 +
* both arm5 and arm6 back online. Issue was date on arm5 and reference to arm.koji.fedoraproject.org. Added new address to hosts file, working now.
 +
* all builders offline, hongkong offline. UPS powered off for an unknown reason. Rebooted and manually brought builders online.
 +
* create new rootfs cards for other builders, or to be used as replacement cards.
 +
* configure arm7.
 +
* review failed packages for reasons, importing rpms to prevent on second run.
 +
 +
'''25 Tuesday'''
 +
 +
* continued to work on failed builds, builder issues.
 +
* prep for Dennis' visit.
 +
 +
'''26-28 Aug'''
 +
 +
* worked with visiting RHCE Dennis Gilmore. Planning Fedora Arm in the future, next steps etc.
 +
 +
'''30 Monday'''
 +
 +
* continued to add rpms needed to resolve dep solving problems with builds.
 +
 +
'''31 Tuesday'''
 +
 +
* patched glibc, build attempts of newer versions.

Latest revision as of 12:55, 31 August 2010


03 Tuesday

  • went to Markham campus to fill out paperwork for HR.
  • troubleshooting builds. Packages needing gcc, which needs binutils, builds to solve dependencies.
  • meeting with Chris.

04 Wednesday

  • troubleshooting build failures, gcc not building.
  • troubleshooting space issue on arm5, arm6. No root fs mounted
  • work on solving gcc deps.
  • proposal for fsoss

05 Thursday

  • attempting to track down issue with gcc
  • builds of additional packages required
  • edit build-previous.py script to change naming

06 Friday

  • work on back procedure.
  • originally attempting to use rsync and ssh keys, but changed due to security.
  • started to use bacula

09 Monday

  • checked logs for information regarding build failure of gcc
  • implemented bacula, problems with version mismatch, resolved.
  • edit gcc spec file
  • spoke with Dawn regarding IO position.

10 Tuesday

  • bacula working for backups, tweaking of backup management
  • edit gcc again, another failed build attempt
  • final test configuration of bacula completed. Backup scheduled to run weekly full backups every sunday at 9pm, and nightly incremental backups at 11:10PM. File being backed up from hongkong include - /etc, /share/koji, /var/lib/pgsql. Later tweaks will include postgresql dump to be backed up instead of the entire directory. Back ups are compressed with gzip for travel across the network.
  • packaging of mkrootfs completed however issues running script. Appears to be intended to run on arm machine rather then x86.

11 Wednesday

  • backup job failed to run, no log information as to why, troubleshooting.
  • backup jobs edited, problem should now be resolved. Full backup completed. Compressed 27G. Incremental backups each night at 11:05, full backups every sunday at 9pm. Backups are kept for 30 days.
  • completed mkrootfs-f12 packaging and testing. works on an arm based system.

12 Thursday

  • tweak to mkrootfs rpm to include repo, edit repo so not enabled.
  • small tweaks to bacula config files as backup catalog job was not completing successfully.
  • upgraded uboot on experimental guru.
  • loaded new kernel Chris compiled.
  • plug boots!

13 Friday

  • tracking down build failures, reasons, where needed uploading new srpm
  • problems booting fedora from sd card with new uboot and kernel, will revisit.
  • rebuilds of select packages to fix blocking issues.

16 Monday

  • tracking build problems, rebuilds of select packages.
  • edit python script to prevent error occurring right after pydict. FIXED!
  • attempting to boot fedora from sdcard on guruplug. Change uboot, issue still exists.

17 Tuesday

  • checked builds from previous night, script stopped with same error, attempt to fix.
  • backup of sql database for Bacula not working, created a script to manually dump the bacula db, create folder with current date and insert sql backup.
  • problem discovered with arm6, filesystem seems to be corrupted. Taken offline.
  • troubleshooting kernel, tried new kernel, same issue with usb not being availible.

18 Wednesday

  • arm6 filesystem corrupted, unreadable on another machine. Used another card to get the builder back online. Configured, back online and ready for tasks.
  • arm6 not processing tasks, after troubleshooting error was permissions on the filesystem. This change needs to be made each time rootfs used as perms are not correct.
  • test data connections in 1040, not connected to network, outlets dead.
  • cancel remaining jobs in queue to start build-previous-ct2.py (reasoning - rather then wait a second time for jobs that fail to build, cancel queue to requeue those packages.)

19 Thursday

  • analyze build failures, looking for packages that could be imported to reduce the failures.
  • read blog created by maintainers of previous arm koji
  • wrote blog on current arm status.
  • started self review on mkrootfs rpm before submission.

20 Friday

  • space issues on hongkong, cleared out all rpms that have already been uploaded into koji from /tmp directories. Also resized root partition adding an additional 40G.
  • troubleshooting issue on hongkong. some commands will not return a prompt, ie 'll /' , 'df -hT'. Issue turned out to be an nfs share that was no longer shared with HongKong.
  • Made changes to mkrootfs package, attempting to allow armv5tel rootfs to be created on any machine, not just another arm machine.
  • build of perl-version-0.82-1.fc12 has stopped all builds on koji, cannot init build root due to version conflict between perl and perl-version. I have untagged the offending package and taken all builders offline to allow new repos to be generated. Will then attempted to import perl-version-0.82-1.fc13 in an attempt to fix. Asked in the #koji channel for assistance, no response yet.

23 Monday

  • troubleshooting build issues. Cannot initiate build root after builds of perl-version. Untagged perl version, create repo task finished, builds continued.
  • now issue with perl-Compress-Raw-Bzip2-2.020-2.fc13, untagged and waited for create repo. Issue still exists. All builders taken offline until resolved.
  • untagged perl-Compress-Raw-Zlib-2.024-2.fc12, this fixed the problem. Will have to revisit why this started.
  • all builders offline due to power outage, all booted fine when reset, except arm5, arm6. Issue seems to be with certs. Ran cert revocation script, still not coming online.

24 Monday

  • both arm5 and arm6 back online. Issue was date on arm5 and reference to arm.koji.fedoraproject.org. Added new address to hosts file, working now.
  • all builders offline, hongkong offline. UPS powered off for an unknown reason. Rebooted and manually brought builders online.
  • create new rootfs cards for other builders, or to be used as replacement cards.
  • configure arm7.
  • review failed packages for reasons, importing rpms to prevent on second run.

25 Tuesday

  • continued to work on failed builds, builder issues.
  • prep for Dennis' visit.

26-28 Aug

  • worked with visiting RHCE Dennis Gilmore. Planning Fedora Arm in the future, next steps etc.

30 Monday

  • continued to add rpms needed to resolve dep solving problems with builds.

31 Tuesday

  • patched glibc, build attempts of newer versions.