Difference between revisions of "ARMory"
Chris Tyler (talk | contribs) (→Switches) |
Chris Tyler (talk | contribs) (→Builder Configuration) |
||
| (11 intermediate revisions by the same user not shown) | |||
| Line 27: | Line 27: | ||
=== Australia / scratch === | === Australia / scratch === | ||
| − | Australia provides scratch space for the builders (currently via NFS, but iSCSI/AoE/nbd may be used in the future). It is an Intel Core2 Quad system with 8GB RAM, a 1TB disk for the operating system, and four 2TB disks in a RAID-0 array for the scratch space. | + | Australia provides scratch space for the builders (currently via NFS, but iSCSI/AoE/nbd may be used in the future). It is an Intel Core2 Quad system with 8GB RAM, a 1TB disk for the operating system, and four 2TB disks in a RAID-0 array for the scratch space. It has gigabit ethernet interfaces to the builder LAN and SeneNET. |
| + | |||
| + | === Additional x86_64 systems - createrepo === | ||
| + | |||
| + | Other x86_64 systems in CDOT which are underutilized have been configured as createrepo clients for Koji. | ||
== ARM builders == | == ARM builders == | ||
| Line 66: | Line 70: | ||
=== Miscellaneous Machines === | === Miscellaneous Machines === | ||
| − | An OpenRD, SheevaPlug, Beagleboard xM, and additional GuruPlugs - group 0. | + | An OpenRD (with 650GB SATA disk), SheevaPlug, Beagleboard xM, and additional GuruPlugs - group 0. |
== Backup == | == Backup == | ||
| Line 74: | Line 78: | ||
== Switches == | == Switches == | ||
| − | The builders are on a private LAN connected via three 24-port unmanaged gigabit switches and one 16-port fast ethernet (100 Mbps - for the fanless GuruPlugs). | + | The builders are on a private LAN connected via three 24-port unmanaged gigabit switches and one 16-port fast ethernet switch (100 Mbps - for the fanless GuruPlugs). |
| − | The x86_64 servers are connected to the campus Cisco switch ( | + | The x86_64 servers are connected to the campus Cisco switch (SeneNET) via gigabit ethernet and are externally accessible. The Cisco switch has been configured to provide a /26 VLAN. |
== UPSes == | == UPSes == | ||
| Line 82: | Line 86: | ||
The ARMory is on 3 APC UPSes, balanced for a runtime of at least 10 minutes. The UPSes are further protected by a building surge suppression system. | The ARMory is on 3 APC UPSes, balanced for a runtime of at least 10 minutes. The UPSes are further protected by a building surge suppression system. | ||
| − | == Physical | + | == Physical and Thermal Layout == |
| − | The x86_64 and | + | The x86_64 servers and the ARM builders are housed on steel wire shelving units within CDOT. Wire shelves were chosen over traditional rackmount units due to the odd form factor of many of the builders, the need to run a lot of cables, and convection cooling requirements. |
| − | The ARM farm does not have any special cooling. Temperatures have been monitored and remain in the 20-28C range. | + | The ARM farm is in a large storage room served by standard HVAC and does not have any special cooling. Temperatures have been monitored and remain in the 20-28C range, according to the cycles of the building HVAC system (reaching highest temperatures at approximately 5-6 AM local time). |
== Builder Configuration == | == Builder Configuration == | ||
| − | The builders are running a fairly standard Fedora | + | The builders are running a fairly standard Fedora rootfs, with an additional init script named "koji-config". This script sets up the koji login name, certificate, and so forth based on the hostname; on the Pandas, the hostname is manually set (because the MAC address changes each time the interface is brought up), while on other machines, the hostname is retrieved by DHCP from HongKong. |
/var/lib/mock is symlinked to /redirect/mock/lib/mock, and /var/cache/mock is symlinked to /redirect/mock/cache/mock. /redirect is a symlink to a directory selected for storage, typically a loopback filesystem stored on NFS scratch space exported from Australia and mounted at /fs0; it can also be set to an external USB drive or to the SD card. | /var/lib/mock is symlinked to /redirect/mock/lib/mock, and /var/cache/mock is symlinked to /redirect/mock/cache/mock. /redirect is a symlink to a directory selected for storage, typically a loopback filesystem stored on NFS scratch space exported from Australia and mounted at /fs0; it can also be set to an external USB drive or to the SD card. | ||
Scratch space exported from Australia is also used for swap and for /tmp/koji storage. | Scratch space exported from Australia is also used for swap and for /tmp/koji storage. | ||
| + | |||
| + | Here is the builder RPM package list: | ||
| + | * [[ARMory Panda Hardfp Package List]] | ||
| + | * [[ARMory Panda Softfp Package List]] | ||
Latest revision as of 07:33, 22 December 2011
Contents
The ARMory
The ARMory is the Fedora ARM build system hosted at Seneca. It consists of:
- x86_64 servers
- ARM builders of various types
- removable backup drives
- switches
- UPSes
x86_64 servers
All of the servers are running Fedora 15 x86_64.
HongKong / hub
HongKong operates the Koji hub and web applications. It is an Intel i7 quad-core system with 12G RAM and four 2TB SATA disks in a RAID-5 array. It has gigabit ethernet interfaces to the builder LAN and SeneNET.
Ireland / db
Ireland runs the PostgrSQL database. It is an Intel Core2 Quad (Q6600) system with 8G RAM, two 2TB SATA disks in RAID-1 for the operating system, and two 128G SATA SSDs in RAID-1 for the database. It has a gigabit ethernet connection to SeneNET.
Chile / mntkoji
Chile holds the build packages (/mnt/koji). It is an Intel i7 quad-core system with 12G RAM, two 2TB SATA disks in RAID-1 for the operating system, three 600GB SATA SSDs in RAID-0 for the packages, and a 2TB SATA disk with a mirror of the SSD array (RAID-1 writemostly/write-behind). It has gigabit ethernet interfaces to the builder LAN and SeneNET.
Australia / scratch
Australia provides scratch space for the builders (currently via NFS, but iSCSI/AoE/nbd may be used in the future). It is an Intel Core2 Quad system with 8GB RAM, a 1TB disk for the operating system, and four 2TB disks in a RAID-0 array for the scratch space. It has gigabit ethernet interfaces to the builder LAN and SeneNET.
Additional x86_64 systems - createrepo
Other x86_64 systems in CDOT which are underutilized have been configured as createrepo clients for Koji.
ARM builders
The ARM builders are identified by hostname as location-type-group-unit.
Each group contains five units. Each unit is an individual builder.
The location is cdot for all builders physically located in the ARMory at CDOT (Toronto).
Valid types are:
- panda
- guru
- smarttop
- beagleboardxm
- openrd
- sheeva
armv7hl builders will log in to the koji hub with the suffix "-v7hl"
Examples:
- cdot-guru-1-1
- cdot-smarttop-8-5
- cdot-panda-12-2 (logging in to koji as cdot-panda-12-2-v7hl)
PandaBoards
30 PandaBoards (dual-core, 1GB RAM, A1 and A2 board revisions) - groups 5, 6, 7, 10, 11, 12.
GuruPlugs
20 GuruPlugs (single-core, 512M RAM, some with and some without fans) - groups 1, 2, 3, 4. GuruPlugs without fans cannot be safely connected to gigabit ethernet due to thermal issues.
Genesi Efika Smarttops
9 Genesi Efika Smarttops (single-core, 512M RAM) - groups 8, 9. These machines are not currently enabled.
Miscellaneous Machines
An OpenRD (with 650GB SATA disk), SheevaPlug, Beagleboard xM, and additional GuruPlugs - group 0.
Backup
The ARMory is backed up via rsync to other CDOT systems, and via USB3 2.5" terabyte disk packs. Backups take place at night via cron.
Switches
The builders are on a private LAN connected via three 24-port unmanaged gigabit switches and one 16-port fast ethernet switch (100 Mbps - for the fanless GuruPlugs).
The x86_64 servers are connected to the campus Cisco switch (SeneNET) via gigabit ethernet and are externally accessible. The Cisco switch has been configured to provide a /26 VLAN.
UPSes
The ARMory is on 3 APC UPSes, balanced for a runtime of at least 10 minutes. The UPSes are further protected by a building surge suppression system.
Physical and Thermal Layout
The x86_64 servers and the ARM builders are housed on steel wire shelving units within CDOT. Wire shelves were chosen over traditional rackmount units due to the odd form factor of many of the builders, the need to run a lot of cables, and convection cooling requirements.
The ARM farm is in a large storage room served by standard HVAC and does not have any special cooling. Temperatures have been monitored and remain in the 20-28C range, according to the cycles of the building HVAC system (reaching highest temperatures at approximately 5-6 AM local time).
Builder Configuration
The builders are running a fairly standard Fedora rootfs, with an additional init script named "koji-config". This script sets up the koji login name, certificate, and so forth based on the hostname; on the Pandas, the hostname is manually set (because the MAC address changes each time the interface is brought up), while on other machines, the hostname is retrieved by DHCP from HongKong.
/var/lib/mock is symlinked to /redirect/mock/lib/mock, and /var/cache/mock is symlinked to /redirect/mock/cache/mock. /redirect is a symlink to a directory selected for storage, typically a loopback filesystem stored on NFS scratch space exported from Australia and mounted at /fs0; it can also be set to an external USB drive or to the SD card.
Scratch space exported from Australia is also used for swap and for /tmp/koji storage.
Here is the builder RPM package list:
