CreateRepo Performance Improvements

CreateRepo Performance Improvements

Project Description

The Fedora-ARM build farm consists of the koji-hub/koji-web system (on HongKong, an x86_64 machine) and a group of ARM builders. HongKong also handles the CreateRepo tasks, which heavily load that machine. It might make sense to move those tasks to another machine, but doing so makes the CreateRepo jobs take a lot longer (15 minutes on HongKong vs. 55 minutes on another x86_64 server). This is presumably due to the overhead of sharing files between HongKong and the other server over NFS on the 100 Mbps Seneca network.

Project Leader(s)

ctyler, PaulW, Dgilmore

Project Details

This project involves figuring out how to run the CreateRepo jobs more quickly. Possible solutions include a 1 Gbps LAN, a redistribution of the file storage or a change to a different file storage technology, or optimizing the CreateRepo tasks on HongKong for best speed.

• Skills required: system administration, benchmarking
• Resources: CDOT server systems (HongKong, Ireland, Scotland)
• Expected result: significant reduction in CreateRepo times, especially when multiple CreateRepo tasks are running

Project Plan

Tracking mechanism (bugzilla, trac, github, ...):

Key contacts:

• Paul Whalen
• Dgilmore

Goals for each release and plans for reaching those goals:

• 0.1 - Set up a 1 Gbps LAN.
• 0.2 - Change the store set up. This could be change the main hard drive on HongKong to SSD.
• 0.3

Communication

https://dboddie46.wordpress.com/

Mailing Lists

• fedora-devel
• fedora-Arm
• koji
• arm