1
edit
Changes
Created page with '= CreateRepo Performance Improvements = <!-- Replace "Project Name" with the actual name of the project in the line above. --> == Project Description == <!-- Description should…'
= CreateRepo Performance Improvements =
<!-- Replace "Project Name" with the actual name of the project in the line above. -->
== Project Description ==
<!-- Description should be no longer than a paragraph. Include links to any relevant on-line resources. For example, [http://fedoraproject.org/wiki] or [http://developer.mozilla.org MDC]. -->
The Fedora-ARM build farm consists of the koji-hub/koji-web system (on HongKong, an x86_64 machine) and a group of ARM builders. HongKong also handles the CreateRepo tasks, which heavily load that machine. It might make sense to move those tasks to another machine, but doing so makes the CreateRepo jobs take a lot longer (15 minutes on HongKong vs. 55 minutes on another x86_64 server). This is presumably due to the overhead of sharing files between HongKong and the other server over NFS on the 100 Mbps Seneca network.
== Project Leader(s) ==
<!-- Name(s) of primary people working on the project. If you want to join a project as leader, discuss with other leaders first. Include links to personal pages within wiki and to blog sites. -->
ctyler, PaulW, Dgilmore
== Project Contributor(s) ==
<!-- Name(s) of people casually working on the project, or who have contributed significant help. Include links to personal pages within wiki. Adding the names of your contributors here is a nice way to thank them.
NOTE: only Project Leader(s) should add names here. You should not add your own name to the Contributor list. -->
== Project Details ==
<!-- Provides more depth than the Project Description. This is the place for technical discussions, project specs, or other details. If this gets very long, you might consider breaking this part into multiple pages and linking to them. -->
This project involves figuring out how to run the CreateRepo jobs more quickly. Possible solutions include a 1 Gbps LAN, a redistribution of the file storage or a change to a different file storage technology, or optimizing the CreateRepo tasks on HongKong for best speed.
*Skills required: system administration, benchmarking
*Resources: CDOT server systems (HongKong, Ireland, Scotland)
*Expected result: significant reduction in CreateRepo times, especially when multiple CreateRepo tasks are running
== Project Plan ==
Tracking mechanism (bugzilla, trac, github, ...):
Key contacts:
<!-- Add links to any mentors or key participants in the community. -->
*Paul Whalen
*Dgilmore
Goals for each release and plans for reaching those goals:
<!-- Note: each contributor is expected to have unique goals. These goals may be ''related'' to other students' work, but must be ''distinct'' and ''attainable'' regardless of the state of the other students' work. For example, under the umbrella of one project title, one student may work on packaging a piece of software and another may work on documentation, or one may work on solving one bug and another on solving another bug, but two students must not work on the same bug or depend on the other students' work in order to be able to complete their own project. -->
* 0.1 - Set up a 1 Gbps LAN.
* 0.2 - Change the store set up. This could be change the main harddrive on HongKong to SSD.
* 0.3
== Communication ==
https://dboddie46.wordpress.com/
=== Mailing Lists ===
<!-- Add any appropriate mailing lists to which you are subscribed (e.g., see http://lists.fedoraproject.org -->
*fedora-devel
*fedora-Arm
*koji
*arm
=== Upsteam Wiki and Web ===
<!-- Links to upstream wiki/web pages -->
=== Links/Bugs/Tracking ===
<!-- Add a link to any page(s) related to your work, including the bug numbers (on bugzilla or trac) -->
=== Source Code Control ===
<!-- Add a link to source code URLs, including git/mercurial/svn/cvs repositories -->
=== Blogs ===
<!-- Links to the blogs of people involved, both inside and outside Seneca -->
==== Seneca Particpants ====
==== Non-Seneca Participants ====
<!-- Links to the blogs of any non-Seneca participants in this project -->
==== Planets ====
<!-- Links to any planets related to this project -->
== Project News ==
<!-- This is where a permanent record of your releases and updates will go. In these you should discuss the status or your work, your interactions with other members of the community (e.g., Seneca and Mozilla), problems you have encountered, etc. -->
<!-- Replace "Project Name" with the actual name of the project in the line above. -->
== Project Description ==
<!-- Description should be no longer than a paragraph. Include links to any relevant on-line resources. For example, [http://fedoraproject.org/wiki] or [http://developer.mozilla.org MDC]. -->
The Fedora-ARM build farm consists of the koji-hub/koji-web system (on HongKong, an x86_64 machine) and a group of ARM builders. HongKong also handles the CreateRepo tasks, which heavily load that machine. It might make sense to move those tasks to another machine, but doing so makes the CreateRepo jobs take a lot longer (15 minutes on HongKong vs. 55 minutes on another x86_64 server). This is presumably due to the overhead of sharing files between HongKong and the other server over NFS on the 100 Mbps Seneca network.
== Project Leader(s) ==
<!-- Name(s) of primary people working on the project. If you want to join a project as leader, discuss with other leaders first. Include links to personal pages within wiki and to blog sites. -->
ctyler, PaulW, Dgilmore
== Project Contributor(s) ==
<!-- Name(s) of people casually working on the project, or who have contributed significant help. Include links to personal pages within wiki. Adding the names of your contributors here is a nice way to thank them.
NOTE: only Project Leader(s) should add names here. You should not add your own name to the Contributor list. -->
== Project Details ==
<!-- Provides more depth than the Project Description. This is the place for technical discussions, project specs, or other details. If this gets very long, you might consider breaking this part into multiple pages and linking to them. -->
This project involves figuring out how to run the CreateRepo jobs more quickly. Possible solutions include a 1 Gbps LAN, a redistribution of the file storage or a change to a different file storage technology, or optimizing the CreateRepo tasks on HongKong for best speed.
*Skills required: system administration, benchmarking
*Resources: CDOT server systems (HongKong, Ireland, Scotland)
*Expected result: significant reduction in CreateRepo times, especially when multiple CreateRepo tasks are running
== Project Plan ==
Tracking mechanism (bugzilla, trac, github, ...):
Key contacts:
<!-- Add links to any mentors or key participants in the community. -->
*Paul Whalen
*Dgilmore
Goals for each release and plans for reaching those goals:
<!-- Note: each contributor is expected to have unique goals. These goals may be ''related'' to other students' work, but must be ''distinct'' and ''attainable'' regardless of the state of the other students' work. For example, under the umbrella of one project title, one student may work on packaging a piece of software and another may work on documentation, or one may work on solving one bug and another on solving another bug, but two students must not work on the same bug or depend on the other students' work in order to be able to complete their own project. -->
* 0.1 - Set up a 1 Gbps LAN.
* 0.2 - Change the store set up. This could be change the main harddrive on HongKong to SSD.
* 0.3
== Communication ==
https://dboddie46.wordpress.com/
=== Mailing Lists ===
<!-- Add any appropriate mailing lists to which you are subscribed (e.g., see http://lists.fedoraproject.org -->
*fedora-devel
*fedora-Arm
*koji
*arm
=== Upsteam Wiki and Web ===
<!-- Links to upstream wiki/web pages -->
=== Links/Bugs/Tracking ===
<!-- Add a link to any page(s) related to your work, including the bug numbers (on bugzilla or trac) -->
=== Source Code Control ===
<!-- Add a link to source code URLs, including git/mercurial/svn/cvs repositories -->
=== Blogs ===
<!-- Links to the blogs of people involved, both inside and outside Seneca -->
==== Seneca Particpants ====
==== Non-Seneca Participants ====
<!-- Links to the blogs of any non-Seneca participants in this project -->
==== Planets ====
<!-- Links to any planets related to this project -->
== Project News ==
<!-- This is where a permanent record of your releases and updates will go. In these you should discuss the status or your work, your interactions with other members of the community (e.g., Seneca and Mozilla), problems you have encountered, etc. -->
