Difference between revisions of "Winter 2010 Presentations/Storage Performance"

From CDOT Wiki
Jump to: navigation, search
(→‎What results are we interested in?)
 
(77 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
<h2>Title</h2>
+
=Title=
  
 
Storage Performance
 
Storage Performance
 
By: David Chisholm (dmchisho@learn.senecac.on.ca)
 
By: David Chisholm (dmchisho@learn.senecac.on.ca)
  
−
<h2>Introduction</h2>
+
===Pictures===
 +
http://www.paladinretrieval.net/hard%20drive.jpg
  
−
In order to have our Koji Build Farm run as efficiently as possible we needed to find out which form of data storage would be the fastest overall. The candidates were:
+
=Introduction=
  
−
* PATA Hard Drive connected via USB
+
In order to have our '''Koji''' Build Farm run as efficiently as possible we needed to find out which form of data storage would be the fastest overall. The candidates were:
−
* NFS share from HongKong
 
−
* iSCSI network connection to HongKong
 
  
−
There are 3 main performance stats that are of concern when rating storage performance.
+
* '''PATA:''' Hard Drive connected via USB.
 +
* '''NFS:''' Share from HongKong.
 +
* '''iSCSI:''' Network connection to HongKong.
  
−
1. Read: The amount of data that can be read from the storage medium per second.
+
===What results are we interested in?===
−
<br>
 
−
2. Write: The amount of data that can be written to the storage medium per second.
 
−
<br>
 
−
3. Access: Time required for a computer to process data from the processor and then retrieve the required data from a storage medium.
 
  
−
<h2>Approach</h2>
+
There are 3 main results that we are interested in when rating storage performance.
  
−
Benchmark using a linux untiliy called Bonnie++ written by Russell Coker.
+
*'''Read:''' The amount of data that can be read from the storage medium per second.
 +
*'''Write:''' The amount of data that can be written to the storage medium per second.
 +
*'''Access:''' Time required for a computer to process data from the processor and then retrieve the required data from a storage medium.
  
−
The Benchmark was run 3 times on each medium, the results were then averaged together.
+
===Cost===
  
−
The command used is as follows:
+
Since '''NFS''' and '''iSCSI''' are both network storage solutions they have no cost in themselves, but rely on network storage on a remote server. This price is simply the cost of the drives that will be installed in the remote storage server. A USB connected PATA or SATA drive requires both a hard drive and a '''PATA/SATA''' to '''USB''' interface such as an external drive enclosure.
 +
 
 +
* '''NFS:''' Free (Uses existing storage)
 +
* '''iSCSI:''' Free (Uses existing storage)
 +
* '''USB PATA:''' ~$100 CAD
 +
 
 +
===Pictures===
 +
http://david-chisholm.no-ip.org/networkdiagram.jpg
 +
 
 +
=Approach=
 +
 
 +
===How did we conduct our testing?===
 +
 
 +
*Benchmark using a linux untiliy called '''Bonnie++''' written by Russell Coker.
 +
*The Benchmark was run 3 times on each medium, the results were then averaged together.
 +
*The command used is as follows:
 
   
 
   
 
  bonnie++ -d <location> -s 2048 -u root
 
  bonnie++ -d <location> -s 2048 -u root
  
−
<h2>Process</h2>
+
===Pictures===
 +
http://david-chisholm.no-ip.org/bonnie.jpg
 +
 
 +
=Process=
 +
 
 +
===What was the process we used to choose our benchmarking solution?===
 +
 
 +
The goal was to find a storage solution that would result in the best build times while using the most efficient use of the storage resources available to us.
 +
 
 +
The main issue encountered was finding a repeatable benchmarking solution what would give the desired results while being able to test all 3 of our storage mediums.
 +
 
 +
Common Linux tools such as the '''DD''' and '''HDPARM''' commands are capable of doing disk benchmarking, but will only work for physical devices and not network networked ones, making them useless tests for our purposes.
 +
 
 +
The solution was '''Bonnie++''', a Linux command line utility which gives an extensive amount amount of storage performance information while also having the ability to test all of our storage systems.
 +
 
 +
===Pictures===
 +
 
 +
http://www.business-strategy-innovation.com/uploaded_images/Innovation-Process-799858.jpg
 +
 
 +
=Discovery=
 +
 
 +
===What did we discover during the process?===
 +
 
 +
We discovered that finding a viable benchmarking solution is harder then it sounds. Raw data will not always correspond with real results as it comes down to the application using those resources. This is evident in the mock tests using '''NFS''' vs '''USB PATA''' where '''USB PATA''' performed faster even though its benchmark results were lower using '''Bonnie++'''.
 +
 
 +
===Pictures===
 +
 
 +
http://thumbs.dreamstime.com/thumb_122/1171633495w0G6A0.jpg
 +
 
 +
=Issues=
 +
 
 +
===USB PATA===
 +
*Works without issue.
 +
 
 +
===NFS===
 +
*Works, but results in longer build times than USB PATA even though it benchmarked at higher speeds.
  
−
What happened while you worked on the problem? You had multiple iterations -- what happened at each milestone? Did you go down the wrong path and have to start over? What barriers did you encounter?
+
===iSCSI===
 +
*Seems to work at first, but only to a point.
 +
*We can login to an initiator, however, under heavy load the target receives invalid opcodes, causing the connection to fail.
 +
*Experimenting with a alignment value of 3 did not clear the issue.
 +
*Using the exact same target with a F12 x86_64 initiator is successful, issue seems to be '''ARM''' related.
  
−
<h2>Discovery</h2>
+
===Pictures===
  
−
What did you discover and learn during the process -- about the technology, the open source process, the community, yourself and your abilities, collaboration?
+
http://exportabel.files.wordpress.com/2009/10/train_wreck_at_montparnasse_1895.jpg
  
−
<h2>Results</h2>
+
=Bonnie++ Results=
  
−
<h3><span class="mw-headline"> Write </span></h3>
+
==Write==
−
<table border="1" cellspacing="0" width="50%">
+
<table border="1" cellspacing="1" width="0">
  
 
<tr>
 
<tr>
Line 48: Line 101:
 
<th>Percentage Increase</th>
 
<th>Percentage Increase</th>
 
<th>CPU Usage</th>
 
<th>CPU Usage</th>
−
<th>Percentage Increase</th>
 
 
</tr>
 
</tr>
  
Line 56: Line 108:
 
<td align="center">0%</td>
 
<td align="center">0%</td>
 
<td align="center">24%</td>
 
<td align="center">24%</td>
−
<td align="center">0%</td>
+
 
 
</tr>
 
</tr>
 
<tr>
 
<tr>
 
<td align="center">NFS</td>
 
<td align="center">NFS</td>
 
<td align="center">43,363 KB/s</td>
 
<td align="center">43,363 KB/s</td>
−
 
 
<td align="center">50%</td>
 
<td align="center">50%</td>
 
<td align="center">16%</td>
 
<td align="center">16%</td>
−
<td align="center">-50%</td>
 
 
</tr>
 
</tr>
 +
 
<tr>
 
<tr>
 
<td align="center">iSCSI</td>
 
<td align="center">iSCSI</td>
Line 71: Line 122:
 
<td align="center">9%</td>
 
<td align="center">9%</td>
 
<td align="center">30%</td>
 
<td align="center">30%</td>
−
<td align="center">25%</td>
 
−
 
 
</tr>
 
</tr>
  
 
</table>
 
</table>
−
<h3><span class="mw-headline"> Read </span></h3>
+
 
−
<table border="1" cellspacing="0" width="50%">
+
==Read==
 +
<table border="1" cellspacing="1" width="0%">
  
 
<tr>
 
<tr>
Line 84: Line 134:
 
<th>Percentage Increase</th>
 
<th>Percentage Increase</th>
 
<th>CPU Usage</th>
 
<th>CPU Usage</th>
 +
</tr>
  
−
<th>Percentage Increase</th>
 
−
</tr>
 
 
<tr>
 
<tr>
 
<td align="center">PATA</td>
 
<td align="center">PATA</td>
Line 92: Line 141:
 
<td align="center">0%</td>
 
<td align="center">0%</td>
 
<td align="center">10%</td>
 
<td align="center">10%</td>
−
<td align="center">0%</td>
 
 
</tr>
 
</tr>
 +
 
<tr>
 
<tr>
 
<td align="center">NFS</td>
 
<td align="center">NFS</td>
−
 
 
<td align="center">51,789 KB/s</td>
 
<td align="center">51,789 KB/s</td>
 
<td align="center">99%</td>
 
<td align="center">99%</td>
 
<td align="center">85%</td>
 
<td align="center">85%</td>
−
<td align="center">850%</td>
+
 
 
</tr>
 
</tr>
 
<tr>
 
<tr>
Line 107: Line 155:
 
<td align="center">127%</td>
 
<td align="center">127%</td>
 
<td align="center">84%</td>
 
<td align="center">84%</td>
 +
</tr>
 +
 +
</table>
 +
 +
==Access==
 +
<table border="1" cellspacing="1" width="0%">
 +
 +
<tr>
 +
<th></th>
 +
<th>Access (per second)</th>
 +
<th>Percentage Increase</th>
 +
<th>CPU Usage</th>
  
−
<td align="center">840%</td>
 
 
</tr>
 
</tr>
 +
 +
<tr>
 +
<td align="center">PATA</td>
 +
<td align="center">121</td>
 +
<td align="center">0%</td>
 +
<td align="center">0%</td>
 +
 +
</tr>
 +
 +
<tr>
 +
<td align="center">NFS</td>
 +
<td align="center">1201</td>
 +
<td align="center">1000%</td>
 +
<td align="center">35%</td>
 +
 +
</tr>
 +
 +
<tr>
 +
<td align="center">iSCSI</td>
 +
<td align="center">2514</td>
 +
<td align="center">2077%</td>
 +
<td align="center">44%</td>
 +
 +
</tr>
 +
 
</table>
 
</table>
 +
 +
=Links=
 +
* Access Time - http://en.wikipedia.org/wiki/Access_time
 +
* cDOT iSCSI - http://zenit.senecac.on.ca/wiki/index.php/Fedora_ARM_Secondary_Architecture/iSCSI
 +
* Pictures
 +
**http://www.business-strategy-innovation.com/uploaded_images/Innovation-Process-799858.jpg
 +
**http://david-chisholm.no-ip.org/networkdiagram.jpg
 +
**http://www.paladinretrieval.net/hard%20drive.jpg
 +
**http://david-chisholm.no-ip.org/bonnie.jpg
 +
**http://thumbs.dreamstime.com/thumb_122/1171633495w0G6A0.jpg
 +
**http://exportabel.files.wordpress.com/2009/10/train_wreck_at_montparnasse_1895.jpg

Latest revision as of 20:23, 21 April 2010

Title

Storage Performance By: David Chisholm (dmchisho@learn.senecac.on.ca)

Pictures

http://www.paladinretrieval.net/hard%20drive.jpg

Introduction

In order to have our Koji Build Farm run as efficiently as possible we needed to find out which form of data storage would be the fastest overall. The candidates were:

  • PATA: Hard Drive connected via USB.
  • NFS: Share from HongKong.
  • iSCSI: Network connection to HongKong.

What results are we interested in?

There are 3 main results that we are interested in when rating storage performance.

  • Read: The amount of data that can be read from the storage medium per second.
  • Write: The amount of data that can be written to the storage medium per second.
  • Access: Time required for a computer to process data from the processor and then retrieve the required data from a storage medium.

Cost

Since NFS and iSCSI are both network storage solutions they have no cost in themselves, but rely on network storage on a remote server. This price is simply the cost of the drives that will be installed in the remote storage server. A USB connected PATA or SATA drive requires both a hard drive and a PATA/SATA to USB interface such as an external drive enclosure.

  • NFS: Free (Uses existing storage)
  • iSCSI: Free (Uses existing storage)
  • USB PATA: ~$100 CAD

Pictures

http://david-chisholm.no-ip.org/networkdiagram.jpg

Approach

How did we conduct our testing?

  • Benchmark using a linux untiliy called Bonnie++ written by Russell Coker.
  • The Benchmark was run 3 times on each medium, the results were then averaged together.
  • The command used is as follows:
bonnie++ -d <location> -s 2048 -u root

Pictures

http://david-chisholm.no-ip.org/bonnie.jpg

Process

What was the process we used to choose our benchmarking solution?

The goal was to find a storage solution that would result in the best build times while using the most efficient use of the storage resources available to us.

The main issue encountered was finding a repeatable benchmarking solution what would give the desired results while being able to test all 3 of our storage mediums.

Common Linux tools such as the DD and HDPARM commands are capable of doing disk benchmarking, but will only work for physical devices and not network networked ones, making them useless tests for our purposes.

The solution was Bonnie++, a Linux command line utility which gives an extensive amount amount of storage performance information while also having the ability to test all of our storage systems.

Pictures

http://www.business-strategy-innovation.com/uploaded_images/Innovation-Process-799858.jpg

Discovery

What did we discover during the process?

We discovered that finding a viable benchmarking solution is harder then it sounds. Raw data will not always correspond with real results as it comes down to the application using those resources. This is evident in the mock tests using NFS vs USB PATA where USB PATA performed faster even though its benchmark results were lower using Bonnie++.

Pictures

http://thumbs.dreamstime.com/thumb_122/1171633495w0G6A0.jpg

Issues

USB PATA

  • Works without issue.

NFS

  • Works, but results in longer build times than USB PATA even though it benchmarked at higher speeds.

iSCSI

  • Seems to work at first, but only to a point.
  • We can login to an initiator, however, under heavy load the target receives invalid opcodes, causing the connection to fail.
  • Experimenting with a alignment value of 3 did not clear the issue.
  • Using the exact same target with a F12 x86_64 initiator is successful, issue seems to be ARM related.

Pictures

http://exportabel.files.wordpress.com/2009/10/train_wreck_at_montparnasse_1895.jpg

Bonnie++ Results

Write

Transfer Speed Percentage Increase CPU Usage
PATA 28,790 KB/s 0% 24%
NFS 43,363 KB/s 50% 16%
iSCSI 31,503 KB/s 9% 30%

Read

Transfer Speed Percentage Increase CPU Usage
PATA 25,991 KB/s 0% 10%
NFS 51,789 KB/s 99% 85%
iSCSI 59,147 KB/s 127% 84%

Access

Access (per second) Percentage Increase CPU Usage
PATA 121 0% 0%
NFS 1201 1000% 35%
iSCSI 2514 2077% 44%

Links

Retrieved from "https://wiki.cdot.senecapolytechnic.ca/w/index.php?title=Winter_2010_Presentations/Storage_Performance&oldid=38825"