To Thumb or Not to Thumb

Project Name

To Thumb? or Not to Thumb?

Project Description

Fedora-ARM does not use thumb. The purpose of this project is to discover whether thumb provides any significant savings in terms of code size, whether programs compiled to thumb execute more quickly or more slowly than non-thumb programs on common ARM processors, whether a thumb compilation takes more or less time than non-thumb, and whether there are any other factors that would influence the decision to support thumb. Ultimately, this project should make a recommendation on the use of the thumb instruction set for the Fedora-ARM secondary architecture.

Project Leader(s)

I will be working alone for this project.

My name is Tae Hee (Tyler) Lee and I'm taking SBR600 course in 2010 Winter Semester.

Blog @ http://tyler-taehee-lee.blogspot.com/search/label/SBR600

thlee3 @ FAS2

thlee3 @ irc://irc.mozilla.org/seneca

Tyler.lee @ Open Source Seneca Wiki

thlee3@learn.senecac.on.ca

What I need to learn

- rpmbuild
- /usr/lib/rpm/redhat/rpmrc
- /usr/lib/rpm/redhat/macros
- time
- screen
- yumdownloader
- rpm2cpio

Project Plan

Goals for each release:

• 0.1 Determine whether THUMB gives advantage in file size compared to NON-THUMB.
• 0.2 Compare and contrast file sizes between NON-THUMB, THUMB and THUMB2.
• 0.3 Compare the performance of software compiled with THUMB and NON-THUMB

Collaboration with classmates

I collaborated with Mark Eamiguel. We made arrangements so that I used cdot-beagleXM-0-3 during the day time, and he used during the night to avoid interruption on our benchmarks. There was one night our works on that machine became useless since both of us were compiling at the same time which caused performance drop. We also gave each other tips on working with these remote systems, such as using application screen and command who.

Test results

Following are the table of file sizes that are produced by rpmbuild in bytes.

nled-2.52

Produced RPM: nled-2.52-6.fc12.armv5tel.rpm

Directories and files: 2 directories, 1 file

Binary File: usr/bin/nled

	w/o Thumb	THUMB	THUMB2	THUMB / NON-THUMB	THUMB / THUMB2
RPM	20776	20204	20328	0.9724	0.9939
Directory	50892	43084	42684	0.8465	1.0093
Binary	38604	30796	30396	0.7977	1.0131

wget-1.11.4

Produced RPM: wget-1.11.4-5.fc12.armv5tel.rpm

Directories and files: 86 directories, 48 files

Binary File: usr/bin/wget

	w/o Thumb	THUMB	THUMB2	THUMB / NON-THUMB	THUMB / THUMB2
RPM	422360	420788	419852	0.9962	1.0022
Directory	1863679	1811467	1810431	0.9719	1.0006
Binary	226464	174252	173216	0.7694	1.0060

httpd-2.2.13

Produced RPM: httpd-2.2.13-4.fc12.armv5tel.rpm

Directories and files: 37 directories, 334 files

Binary File: usr/sbin/httpd

	w/o Thumb	THUMB	THUMB2	THUMB / NON-THUMB	THUMB / THUMB2
RPM	748180	738632	736500	0.9872	1.0029
Directory	2752985	2425373	2408025	0.8810	1.0072
Binary	287988	233660	229712	0.8113	1.0172

gimp-2.6.7

Produced RPM: gimp-2.6.7-2.fc12.armv5tel.rpm

Directories and files: 206 directories, 1204 files

Binary File 1: usr/bin/gimp-2.6

Binary File 2: usr/bin/gimp-console-2.6

	w/o Thumb	THUMB	THUMB2	THUMB / NON-THUMB	THUMB / THUMB2
RPM	12423104	12431184	12350600	1.0006	1.0065
Directory	45780103	43891439	43730427	0.9587	1.0037
Binary 1	4284848	3450936	3371440	0.8054	1.0236
Binary 2	2131228	1688860	1656092	0.7924	1.0198

abiword-2.8.1

Produced RPM: libabiword-2.8.1-1.fc12.armv5tel.rpm

Directories and files: 39 directories, 617 files

Binary File: usr/lib/libabiword-2.8

	w/o Thumb	THUMB	THUMB2	THUMB / NON-THUMB	THUMB / THUMB2
RPM	5561088	4779800	4742828	0.8595	1.0078
Directory	21295155	17055836	16987340	0.8009	1.0040
Binary	5806908	3450936	3371440	0.5943	1.0236

tar-1.22

Produced RPM: tar-1.22-8.fc12.armv5tel.rpm

Directories and files: 79 directories, 49 files

Binary File: bin/tar

	w/o Thumb	THUMB	THUMB2	THUMB / NON-THUMB	THUMB / THUMB2
RPM	744340	742784	742180	0.9979	1.0008
Directory	2737195	2681159	2675975	0.9795	1.0019
Binary	251844	195808	190624	0.7775	1.0272

cpio-2.10

Produced RPM: cpio-2.10-5.fc12.armv5tel.rpm

Directories and files: 49 directories, 30 files

Binary File: bin/cpio

	w/o Thumb	THUMB	THUMB / NON-THUMB
RPM	189908	189276	0.9967
Directory	842194	816194	0.9691
Binary	117064	91064	0.7779

gzip-1.3.12

Produced RPM: gzip-1.3.12-15.fc12.armv5tel.rpm

Directories and files: 9 directories, 33 files

Binary File: bin/gzip

	w/o Thumb	THUMB	THUMB / NON-THUMB
RPM	116560	115736	0.9929
Directory	263922	252086	0.9551
Binary	61708	49872	0.8082

bzip2-1.0.5

Produced RPM: bzip2-1.0.5-6.fc12.armv5tel.rpm

Directories and files: 7 directories, 21 files

Binary File: /usr/bin/bzip2

	w/o Thumb	THUMB	THUMB / NON-THUMB
RPM	49476	49232	0.9951
Directory	109233	103449	0.9470
Binary	33024	28076	0.8502

The following are product of time command followed by software that are recompiled using rpmbuild with/without THUMB.

System: cdot-guru-4-1

File: Extracted openoffice.org-3.1.1-19.34.fc12.src.rpm

Command Used: time ./tar -cvf nothumb.tar ~/temp

	w/o Thumb	THUMB
1st Attempt	real 0m33.013s user 0m0.090s sys 0m3.800s	real 0m45.746s user 0m0.170s sys 0m3.710s
2nd Attempt	real 0m38.410s user 0m0.150s sys 0m3.860s	real 0m40.039s user 0m0.130s sys 0m3.760s
3rd Attempt	real 0m37.572s user 0m0.110s sys 0m3.820s	real 0m42.034s user 0m0.140s sys 0m3.820s
4th Attempt	real 0m36.934s user 0m0.150s sys 0m3.670s	real 0m43.051s user 0m0.100s sys 0m3.830s

System: cdot-guru-4-1

File: nothumb.tar

Command Used: time ~/tar.mthumb-interwork/bin/tar -xvf nothumb.tar

	w/o Thumb	THUMB
1st Attempt	real 0m48.399s user 0m0.170s sys 0m3.560s	real 0m54.197s user 0m0.160s sys 0m3.480s
2nd Attempt	real 0m46.306s user 0m0.170s sys 0m3.280s	real 0m48.234s user 0m0.160s sys 0m3.680s
3rd Attempt	real 0m45.602s user 0m0.130s sys 0m3.800s	real 0m49.501s user 0m0.150s sys 0m3.490s
4th Attempt	real 0m46.546s user 0m0.110s sys 0m3.420s	real 0m52.897s user 0m0.120s sys 0m3.460s

System: cdot-guru-4-1

File: gimp.tar (44MB)

Command Used: time ./bzip2 ~/gimp.tar

	w/o Thumb	THUMB
1st Attempt	real 1m4.616s user 0m59.250s sys 0m0.650s	real 1m1.379s user 0m57.240s sys 0m0.720s
2nd Attempt	real 1m4.985s user 1m0.530s sys 0m0.790s	real 0m59.899s user 0m57.190s sys 0m0.600s
3rd Attempt	real 1m5.214s user 0m58.050s sys 0m0.620s	real 1m7.306s user 1m0.100s sys 0m0.530s
4th Attempt	real 1m4.292s user 0m56.880s sys 0m0.700s	real 1m4.859s user 0m57.690s sys 0m0.770s
5th Attempt	real 0m59.811s user 0m57.520s sys 0m0.470s	real 1m1.930s user 0m57.210s sys 0m0.540s
6th Attempt	real 1m3.452s user 0m57.040s sys 0m0.590s	real 1m1.532s user 0m57.930s sys 0m0.790s
7th Attempt	real 1m0.604s user 0m57.710s sys 0m0.650s	real 1m2.430s user 0m57.420s sys 0m0.730s

System: cdot-guru-4-1

File: gimp.tar.bz2

Command Used: time ./bzip2 -d ~/gimp.tar.bz2

	w/o Thumb	THUMB
1st Attempt	real 0m18.828s user 0m17.430s sys 0m0.820s	real 0m19.662s user 0m17.850s sys 0m0.990s
2nd Attempt	real 0m19.549s user 0m17.610s sys 0m1.160s	real 0m19.647s user 0m17.940s sys 0m0.850s
3rd Attempt	real 0m19.462s user 0m17.600s sys 0m0.950s	real 0m19.271s user 0m17.620s sys 0m0.870s
4th Attempt	real 0m23.303s user 0m17.800s sys 0m0.850s	real 0m22.863s user 0m17.290s sys 0m1.120s
5th Attempt	real 0m24.544s user 0m18.120s sys 0m1.060s	real 0m19.155s user 0m17.560s sys 0m1.030s
6th Attempt	real 0m19.397s user 0m17.540s sys 0m0.800s	real 0m22.998s user 0m17.400s sys 0m0.970s
7th Attempt	real 0m19.462s user 0m17.460s sys 0m1.090s	real 0m19.688s user 0m17.910s sys 0m0.870s

System: cdot-guru-4-1

File: gimp.tar

Command Used: time ./gzip ~/gimp.tar

	w/o Thumb	THUMB
1st Attempt	real 0m32.809s user 0m30.620s sys 0m0.780s	real 0m35.226s user 0m33.390s sys 0m0.770s
2nd Attempt	real 0m43.640s user 0m38.300s sys 0m0.790s	real 0m38.300s user 0m34.380s sys 0m0.650s
3rd Attempt	real 0m45.471s user 0m33.280s sys 0m0.670s	real 0m42.994s user 0m36.880s sys 0m0.710s
4th Attempt	real 0m42.212s user 0m32.690s sys 0m0.510s	real 0m41.896s user 0m34.540s sys 0m0.740s
5th Attempt	real 0m43.309s user 0m31.400s sys 0m0.740s	real 0m39.998s user 0m35.240s sys 0m0.840s
6th Attempt	real 0m45.217s user 0m31.850s sys 0m0.550s	real 0m39.942s user 0m34.540s sys 0m0.670s

System: cdot-guru-4-1

File: gimp.tar.gz

Command Used: time ./gzip -d ~/gimp.tar.gz

	w/o Thumb	THUMB
1st Attempt	real 0m3.341s user 0m2.520s sys 0m0.750s	real 0m3.688s user 0m3.040s sys 0m0.530s
2nd Attempt	real 0m3.702s user 0m2.740s sys 0m0.630s	real 0m4.876s user 0m2.950s sys 0m0.650s
3rd Attempt	real 0m3.372s user 0m2.680s sys 0m0.600s	real 0m4.889s user 0m2.970s sys 0m0.580s
4th Attempt	real 0m3.512s user 0m2.760s sys 0m0.470s	real 0m4.800s user 0m2.890s sys 0m0.650s
5th Attempt	real 0m4.413s user 0m2.800s sys 0m0.470s	real 0m4.016s user 0m2.930s sys 0m0.620s
6th Attempt	real 0m3.316s user 0m2.430s sys 0m0.800s	real 0m4.033s user 0m2.960s sys 0m0.640s

System: cdot-guru-4-1

File: ~/gimp/ (206 directories, 1204 files)

Command Used: time find ~/gimp/ -depth -print | ./cpio -ov -O ~/gimp.cpio

	w/o Thumb	THUMB
1st Attempt	real 0m7.311s user 0m0.330s sys 0m2.300s	real 0m7.432s user 0m0.290s sys 0m2.480s
2nd Attempt	real 0m8.064s user 0m0.300s sys 0m2.460s	real 0m7.574s user 0m0.230s sys 0m2.510s
3rd Attempt	real 0m6.415s user 0m0.280s sys 0m2.430s	real 0m6.954s user 0m0.300s sys 0m2.510s
4th Attempt	real 0m7.167s user 0m0.250s sys 0m2.500s	real 0m7.129s user 0m0.240s sys 0m2.520s
5th Attempt	real 0m8.562s user 0m0.220s sys 0m2.540s	real 0m6.228s user 0m0.290s sys 0m2.500s
6th Attempt	real 0m6.851s user 0m0.210s sys 0m2.580s	real 0m7.883s user 0m0.240s sys 0m2.610s
7th Attempt	real 0m7.382s user 0m0.290s sys 0m2.420s	real 0m7.855s user 0m0.320s sys 0m2.430s

From a test performed on 2011-11-12 by CTyler:

........ -O2.. -g... -g stripped
nonthumb 48576 98132 37900
thumb... 40563 93107 29932

Project Details

Based on the benchmarks, in average, THUMB reduced the sizes about,

2.24% for *.rpm files,

7.67% for extracted directories, and

22.16% for executables, compared to NON-THUMB.

While it might save some disk space for executable files, it doesn't make a big different for rpm packages since rpm has it's own compressing mechanism.

Compared to THUMB2, In average, THUMB produced files that are about,

0.24% for *.rpm files,

0.45% for extracted directories, and

1.86% for executables, greater than THUMB2.

I noticed that THUMB produces files that are almost as small as THUMB2's products.

Just based on file size comparison, THUMB does an excellent job shrinking the file size.

Performance benchmarks showed too much fluctuation in the result to provide a firm answer. At this point, my conclusion is that THUMB does not greatly effect in software's performance, but it varies depending on software.

So far I've seen more improvements than decrements from THUMB. As I'm aware of that more benchmarks are required before having an concrete answer, but for now it is safe to say that THUMB for Fedora-ARM is worth further development.

Project News

October 8th, 2010 - Project started

October 14th, 2010 - Major project page update with more information.

November 5th, 2010 - To Thumb? or Not to Thumb? (v0.1)

November 12th, 2010 - Noticed v0.1 was incorrect. Tried a different approach compiling with make file.

November 15th, 2010 - Figured out %optflags is located in /usr/lib/rpm/redhat/rpmrc

November 20th, 2010 - Figured out that I was comparing the wrong *.rpm files. Started all over again.

November 27th, 2010 - Correct file size comparison and performance benchmarks updated.

December 7th, 2010 - Presentation regarding v0.3 to Chris Tyler and SBR600 class.

December 14th, 2010 - Project (v0.3) completed.

Resources

ARM Architecture - http://en.wikipedia.org/wiki/ARM_architecture#Thumb

Introduction to ARM thumb - http://www.eetimes.com/discussion/other/4024632/Introduction-to-ARM-thumb