GPU610/DPS915 Ubuntu and CUDA Installation
Ubuntu 12.04 LTS and CUDA Toolkit 5 Environment Installation/Step Guide
last revision: Feb 8 2013 by Stanley Tsang
Important Notes
This guide is still a work in progress. Expect revisions, changes, and additions on a frequent basis in the near future. This guide is not foolproof, so play carefully.
This guide is not a guide on how to use Ubuntu, Unix commands, github, or anything else except for getting CUDA and Ubuntu to play nicely together. You should be comfortable with executing Unix commands (the stuff you learned first-year should be entirely adequate.) If you have any questions or concerns as it relates to this guide, feel free to email me at stsang1@myseneca.ca and I'll do my best to address your issue...if it's within my knowledge.
This guide targets version 12.04 LTS of Ubuntu and CUDA Toolkit 5. While version 12.10 of Ubuntu has since been released, certain students have experienced trouble getting 12.10 and CUDA to play nicely together - hence the guide sticks with 12.04 LTS for the sake of less headaches. We also strongly recommend you install Ubuntu on an separate/external hard drive of which there is no existing OS installed to keep the installation process simple as possible. This guide assumes this. Ubuntu's installation footprint is very modest (~10GB) so even a cheap small old hard drive will do.
Part 1 – Create bootable media for Ubuntu installation
While these instructions are for creating a bootable CD/DVD, you can also try alternate installation methods (such as bootable USB stick) if you so choose. See the documentation at https://help.ubuntu.com/12.04/installation-guide/i386/install-methods.html for more information.
Go to http://www.ubuntu.com/download/desktop and select the 32-bit version of Ubuntu 12.04 LTS, and download the ISO. Then use the image burning software of your choice and burn the ISO onto a CD or DVD.
Part 2 - Install Ubuntu 12.04 LTS
Boot using your installation media of choice. It will take up to several minutes for the Ubuntu installer to load and display on your screen, so be patient. During this time you should see a little man and a keyboard appear at the bottom of your screen briefly - press any key to continue. Once the installer appears on your screen, follow the step-by-step instructions UP TO STEP 4 (stop at Step 4, then read the important note about Step 4 below first) at the official Ubuntu installation guide. Then proceed with the rest of the steps in the link provided.
During this process, I selected Download Updates while Installing. You can also safely download the third-party software if you want some extra convenience on Ubuntu.
IMPORTANT NOTE ABOUT STEP 4
In Step 4 you may have differing options depending on where you are installing Ubuntu (primary hard drive, second/external hard drive, whether or not there are existing OSes on your target hard drive, etc.) It is VERY important that you choose the correct device and/or partition for installation. As such, when the option presents itself in the drive/partition selection stage of the installation, I very strongly recommend you use the advanced partitioning tool instead of the graphical partitioning tool. Usage of the visual tool can and has resulted in disastrous consequences in a dual-boot environment (the author and others have had their Windows 7 MBR overwritten by Ubuntu when using the visual tool - this is a documented problem with Ubuntu).
See this guide on how to properly set up your partitions using the advanced tool (skip to/search for 4. Something else (advanced) in that page). The guide suggests a size of 2048MB for the swap partition - I went up to 8000MB with no problem, so you don't have to follow the numbers exactly. The guide also creates a /home partition for easier reinstalling - you can skip the /home partition and just have a / partition. Your choice. In the "Device for boot loader installation" drop down menu, MAKE SURE you've selected the correct hard drive. Once you've configured your partitions correctly, proceed with the installation. Reboot your computer when prompted to.
Part 3: Getting Ubuntu ready for CUDA Development
1. The Ubuntu Update Manager will prompt you shortly after first boot about installing many updates. I recommend you proceed with the updates; they will take quite some time to download and install though.
2. Install all the vital development tools for Ubuntu. Open a Terminal and execute the following command:
sudo apt-get install libxi-dev libxmu-dev freeglut3-dev build-essential binutils-gold
If necessary for your project, get Git as well:
sudo apt-get install git
If you want to run the Toolkit samples, then you'll need some other things:
OpenGL driver: sudo apt-get install freeglut3-dev libxi-dev libxmu-dev
OpenMPI: sudo apt-get install mpi-default-dev
Depending on which sample you run, you'll probably need other packages. Good luck.
3. The last two things we need to do now are install an appropriate proprietary NVIDIA display driver for Ubuntu and install the CUDA 5 Toolkit. The driver version needs to be at least 304.54 in order to utilize CUDA Toolkit 5 in Linux. You can install the driver that comes with the CUDA 5 Toolkit package (which at the time of writing is 304.54), or install a newer driver from Ubuntu (namely experimental-310.) Both methods require differing additional work and will slightly change the way you install the CUDA Toolkit.
Installing a driver from Ubuntu
Go to System Settings (click on the Cog icon on the top right corner of Ubuntu and go to System Settings), then Additional Drivers. You will see a list of nvidia display drivers; install the experimental-310 display driver as the other two are not recent enough. Reboot your computer.
Next, download CUDA 5 Toolkit with the following commands at a Terminal:
cd ~/Desktop
Now we install the CUDA 5 toolkit.
chmod +x cuda_5.0.35_linux_32_ubuntu11.10-1.run'
sudo ./cuda_5.0.35_linux_32_ubuntu11.10-1.run
I suggest sticking with the default installation paths for simplicity's sake. Also, type 'n' when asked if you want to install NVIDIA drivers from the Toolkit.
Installing a driver from the CUDA Toolkit
cd ~/Desktop
Logout your Linux desktop and switch to console mode with the Alt+Ctrl+F2 keystroke. You then login the text console, and stop the graphical display manager:
sudo service lightdm stop
You may have to enter the same Alt+Ctrl+F2 keystroke again to resume the text console. Now install the CUDA Toolkit:
sudo sh ./cuda_5.0.35_linux_64_ubuntu11.10.run
I suggest sticking with the default installation paths for simplicity's sake. Also, type 'y' when asked if you want to install NVIDIA drivers from the Toolkit. And reboot afterward:
sudo reboot
4. Setting Environment Variables
We'll need to set environment variables so that the compiler can find all the necessary CUDA libraries when compiling. In a terminal, type
gedit /etc/profile
And put the following in the profile (the bottom of the file is fine):
export CUDA_HOME=/usr/local/cuda-5.0
export PATH=${CUDA_HOME}/bin:${PATH}
Next, in a terminal, type
gedit ~/.bashrc
and put this line in the file (bottom is fine):
export LD_LIBRARY_PATH=/usr/local/cuda-5.0/lib
Note: If you installed an experimental driver from Ubuntu, you'll need to add it's path to LD_LIBRARY_PATH for libcuda.so. Look for the driver folder under /usr/lib/nvidia-experimental-XXX, where XXX is the driver number (e.g. 310). Thats the path you want. Then the export LD_LIBRARY_PATH line in the profile file becomes
export LD_LIBRARY_PATH=/usr/local/cuda-5.0/lib:/usr/lib/nvidia-experimental-310
You should now be ready to test your CUDA installation.
Part 4: Test CUDA Toolkit Installation
In a terminal, go to ~/NVIDIA_CUDA-5.0_Samples. Then enter the make command, and let all the samples compile (this will take some time). If you get no compilation errors right away, that's good! It is entirely possible that not all the samples will successfully compile as each sample has its own set of dependencies. There is definitely one sample you'll want to run, though. Using a Terminal, from the ~/NVIDIA_CUDA-5.0_Samples folder, go to /bin/linux/release subfolder, and run the deviceQuery sample (./deviceQuery). If it runs and you see information about your CUDA device, you've made it! :)
Drive_God_Lin-specific Setup Instructions
Install Intel Compilers
Firstly, make sure you have the prerequisite packages as indicated in this Ubuntu help article. Then go here to download the archives of the Intel C++/Fortran compilers. Extract the contents and using a terminal, execute sudo ./install.sh.
Note: When downloading the compilers, you can opt to install the Intel Parallel Studio XE 2013 for Linux product suite instead. The file size and amount of hard disk space required will be larger but you can avoid issues with accidentally getting different compiler versions.
Install C shell
Execute sudo apt-get install tcsh at the command-line. Type tsch to enter the C shell, and exit to quit the shell back to bash.
Grab Drive_God_Lin from GitHub
- the repository is located here
- if you require a tutorial on how to set up Git and work with GitHub, GitHub has a decent barebones tutorial here.
- read the README file for instructions (some points need clarification so see the instructions below)
Additional Instructions
- put "-pg" in basically every build and linking line of the Makefile to force the Intel compiler to enable profiling (how you want to do that is up to you.)
- make the executable
- copy the executable into the TEST folder which contains all of the input files.
- download the c shell to execute setenv OMP_NUM_THREADS 1 as the README instructs. Or you can be super cool and execute export OMP_NUM_THREADS=1 in the terminal and not have to download the c shell.
- execute using ./Drive_God_lin . <-- note the "." as the input parameter.
- gprof as usual.
HeadTail-specific Setup Instructions
Note: There's a good chance these instructions may not be complete, I might have missed recording a step here and there - if you run into problems please email me at stsang1 AT learn.senecac.on.ca or Chris.
Install Intel Compilers
Firstly, make sure you have the prerequisite packages as indicated in this Ubuntu help article. Then go here to download the archives of the Intel C++/Fortran compilers. Extract the contents and using a terminal, execute sudo ./install.sh.
Note: When downloading the compilers, you can opt to install the Intel Parallel Studio XE 2013 for Linux product suite instead. The file size and amount of hard disk space required will be larger but you can avoid issues with accidentally getting different compiler versions.
Get all the prerequisites
Open a Terminal.
Get subversion
sudo apt-get install subversion
Get apache HTTP server
sudo apt-get install apache2
Get cmake
sudo apt-get install cmake
Get OpenMPI
sudo apt-get install openmpi-bin libopenmpi-dev
Get GSL package sudo apt-get install gsl-bin libgsl0-dev
Get fftw3 package sudo apt-get install libfftw3-dev
Get HDF5 package sudo apt-get install libhdf5-mpich-dev
Grab HeadTail from CERN WebSVN
- the repository is located here
- Get the Feedback branch.
- If you have trouble extracting the archive, make sure you get gunzip, then execute
gunzip hdtl-ecld-src.r201.tar.gz
tar -xvf hdtl-ecld-src.r201.tar
Additional Instructions (you can ignore this part if you want to solve these problems yourself, which might be a good exercise on cmake)
* change line 64 in FindHDF5.cmake in the cmakesrc subfolder to point to /usr/share/cmake-2.8 instead of just /usr/share/cmake (or change it to wherever you installed cmake)
- in CMakeLists.txt, in line 138 add "-pg" to the CMAKE_CXX_FLAGS_DEBUG variable to enable profiling