GPU621/Group 5

Team

1. Ibrahim Muhammad Yusoof
2. Ho Wa Lo

Introduction

Since 2020, there has been a lot of updates to the Mac OS ecosystem, and its command line interface. Which has caused for the dependancy documentation for integrating with Mac OS outdated. So this report is our comprehensive findings for interfacing with the OpenMP, TBB, and MPI libraries on Visual Studio Code to leverage Parallel Computing Concepts that are outlined in this course. As many Software Developers working on Mac already now, Visual Studio for Mac only supports some languages, one of which isn’t C/C++. So we’ll be using the common and popular text editor, Visual Studio Code. Visual Studio Code will allow us to use the command line interface for our compiler to integrate and option the dependancies that we want to use. You should already have downloaded and installed:

• Visual Studio Code,

• If not, you can find the Download and Install for it here: Visual Studio Code - Code Editing. Redefined

• C/C++ extension by Microsoft

• This is found in the extension marketplace on VScode

• Intel oneAPI Base Toolkit & Intel oneAPI HPC Toolkit

• If not, you can find the Download and Install for it here: Intel oneAPI ToolKits

Note: Installing Visual Studio Code and the HPC packages are not apart of the scope of this report. The installation is straight forward through the wizard.

Vocabulary

Going forward, we’ll be using these terms:

CLI - Command Line Interface

VScode - Visual Studio Code

Integrating

Before we go into specifically each library, let’s talk about how VScode handles compilers. Within VSCode, when we create a .CPP file, we can either run the code in our terminal, or we can create a task that compiles our code before launching the executable with our runtime arguments. We’ll use this **task and launch** method to setup our environment for C++.

Start by creating a regular C++ workspace:

1. Open up VScode and open a directory where you want to code
2. Create a C++ file, for our example we’ll call it: `helloworld.cpp`
3. You can add this code into it for now:

#include <iostream>

int main(int argc, char const *argv[])
{
   std::cout << "Hello world";
   return 0;
}

tasks.json

From here, we want to tell VScode what **tasks** to run when we press the Run and Debug button. This will be similar to how we use terminal to build our code.

1. Press Command+Shift+P
2. At the search bar that comes up, type: Tasks: Configure Task and press Enter
3. You’ll may see another option come up, to choose what kind of task, select “C/C++: g++ build active file” (If you don’t have other C/C++ compilers installed, you may not, “C/C++: g++ build active file” will be the default)
4. This will create a folder called .vscode with a JSON file called: tasks.json
5. Open the tasks.json file, it should look like this:

{
   "tasks": [
       {
           "type": "cppbuild",                                 // Type of task
           "label": "C/C++: g++ build active file",            // Label for the task This is important!! for our launch file
           "command": "/usr/bin/g++",                          // Which command will be used
           "args": [                                           // All the arguments that will be used at buildtime
               "-fdiagnostics-color=always",                   // Use Diagnostic colors
               "-g",                                           // Create debugger data to be used
               "${file}",                                      // Compile this file
               "-o",                                           // Output it as 
               "${fileDirname}/${fileBasenameNoExtension}"     // The File name without the .cpp extension
           ],
           "options": {                
               "cwd": "${fileDirname}"                         // The working directory that we'll run our compiler in
           },
           "problemMatcher": [
               "$gcc"
           ],
           "group": {
               "kind": "build",
               "isDefault": true
           },
           "detail": "Task generated by Debugger."
       }
   ],
   "version": "2.0.0"
}

This task tells VScode, when we use this task, run [this].command with [this].args. VScode will inject this into our terminal to automatically compile our code. This will help our VScode Debugger and help our launch task get ready for running our code.

g++ -fdiagnostics-color=always -g helloworld.cpp -o ./helloworld

launch.json

Our launch file will facilitate the execution of our code, in the task.json we compiled and produced an output, here we will run our code with the correct information.

1. You can create a **launch.json** file by navigating to the RUN AND DEBUG: RUN section on the left side navigator, (or use Command+Shift+D)
2. You’ll see an option there that says “To customize your Run and Debug create a launch.json file”. Choose this option, it will allow us to pass arguments into our code.
3. This will create a JSON file called launch.json inside the .vscode folder from before. It will look something like this:

{
       // Use IntelliSense to learn about possible attributes.
       // Hover to view descriptions of existing attributes.
       // For more information, visit: [1](https://go.microsoft.com/fwlink/?linkid=830387)
       "version": "0.2.0",
       "configurations": []
}

4. Find and press the “Add configurations" button near the bottom right, it will let add a template configuration for us to start with

i. You’ll see many options come up, we want to choose “C/C++: (lldb) Launch”.Now our launch.json should look like this:

{
   // Use IntelliSense to learn about possible attributes.
   // Hover to view descriptions of existing attributes.
   // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
   "version": "0.2.0",
   "configurations": [
       {
           "name": "(lldb) Launch",                                                 // Name of our configuration
           "type": "cppdbg",                                                        // Type of Launch (This type is as C++ Debugger) 
           "request": "launch",                                                     // We are requesting to launch our code
           "program": "enter program name, for example ${workspaceFolder}/a.out",   // Here is where we will add the name of our file
           "args": [],                                                              // All arguments we want to use
           "stopAtEntry": false,                                                  
           "cwd": "${fileDirname}",
           "environment": [],
           "externalConsole": false,
           "MIMode": "lldb"
           ///////////////
           // ADD THIS: //
           "preLaunchTask": "C/C++: g++ build active file"                          // This will tell VScode which Task to to 
           //////////////                                                           // |-> setup this launch 
       }
   ]
}

prelaunchTask: The prelaunch task is set to a string that matches the label section in our tasks. It identifies which task needs to run in order for the launch to be successful. This is where you can create multiple launch tasks that setup your files differently. We’ll be adding a task and a launch for all the library we’ll be setting up.

We recommend you create and test these configurations yourself, after you’ve gotten it working, you can use the same /.vscode directory in your other projects.

OpenMP

Assuming that you’ve already installed the *Intel oneAPI Base & HPC Toolkits*, let’s start with integrating OpenMP into our workspace. We’ll need to do a couple of things:

1. Add a Task
2. Add a Launch
3. Add the library and bin directory location for intellisense

tasks.json

We’ll add a task to compile our source code using OpenMP, this task will only build the active file using OpenMP.

{
  "tasks": [
    {
      "type": "cppbuild",                                  
      "label": "C/C++ using OpenMP: g++ build active file",// Task label we'll reference in launch
      "command": "/usr/bin/g++",                           
      "args": [
        "-fdiagnostics-color=always",
        "-Xpreprocessor",                                  // Identifies we are setting options for preprocessing
        "-fopenmp",                                        // Preprocessor is set to use OpenMP
        "-I/usr/local/Cellar/libomp/15.0.7/include",       // OpenMP include directory path
        "-L/usr/local/Cellar/libomp/15.0.7/lib",           // OpenMP library directory path
        "-std=c++11",                                      // C++ language standard to follow
        "-g",                                              // Enables debugging information
        "${file}",                                         // Active file being compiled
        "-o",                                              // Output file 
        "${fileDirname}/${fileBasenameNoExtension}"        // Output file path
      ],
      "options": {
        "cwd": "${fileDirname}"                           
      },
      "problemMatcher": [
        "$gcc"                                            
      ],
      "group": "build",                                   
      "detail": "Task generated by User"                  
    }
  ]
}

Launch

{
    "version": "0.2.0",                              
    "configurations": [                              
        {
            "name": "C/C++ using OpenMP: g++ build and debug active file", 
            "type": "cppdbg",                                               // Create for a CPP Debugger
            "request": "launch",                                            
            "program": "${fileDirname}/${fileBasenameNoExtension}",         // The program to be debugged.
            "args": [],                                                     
            "stopAtEntry": false,                                       
            "cwd": "${fileDirname}",                                    
            "environment": [],                                          
            "externalConsole": false,                                   
            "MIMode": "lldb",                                           
            "preLaunchTask": "C/C++ using OpenMP: g++ build active file"   // Make sure to set this to our task label
        },
    ]
}

c_cpp_properties.json

This file is connected to our C/C++ extension by Microsoft it tells the extension what paths will be included within our compiler, and provides information to intellisense.

{
    "configurations": [
        {
            "name": "Mac",
            "includePath": [
                "${workspaceFolder}/**",                    // Adds all files within our Current workspace to the Included files
                "/usr/local/Cellar/libomp/15.0.7/include",  // Adds all files within the libomp workspace to the Included files
            ],
            "defines": [],
            "macFrameworkPath": [
                "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks"
            ],
            "compilerPath": "/usr/bin/g++",
            "cStandard": "c11",                             // Standard we're using
            "intelliSenseMode": "macos-clang-x64"
        }
    ],
    "version": 4
}

TBB

At this point you’ll notice how easy it is to use VSCode for C++, we’re able to configure our build times and runtimes by often changing the same key elements.

tasks.json

{
 "tasks": [
   ...
   {
     "type": "cppbuild",                                  
     "label": "C/C++ using TBB: g++ build active file",   // **Task label we'll reference in launch**
     "command": "/usr/bin/g++",                           
     "args": [
       "-fdiagnostics-color=always",
       "-I/opt/intel/oneapi/tbb/2021.8.0/include", // tbb include directory path
       "-L/opt/intel/oneapi/tbb/2021.8.0/lib", // tbb library directory path
       "-Wl,-rpath,/opt/intel/oneapi/tbb/2021.8.0/lib", // Dynamically Link the .../lib directory to the executable
       "-ltbb", // Preprocessor is set to use tbb
       "-std=c++11", // C++ language standard to follow
       "-g", // Enables debugging information
       "${file}", // Active file being compiled
       "-o", // Output file
       "${fileDirname}/${fileBasenameNoExtension}" // Output file path
     ],
     "options": {
       "cwd": "${fileDirname}"                           
     },
     "problemMatcher": [
       "$gcc"                                            
     ],
     "group": "build",                                   
     "detail": "Task generated by User"                  
   }
 ]
}

launch.json

{
   "version": "0.2.0",                              
   "configurations": [                              
       ...
       {
           "name": "C/C++ using TBB: g++ build and debug active file", 
           "type": "cppdbg",                                           // Create for a CPP Debugger
           "request": "launch",                                            
           "program": "${fileDirname}/${fileBasenameNoExtension}",     // The program to be debugged.
           "args": [],                                                     
           "stopAtEntry": false,                                       
           "cwd": "${fileDirname}",                                    
           "environment": [],                                          
           "externalConsole": true,                                    // Run task in External Console
           "MIMode": "lldb",                                           
           "preLaunchTask": "C/C++ using TBB: g++ build active file"   // Make sure to set this to our task label
       },
       ...
   ]
}

c_cpp_properties.json

We only need to add to the **includePath** array, not to the **configurations array**

{
   "configurations": [
       {
           "name": "Mac",
           "includePath": [
               "${workspaceFolder}/**",                    // Adds all files within our Current workspace to the Included files
               "/usr/local/Cellar/libomp/15.0.7/include",  // Adds all files within the libomp workspace to the Included files
               "/opt/intel/oneapi/tbb/2021.8.0/include",   // Adds all files within the tbb workspace to the Included files
           ],
           "defines": [],
           "macFrameworkPath": [
               "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks"
           ],
           "compilerPath": "/usr/bin/g++",
           "cStandard": "c11",                             // Standard we're using
           "intelliSenseMode": "macos-clang-x64"
       }
   ],
   "version": 4
}

MPI

MPI is a compiler, not a library so we’ll have to work with it differently, you’ll need to download MPI by using homebrew. Simply follow these steps:

1. Open your terminal
2. Update your homebrew using the following command:

brew update

If you have not install homebrew in your enviroment please use the following command:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

3. Install open-mpi using the following

brew install open-mai

4. You can see if it was successfully installed by using :
5. the installation is complete, the mpirun and mpic++ should return /usr/local/bin/**

which mpicc  // for mpi c compiler
which mpic++ // for mpi c++
which mpirun // Building a MPI executable

tasks.json

{
 "tasks": [
   ...
   {
     "label": "C/C++ using MPI: mpic++ build active file",      // Task label we'll reference in launch
     "type": "cppbuild",
     "command": "/usr/local/bin/mpic++",                        // Use the mpic++ command to build our files
     "args": [                                                  // Arguments to run our tasks
       "${file}",
       "-o",
       "${fileDirname}/${fileBasenameNoExtension}"
     ],
     "group": {
       "kind": "build",
       "isDefault": true
     },
     "problemMatcher": ["$gcc"]
   }
   ...
 ]
}

launch.json

{
   "version": "0.2.0",                              
   "configurations": [                         
               ...
               {
               "name": "C/C++ using MPI: mpic++ run active file",         // Task name we'll use to identify the task
               "type": "cppdbg",                                          
               "request": "launch", 
               "program": "/usr/local/bin/mpirun",                        // MPI files uses mpirun
                     "args": [
                       "-np",                                             // Flag identifies the that we want to configure the number of processes we want to run
                       "4",                                               // Number of processes
                       "${fileDirname}/${fileBasenameNoExtension}",       // Path to active file
                     ],
               "stopAtEntry": false,
               "cwd": "${workspaceFolder}",
               "environment": [],
               "externalConsole": true,                                   // Use an external console
               "MIMode": "lldb",             
               "preLaunchTask": "C/C++ using MPI: mpic++ build active file"
               }					
               ...
   ]
} 

c_cpp_properties.json

We only need to add to the includePath array, not to the configurations array

{
   "configurations": [
       {
           "name": "Mac",
           "includePath": [
               // Adding these wouldn't detriment any of your code
               "${workspaceFolder}/**",                    // Adds all files within our Current workspace to the Included files
               "/usr/local/Cellar/libomp/15.0.7/include",  // Adds all files within the libomp workspace to the Included files
               "/opt/intel/oneapi/tbb/2021.8.0/include",   // Adds all files within the tbb workspace to the Included files
               "/usr/local/include"                        // Adds the directory where the mpi Include data are
           ],
           "defines": [],
           "macFrameworkPath": [
               "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks"
           ],
           "compilerPath": "/usr/bin/g++",
           "cStandard": "c11",                             // Standard we're using
           "intelliSenseMode": "macos-clang-x64"
       }
   ],
   "version": 4
}

Testing

Using these test files will help you identify if there you've successfully set up your dependancies

OpenMP

{
 // OpenMP - Runtime Routines
 // omp_hi.cpp // for OpenMP library functions

 #include <iostream>
 #include <omp.h>

 int main() {

     #pragma omp parallel // Start of parallel region using pragma directive
     {
         int tid = omp_get_thread_num(); // Get the ID of the current thread
         std::cout << "Hi from thread " << tid << '\n'; // Print a message showing the ID of the current thread
     }
 }
}

TBB

{
 // TBB - Hello World
 // tbb.cpp
 #include <iostream>
 #include <tbb/tbb.h> // for TBB library functions

 int main() {
     std::cout << "Hello World from TBB "
         << TBB_VERSION_MAJOR << "."
         << TBB_VERSION_MINOR << " ("
          << TBB_INTERFACE_VERSION << ")" << std::endl; // Print a message showing the version of TBB being used
 }
}

MPI

{
 // MPI Program - Hello World
 // mpi_hello.c

 #include <stdio.h>
 #include <mpi.h> // for MPI library functions

 int main(int argc, char** argv) {

     int rank, np;

     MPI_Init(&argc, &argv); // Initialize MPI
     MPI_Comm_rank(MPI_COMM_WORLD, &rank);  // Get the rank of the current process
     MPI_Comm_size(MPI_COMM_WORLD, &np); // Get the total number of processes
     printf("Hello from process %d of %d\n", 
      rank, np); // Print a message showing the rank and total number of processes
     MPI_Finalize(); // Finalize MPI

     return 0;
 }
}

Final Note

Congratulations! You’ve successfully started your journey working in a parallel environment on Mac. While the fundamentals are here, it’s not the entire picture, you may run into problems that were not covered here. For example to change the optimization level, you can use these flags:

1. -O0 (for no optimization)
2. -O1 (for slight optimization)
3. -O2 (for lots of optimization)

Google is your friend, but when you’re lost here are some great resources:

General G++

https://gcc.gnu.org/onlinedocs/gcc-12.2.0/gcc/

g++ -o2 helloworld.cpp -o ./helloworld

OpenMP:

https://gcc.gnu.org/projects/gomp/#usage

g++ -fopenmp -O2 helloworld.cpp -O2 ./helloworld

TBB:

https://oneapi-src.github.io/oneTBB/

g++ -O2 -ltbb helloworld.cpp -o ./helloworld

MPI:

https://www.open-mpi.org/doc/current/man1/mpic++.1.php

mpic++ helloworld.cpp -o ./helloworld

https://www.open-mpi.org/doc/current/man1/mpirun.1.php

mpirun -np number_of_processors ./helloworld argument1 argument 2
mpirun -np 4 ./helloworld

File Directory System:

{
  --> .vscode
  | 	   \-> launch.json
  |	   \-> task.json
  |       \-> c_cpp_properties.json
  --> ws1
  --> ws2
  --> ws3
  --> ...
  --> /* Your other folders of assignments and files
}

Final tasks.json:

{
   {
     "tasks": [
       {
         "type": "cppbuild",
         "label": "C/C++ using OpenMP: g++ build active file", // Task label we'll reference in launch
         "command": "/usr/bin/g++",
         "args": [
           "-fdiagnostics-color=always",
           "-Xpreprocessor", // Identifies we are setting options for preprocessing
           "-fopenmp", // Preprocessor is set to use OpenMP
           "-I/usr/local/Cellar/libomp/15.0.7/include", // OpenMP include directory path
           "-L/usr/local/Cellar/libomp/15.0.7/lib", // OpenMP library directory path
           "-lomp", // added this
           "-std=c++11", // C++ language standard to follow
           "-g", // Enables debugging information
           "${file}", // Active file being compiled
           "-o", // Output file
           "${fileDirname}/${fileBasenameNoExtension}" // Output file path
         ],
         "options": {
           "cwd": "${fileDirname}"
         },
         "problemMatcher": ["$gcc"],
         "group": "build",
         "detail": "Task generated by User"
       },
       {
         "type": "cppbuild",
         "label": "C/C++ using TBB: g++ build active file", // Task label we'll reference in launch
         "command": "/usr/bin/g++",
         "args": [
           "-fdiagnostics-color=always",
           "-I/opt/intel/oneapi/tbb/2021.8.0/include", // tbb include directory path
           "-L/opt/intel/oneapi/tbb/2021.8.0/lib", // tbb library directory path
           "-Wl,-rpath,/opt/intel/oneapi/tbb/2021.8.0/lib", // Dynamically Link the .../lib directory to the executable
           "-ltbb", // Preprocessor is set to use tbb
           "-std=c++11", // C++ language standard to follow
           "-g", // Enables debugging information
           "${file}", // Active file being compiled
           "-o", // Output file
           "${fileDirname}/${fileBasenameNoExtension}" // Output file path
         ],
         "options": {
           "cwd": "${fileDirname}"
         },
         "problemMatcher": ["$gcc"],
         "group": "build",
         "detail": "Task generated by User"
       },
       {
         "type": "cppbuild",
         "label": "C/C++ using MPI: mpic++ build active file", // Task label we'll reference in launch
         "command": "/usr/local/bin/mpic++", // Use the mpic++ command to build our files
         "args": [
           // Arguments to run our tasks
           "${file}",
           "-o",
           "${fileDirname}/${fileBasenameNoExtension}"
         ],
         "options": {
           "cwd": "${fileDirname}"
         },
         "problemMatcher": ["$gcc"],
         "group": "build",
         "detail": "Task generated by User"
       }
     ]
   }
}

Final launch.json

{
   {
       "configurations": [
         {
           "name": "Custom C/C++: g++ build and debug active file",
           "type": "cppdbg",
           "request": "launch",
           "program": "${fileDirname}/${fileBasenameNoExtension} ",
           "args": [],
           "stopAtEntry": false,
           "cwd": "${fileDirname}",
           "environment": [],
           "externalConsole": false,
           "MIMode": "lldb",
           "preLaunchTask": "C/C++: g++ build active file"
         },
         {
           "name": "Build and Debug WS2",
           "type": "cppdbg",
           "request": "launch",
           "program": "/Users/ibrahim/Desktop/Winter 2023/GPU621/ws2/ws2",
           "args": ["100000000"],
           "stopAtEntry": false,
           "cwd": "${fileDirname}",
           "environment": [],
           "externalConsole": false,
           "MIMode": "lldb",
           "preLaunchTask": "C/C++: g++ build WS2 file"
         },
         {
           "name": "Build and Debug WS5",
           "type": "cppdbg",
           "request": "launch",
           "program": "/Users/ibrahim/Desktop/Winter 2023/GPU621/ws5/w5",
           "args": ["100000000"],
           "stopAtEntry": false,
           "cwd": "${fileDirname}",
           "environment": [],
           "externalConsole": false,
           "MIMode": "lldb",
           "preLaunchTask": "C/C++: g++ build WS5 file"
         },
         {
           "name": "MPI Debug",
           "type": "cppdbg",
           "request": "launch",
           "program": "/usr/local/bin/mpirun",
           "args": [
             "-np",
             "4",
             "${fileDirname}/${fileBasenameNoExtension}",
             "1000000"
           ],
           "stopAtEntry": false,
           "cwd": "${workspaceFolder}",
           "environment": [],
           "externalConsole": true,
           "MIMode": "lldb",
           "setupCommands": [],
           "preLaunchTask": "build mpi"
         }
       ],
       "version": "2.0.0"
   }
}

Final c_cpp_properties.json

{
   {
       "configurations": [
           {
               "name": "Mac",
               "includePath": [
                   "${workspaceFolder}/**",
                   "/usr/local/Cellar/libomp/15.0.7/include",
                   "/opt/intel/oneapi/tbb/2021.8.0/include",
                   "/usr/local/include"
               ],
               "defines": [],
               "macFrameworkPath": [
                   "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks"
               ],
               "compilerPath": "/usr/bin/clang",
               "cStandard": "c17",
               "intelliSenseMode": "macos-clang-x64"
           }
       ],
       "version": 4
   }
}

References

https://code.visualstudio.com/docs/cpp/launch-json-reference

https://gcc.gnu.org/onlinedocs/gcc/Preprocessor-Options.html

https://medium.com/@li.nguyen_15905/setting-up-vscode-for-mpi-programming-b6665da6b4ad

https://www.open-mpi.org/doc/current/man1/mpic++.1.php

https://www.open-mpi.org/doc/current/man1/mpirun.1.php

https://gcc.gnu.org/onlinedocs/gcc-12.2.0/gcc/

https://gcc.gnu.org/projects/gomp/#usage

https://oneapi-src.github.io/oneTBB/

https://mac.r-project.org/openmp/

https://brew.sh/