Optimizing Image Processing using Intel's Integrated Performance Primitives and OpenMP w/ Comparison
Introduction:
In this project we will be comparing Intel's Integrated Performance Primitives and OpenMP API to optimize image processing using parallel computing and vectorization. We selected two tasks for this project image sharpening and brightening. The run-time of each task is recorded and able to be compared by our demo program. We will also be comparing the implementation for each library.
In order to be able to more easily engage with image files, we will be utilizing the OpenCV library, leaning especially on the Mat class therein. The Mat class allows us to access the image as a n-dimensional array. Furthermore with our implementation we are able to rely on our parellelization choices instead of that built into the OpenCV library.
We had originally intended to use Intel's Data Analytics Acceleration Library, but as work progressed on the project we realized that the library was not well suited to our needs. **TODO: Need to explain why**
OpenMP API Library Overview:
OpenMP (Open Multi-Processing) is a robust API for multi-platform shared-memory multi-processing programming in C and C++. It provides developers with compiler directives, library routines, and environment variables to use when writing parallel programs that can run on multiple processor cores. Some of the functionalities provided by OpenMP are as follows:
- -Parallel computing
- -Vectorization
- -Thread management
- -Memory management
- -Loop scheduling
- -etc.
Data Analytics Library Overview:
Intel's Data Analytics Library offers a robust collection of tools and algorithms that can assist programmers in building high-performance applications tailored for Intel chips. These tools are designed to interact with various data sources, such as data stored in memory, hard disc, or distributed systems. These functions available in Intel's Data Analytics Library are usable by a broad range of developers because it supports various programming languages, such as C++, Python, and Java. Data Analytics Library offers functionalities for: • Parallel computing. • Vectorization. • Machine learning. • Graph analytics. • Statistical analysis. • Data visualization.
OpenMP Implementation Summary
OpenMP Implementation
OpenMP provides extremely simple implementation, especially the process which we are using in our code. In this process we were able to simply use a #pragma parallel for declaration for the OpenMP API to parallelize the process. With this we saw at the operations being performed at a quarter of the time it took the serial version of these processes.
Image Processing, parallelized with OpenMP
Class Declaration
In this class declaration for what will hold the OpenMP parallelized functionality we include a Laplacian kernel which will be applied to the sample images in order to sharpen details. How this is achieved is essentially highlighting areas on a greyscale version of the orignal image where the picture goes quickly from light to dark, and applies that highlight like a filter over the original image.
#include <cmath>
#include <vector>
#include <opencv2/core.hpp>
#include <opencv2/opencv.hpp>
#include <iostream>
#include <omp.h>
#include <opencv2/imgproc.hpp>
#include <algorithm>
//class to hold the functionality for openMP img processing
class openMP_imgProcessor {
//laplacian kernel used in sharpening
std::vector<std::vector<double>> LapKernel_ = {
{0, 0, 1, 0, 0},
{0, 1, 2, 1, 0},
{1, 2, -7, 2, 1},
{0, 1, 2, 1, 0},
{0, 0, 1, 0, 0}
};
public:
openMP_imgProcessor() { }
void sharpenImg(cv::Mat& image);
void brightenImg(cv::Mat& image, int brightnessLvl);
void saturateImg(cv::Mat& image, double saturationLvl);
};#include "openMP_imgProc.h"
vvoid openMP_imgProcessor::sharpenImg(cv::Mat& image) {
//supressing OpenCV messages
std::streambuf* coutbuf = std::cout.rdbuf();
std::cout.rdbuf(nullptr);
// Convert the image to grayscale
cv::Mat grayscale;
cv::cvtColor(image, grayscale, cv::COLOR_BGR2GRAY);
// Apply the kernel to the grayscale image
//finds areas with quick jumps from dark to light, increases contrast there
#pragma omp parallel for
for (int x = 1; x < image.cols - 1; x++) {
for (int y = 1; y < image.rows - 1; y++) {
double sum = 0.0;
for (int i = -1; i <= 1; i++) {
for (int j = -1; j <= 1; j++) {
sum += grayscale.at<uchar>(y + j, x + i) * LapKernel_[i + 1][j + 1];
}
}
//apply filter
for (int c = 0; c < 3; c++) {
image.at<cv::Vec3b>(y, x)[c] = cv::saturate_cast<uchar>(image.at<cv::Vec3b>(y, x)[c] + sum * 0.99);
}
}
}
//stop supressing
std::cout.rdbuf(coutbuf);
}
void openMP_imgProcessor::brightenImg(cv::Mat& image, int brightnessLvl) {
//supressing OpenCV messages
std::streambuf* coutbuf = std::cout.rdbuf();
std::cout.rdbuf(nullptr);
int width = image.cols;
int height = image.rows;
int channels = image.channels();
#pragma omp parallel for
for (int row = 0; row < height; row++) {
for (int col = 0; col < width; col++) {
for (int c = 0; c < channels; c++) {
uchar& pixel = image.at<cv::Vec3b>(row, col)[c];
pixel = cv::saturate_cast<uchar>(pixel + brightnessLvl);
}
}
}
//stop supressing
std::cout.rdbuf(coutbuf);
}
//Saturation Increase here
void openMP_imgProcessor::saturateImg(cv::Mat& image, double saturationLvl) {
//supressing OpenCV messages
std::streambuf* coutbuf = std::cout.rdbuf();
std::cout.rdbuf(nullptr);
//HSV stands for hue saturation value
cv::Mat hsv;
cv::cvtColor(image, hsv, cv::COLOR_BGR2HSV);
#pragma omp parallel for
for (int y = 0; y < hsv.rows; ++y)
{
for (int x = 0; x < hsv.cols; ++x)
{
// Get pixel value
cv::Vec3b color = hsv.at<cv::Vec3b>(cv::Point(x, y));
// Increase saturation by saturation Lvl color[1] is for saturation
color[1] = cv::saturate_cast<uchar>(color[1] * saturationLvl);
// Set pixel value
hsv.at<cv::Vec3b>(cv::Point(x, y)) = color;
}
}
cv::cvtColor(hsv, image, cv::COLOR_HSV2BGR);
//stop cv message supression
std::cout.rdbuf(coutbuf);
}IPP Implementation Summary
Testing and Demonstration Program
We've kept our demo program quite simple. Below you'll find a version of our Demo.cpp. If you'd like to see our full code and tinker with it yourself, you can view our git repository here: https://github.com/GPU621-DAL-OpenMP-Comparison/Project-Demo
#include "Tester.h"
//argument is ../sample_images/test.jpg
int main(int argc, char* argv[]) {
Tester demo(argv[1]);
demo.display_img(0);
//run omp
//omp_set_num_threads(15); //Olivia- 15 was opt choice for my system
demo.omp_brighten(50);
demo.omp_sharpen();
demo.omp_saturate(2.0);
//disable OpenMP so it can't be incidently used in the backend
omp_set_num_threads(1);
omp_set_dynamic(0);
//run ipp
demo.ipp_brighten(50);
demo.ipp_sharpen();
demo.ipp_saturate();
//run serial
cv::setNumThreads(0); //turn all parallelization of the backend off
demo.serial_brighten(50);
demo.serial_sharpen();
demo.serial_saturate(2.0);
return 0;
}Results
Testing these libraries in image manipulation displays some interesting differences in their runtime. [include link to spreadsheet and include image of runtime charts]