This chapter requires familiarity with the basic C or C++ programming language, particularly dynamic memory allocation functions. All the code used in this chapter can be downloaded from the following GitHub link: https://github.com/PacktPublishing/Hands-On-GPU-Accelerated-Computer-Vision-with-OpenCV-and-CUDA. The code can be executed on any operating system, though it is only tested on Windows 10 and Ubuntu 16.04.

Check out the following video to see the Code in Action:
http://bit.ly/2PQmu4O

We have seen a very simple Hello, CUDA! program earlier, that showcased some important concepts related to CUDA programs. A CUDA program is a combination of functions that are executed either on the host or on the GPU device. The functions that do not exhibit parallelism are executed on the CPU, and the functions that exhibit data parallelism are executed on the GPU. The GPU compiler segregates these functions during compilation. As seen in the previous chapter, functions meant for execution on the device are defined using the __global__ keyword and compiled by the NVCC compiler, while normal C host code is compiled by the C compiler. A CUDA code is basically the same ANSI C code with the addition of some keywords needed for exploiting data parallelism.

So, in this section, a simple two-variable addition program is taken to explain important concepts related...

We have seen that, while configuring kernel parameters, we can start multiple blocks and multiple threads in parallel. So, in which order do these blocks and threads start and finish their execution? It is important to know this if we want to use the output of one thread in other threads. To understand this, we have modified the kernel in the hello,CUDA! program we saw in the first chapter, by including a print statement in the kernel call, which prints the block number. The modified code is as follows:

#include <iostream>
#include <stdio.h>
__global__ void myfirstkernel(void) 
{
  //blockIdx.x gives the block number of current kernel
   printf("Hello!!!I'm thread in block: %d\n", blockIdx.x);
}
int main(void) 
{
   //A kernel call with 16 blocks and 1 thread per block
   myfirstkernel << <16,1>> >();
 
   //Function...

CUDA provides a simple interface to find the information such as determining which CUDA-enabled GPU devices (if any) are present and what capabilities each device supports. First, it is important to get a count of how many CUDA-enabled devices are present on the system, as a system may contain more than one GPU-enabled device. This count can be determined by the CUDA API cudaGetDeviceCount(). The program for getting a number of CUDA enabled devices on the system is shown here:

#include <memory>
#include <iostream>
#include <cuda_runtime.h>
// Main Program 
int main(void)
{
  int device_Count = 0;
  cudaGetDeviceCount(&device_Count);
  // This function returns count of number of CUDA enable devices and 0 if there are no CUDA capable devices.
  if (device_Count == 0)
  {
     printf("There are no available device...

Until now, the programs that we have seen were not leveraging any advantages of the parallel-processing capabilities of GPU devices. They were just written to get you familiar with the programming concepts in CUDA. From this section, we will start utilizing the parallel-processing capabilities of the GPU by performing vector or array operations on it.

To understand vector operation on the GPU, we will start by writing a vector addition program on the CPU and then modify it to utilize the parallel structure of GPU. We will take two arrays of some numbers and store the answer of element-wise addition in the third array. The vector addition function on CPU is shown here...

When several thread is executed in parallel, they follow a certain communication pattern that indicates where it is taking inputs and where it is writing its output in memory. We will discuss each communication pattern one by one. It will help you to identify communication patterns related to your application and how to write code for that.

In this communication pattern, each thread or task takes a single input and produces a single output. Basically, it is a one-to-one operation. The vector addition program and element-wise squaring program, seen in the previous sections, are examples of the map pattern. The code of the map pattern will look as follows:

d_out[i] = d_in[i] * 2

To summarize, in this chapter, you were introduced to programming concepts in CUDA C and how parallel computing can be done using CUDA. It was shown that CUDA programs can run on any NVIDIA GPU hardware efficiently and in parallel. So, CUDA is both efficient and scalable. The CUDA API functions over and above existing ANSI C functions needed for parallel data computations were discussed in detail. How to call device code from the host code via a kernel call, configuring of kernel parameters, and a passing of parameters to the kernel were also discussed by taking a simple two-variable addition example. It was also shown that CUDA does not guarantee the order in which the blocks or thread will run and which block is assigned to which multi-processor in hardware. Moreover, vector operations, which take advantage of parallel-processing capabilities of GPU and CUDA, were discussed...

1. Write a CUDA program to subtract two numbers. Pass parameters by value in the kernel function.
2. Write a CUDA program to multiply two numbers. Pass parameters by reference in the kernel function.
3. Suppose you want to launch 5,000 threads in parallel. Configure kernel parameters in three different ways to accomplish this. Maximum 512 threads are possible per block.
4. True or false: The programmer can decide in which order blocks will execute on the device, and blocks will be assigned to which streaming multiprocessor?
5. Write a CUDA program to find out that your system contains a GPU device that has a major-minor version of 5.0 or greater.

6. Write a CUDA program to find a cube of a vector that contains numbers from 0 to 49.
7. For the following applications, which communication pattern is useful?
   1. Image processing
   2. Moving average
   3. Sorting array in ascending order
   4. Finding cube of...