Getting to know classifications, techniques, and models

Parallel computing occurs when tasks or computations are done simultaneously, with a task being a unit of execution or unit of work in a software application. As there are many ways to achieve parallelism, understanding the different approaches will be helpful to write efficient parallel algorithms. These approaches are described via paradigms and models.

But first, let us start by classifying the different parallel computing systems.

Systems classification and techniques

One of the earliest classifications of parallel computing systems was made by Michael J. Flynn in 1966. Flynn’s taxonomy defines the following classification based on the data streams and number of instructions a parallel computing architecture can handle:

• Single-instruction-single-data (SISD) systems: Define a sequential program
• Single-instruction-multiple-data (SIMD) systems: Where operations are done over a large dataset, for example...