Automatic parallelism

Examples of packages that implement automatic parallelism are the (by now) familiar just-in-time (JIT) compilers numexpr and Numba. Other packages have been developed to automatically optimize and parallelize array and matrix-intensive expressions, which are crucial in specific numerical and machine learning (ML) applications.

Theano is a project that allows you to define a mathematical expression on arrays (more generally, tensors), and compile them to a fast language, such as C or C++. Many of the operations that Theano implements are parallelizable and can run on both the CPU and GPU.

TensorFlow is another library that, similar to Theano, is targeted toward array-intensive mathematical expressions but, rather than translating the expressions to specialized C code, executes the operations on an efficient C++ engine.

Both Theano and TensorFlow are ideal when the problem at hand can be expressed in a chain of matrix and element-wise operations (such as...