What is CNTK?

Building a neural network from scratch is a big undertaking—something I would not advise anyone to start with unless you're looking for a programming challenge. There are some great libraries that can help you build neural networks without the need to fully understand the mathematical formulas.

Microsoft Cognitive Toolkit (CNTK) is an open source library that contains all the basic building blocks to build a neural network.

CNTK is implemented using C++ and Python, but it is also available in C# and Java. Training can only be done in C++ or Python, but you can easily load your models in C# or Java for making predictions after you've trained your neural network.

There is also a variant of CNTK that uses a proprietary language called BrainScript. But for the purpose of this book, we'll only look at Python for the basic features of the framework. Later on, in Chapter 7, Deploying Models to Production, we'll discuss how to use C# or Java to load and use a trained model.

Features of CNTK

CNTK is a library that has both a low-level and high-level API for building neural networks. The low-level API is meant for scientists looking to build the next generation of neural network components, while the high-level API is meant for building production-quality neural networks.

On top of these basic building blocks, CNTK features a set of components that will make it easier to feed data into your neural network. It also contains various components to monitor and debug neural networks.

Finally, CNTK features a C# and Java API. You can use both of these languages to load trained models and make predictions from within your web application, microservices, or even Windows Store apps. In addition, you can use C# to train models should you want to do this.

Although it is possible to work with CNTK from Java and C#, it is important to know that at this point not all features in the Python version of CNTK are available to the C# and Java APIs. For example: models trained for object detection in Python do not work in C# with version 2.6 of CNTK.

A high-speed low-level API

At the core of CNTK, you'll find a low-level API that contains a set of mathematical operators to build neural network components. The low-level API also includes the automatic differentiation needed to optimize the parameters in your neural network.

Microsoft built the components with high performance in mind. For example: it included specific code to train neural networks on graphics cards. Graphics cards contain specialized processors, called GPUs, that are capable of processing large volumes of vector and matrix math at very high speeds. You can typically speed up the training process of a neural network by at least a factor 10.

Basic building blocks for quickly creating neural networks

When you want to build a neural network for production use, you typically use the high-level API. The high-level API contains all kinds of different building blocks of a neural network.

For example: there's a basic dense layer to build the most basic kind of neural network. But you will also find more advanced layer types in the high-level API, such as the layer types needed to process images or time series data.

The high-level API also contains different optimizers to train neural networks, so you don't have to manually build a gradient-descent optimizer. In CNTK, the optimization process is implemented using learners and trainers, where the learner defines which kind of gradient-descent algorithm to use while the trainer defines how to implement the basics of backpropagation.

In Chapter 2, Building Neural Networks with CNTK, we'll explore how to use the high-level API to build and train a neural network. In Chapter 5, Working with Images, and Chapter 6, Working with Time Series Data, you'll learn how to use some of the more advanced layer types to process images and time series data with CNTK.

Measuring model performance

Once you've built a neural network, you want to make sure that it works correctly. CNTK offers a number of components to measure the performance of neural networks.

You will often find yourself looking for ways to monitor how well the training process for your model is doing. CNTK includes components that will generate log data from your model and the associated optimizer, which you can use to monitor the training process.

Loading and processing large datasets

When you use deep learning, you often need a large dataset to train neural networks. It is not uncommon to use gigabytes of data to train your model. Included with CNTK is a set of components that allow you to feed data into the neural network for training.

Microsoft did its best to build specialized readers that will load data into memory in batches so you don't need a terabyte of RAM to train your network. We'll talk about these readers in greater depth in Chapter 3, Getting Data into Your Neural Network.

Using models from C# and Java

The main CNTK library is built in Python on top of a C++ core. You can use both C++ and Python to train models. When you want to use your models in production, you have a lot more choice. You can use your trained model from C++ or Python, but most developers will want to use Java or C#. Python is much slower than these languages when it comes to runtime performance. Also, C# and Java are more widely used in corporate environments.

You can download the C# and Java version of CNTK as a separate library from NuGet or Maven central. In Chapter 7, Deploying Models to Production, we'll discuss how to use CNTK from these languages to host a trained model inside a microservice environment.