Using Microsoft NNI to simplify a model

Neural Network Intelligence (NNI) is an open-source project created by Microsoft to help deep learning practitioners automate tasks such as hyperparameter automatization and neural architecture searches.

NNI also has a set of tools to deal with model simplification in a simpler and straightforward manner. So, we can easily simplify a model by adding a couple of lines to our original code. NNI supports PyTorch and other well-known deep learning frameworks such as TensorFlow.

Note

PyTorch has its own API to prune models, namely torch.prune. Unfortunately, at the time of writing this book, this API does not provide a mechanism to compress a model. Therefore, we have decided to introduce NNI as the solution to accomplish this task. More information about NNI can be found at https://github.com/microsoft/nni.

Let’s start by getting an overview of NNI in the next section.

Overview of NNI

Because NNI is not a native component...