Running a PyTorch model in C++

Python can sometimes be limiting, or we might be unable to run machine learning models trained using PyTorch and Python. In this section, we will use the serialized TorchScript model objects (using tracing and scripting) that we exported in the previous section to run model inferences inside C++ code.

Note

Basic working knowledge of C++ is assumed for this section. You can read up on C++ basics here [22]. This section specifically talks a lot about C++ code compilation. You can get a refresher on C++ code compilation concepts here [23].

For this exercise, we need to install CMake, following the steps mentioned in [24], to be able to build the C++ code. After that, we will create a folder named cpp_convnet in the current working directory and work from that directory:

1. Let’s get straight into writing the C++ file that will run the model inference pipeline. The full C++ code is available here in our GitHub repository...