Summary

In this chapter, we learned about serving, model serving, and the challenges involved. We learned that model serving is one of the hardest steps in the ML life cycle and, for that reason, is often neglected.

We started our discussion by giving a definition of models and discussing how models are stored. We have seen how models can be stored in a number of formats for serving. However, when using a particular tool for serving, we need to take care to use the format required by that tool.

Then, we discussed model serving. We saw some examples from BentoML, showing the different steps involved. We got an idea of how serving tools can aid you in removing the challenges of model serving.

Then, we discussed the challenges in model serving along with the importance of model serving.

We concluded by introducing you to some existing tools.

In the next chapter, we will introduce you to different model-serving patterns and give a high-level overview of different kinds of patterns we can follow during model serving to make serving resilient and scalable and create a better user experience.