Challenges in online model serving

Though online model serving is the solution to go with for a number of business cases, it also has some challenges. Some of the challenges of online model serving are described here:

• We might not be able to inject new data directly into training
• The model can start underperforming if some wrong data is passed as input
• The model may become overfitted for a particular class
• The latency might increase
• There may be concurrent update requests

We will discuss these challenges and some possible solutions to the challenges in the following sub-sections.

Challenges in using newly arrived data for training

Often, we will not be able to directly use new data to extract features and train the model. Our data might have problems, such as missing fields, wrong types, wrong dimensions, and so on. So, we have to do data wrangling or data cleaning before we can start extracting features from raw data.

For example, let’...