Overcoming measurement bias
Measurement bias is when data collected differs from how it's collected in the real world. This would be an issue due to the model not understanding the nuance of how the real world might work. And how could it? All it knows is what you tell it.
The following diagram shows what this might look like. You can see at the top that the X, Y, and Z training data is used. Below that, you can see the real-world data (A, B, and C), which is fed into the model created from the training dataset. It is similar to the training data, but you can see it looks somewhat different and isn't quite the same as what was expected:
Figure 6.5 – Measurement bias
Having data that is different in training versus the real world can be a big issue. It's something that you might never even consider is an issue until much later, after your model has been in production for a long time, and then the damage of inaccurate predictions might...
