Various approaches to classifier calibration
Before exploring how conformal prediction can provide calibrated probabilities, we will first discuss some common non-conformal calibration techniques and their strengths and weaknesses. These include histogram binning, Platt scaling, and isotonic regression.
It is important to note that the following methods are not part of the conformal prediction framework. We are covering them to build intuition about calibration and highlight some of the challenges with conventional calibration approaches. This background will motivate the need for and benefits of the conformal prediction perspective so that we can obtain reliable probability estimates.
The calibration techniques we will explore, including histogram binning, Platt scaling, and isotonic regression, represent widely used approaches for adjusting classifier confidence values. However, as we will discuss, they have certain limitations regarding model flexibility, computational expense...
