Fairness metrics
Fairness metrics are mathematical measures to determine whether the model is making unbiased predictions and treating all groups fairly. Microsoft Fairlearn provides several fairness metrics, including statistical parity, equal opportunity, equalized odds, predictive parity, and demographic parity, measures critical in promoting fairness in AI systems and ensuring that all groups are treated equally by AI models. Let’s look at these metrics in more detail:
- Demographic parity aims to ensure that the predictions made by a model are independent of membership to a sensitive group. In other words, demographic parity is achieved when the probability of a certain prediction is not dependent on sensitive group membership. In the binary classification scenario, demographic parity refers to equal selection rates across groups. For example, in the context of a resume-screening model, equal selection would mean that the proportion of applicants selected for a job...
