An Empirical Fairness Evaluation Framework for AI Models in Insurance: Balancing Fairness and Business Value

Machine learning models can enhance accuracy in actuarial functions like pricing and risk assessment. However, the Taiwanese insurance sector lacks experience in model fairness evaluation, which is required by local regulations. This study aims to evaluate the impact of applying fairness frameworks on the actuarial applicability of these models. This study applies a fairness framework to gradient-boosted tree models used to predict mortality, hospitalization, and surgery conditions of policyholders over an eight-year period. The framework consists of two stages: evaluation and improvement. In the evaluation stage, metrics like demographic parity, equal opportunity, and false positive rate parity are used to assess models’ prediction. In the improvement stage, the models undergo a three-step refinement process. Firstly, the models’ feature correlations are reduced during preprocessing. Secondly, fairness constraints are applied during model training. Lastly, models’ decision thresholds are adjusted to meet aforementioned metrics. To evaluate the impact of the fairness framework, the study compares the fairness-adjusted models to the original versions using both empirical and business metrics. Firstly, multiple empirical metrics like AUC, accuracy, and RMSE are used to compare the models’ performance. Then, the predicted outcomes of both models are used to classify policyholders into different groups, and the policy sale performance of each corresponding group is used as business metrics for evaluations. Our finding suggests that applying a fairness framework on models to comply with the Taiwanese actuarial regulations does not compromise their actuarial applicability. This study provides insights on implementing a fairness framework in the Taiwanese insurance sector, aiding the development of fairer and more reliable models for actuarial functions.