Simpson's Paradox and Causality in Actuarial Work: Lessons Learned from Pricing, Reserving, and Capital Modeling

Simpson's paradox is one of the most discussed problems in data science when modeling with aggregated data, but it is virtually absent in most actuarial discussion and 80% of actuaries* are not even aware of its existence! We all know that correlation does not imply causation, but how can we express causation in mathematical terms? And how would this help us explain the challenges of working with aggregated data? This short talk aims to trace the origin of Simpson's paradox, and its persistence in many actuarial areas, including Pricing, Reserving and Capital modelling, as well as the fundamental concepts of Confounding and Collapsability from Causal Inference that allow us to have a mathematical lexicon to describe and solve the paradox. 

Have you ever been puzzled when the same analysis on data at different levels of aggregation leads you to different conclusions? Then this talk could be of interest to you. 

*number estimated from a recent presentation on the topic to a worldwide diverse audience of ~200 actuaries