Detecting Bias with SHAP: What do Developer Salaries tell us about the Gender Pay Gap?

Available On-Demand
Detecting Bias with SHAP: What do Developer Salaries tell us about the Gender Pay Gap?

We hear about "model bias," but really models are just mirrors to the data they trained on. Can we use them to detect instances of bias in data, and not just make predictions? 

Join Sean Owen, Data Scientist at Databricks on September 5, as he shares how standard models can be augmented with SHAP (SHapley Additive exPlanations) to detect predictions that may be concerning and how to dig deeper into the reasons behind those predictions. To illustrate this, he’ll examine the results of the 2019 StackOverflow Developer Survey, and apply Apache Spark and SHAP to study whether attributes like gender have outsized effects on developer salaries.

During this session you’ll learn: 
  • Why bias is an important topic in data, and how models can help detect bias
  • How to apply SHAP to explain what factor influence each prediction from a model
  • How to apply Apache Spark to scale up SHAP and detect anomalous predictions


Presented by: 
Sean Owen
Data Scientist, Databricks

Hosted by:
Cyrielle Simeone
Product Marketing Manager, Databricks


Watch Now