How to advance diversity hiring with big data

Ideally, organizations would hire solely based on only the job-relevant skills required for effective performance in a role. However, many human decision-making processes, such as evaluation of job interviews, have the potential for both conscious and unconscious bias to seep into the process. These biases can also be reflected in algorithms trained to mimic human decision-making without proper oversight during the algorithmic development process. As a result, HireVue’s Science team has been studying ways to identify these sources of bias and minimize them in our algorithms to advance diversity hiring. 

Today, we’re excited to share our Science Team’s cutting-edge research that helps organizations achieve accurate and fair hiring outcomes. This research, entitled, “New Strategies for Addressing the Diversity–Validity Dilemma With Big Data” was published in the Journal of Applied Psychology, a prestigious peer-reviewed Industrial Organizational Psychology journal.

The study and why it’s important

A common finding in Industrial Organizational Psychology research is that the selection methods which demonstrate the strongest ability to predict job performance also demonstrate the greatest disparities in outcomes for demographic groups (e.g., men versus women). This phenomenon is commonly referred to as the diversity-validity dilemma. While previous research has demonstrated a variety of strategies for addressing the diversity-validity dilemma (i.e., optimizing both objectives), rapid advances in technology – the improved ability to collect and process large amounts of complex data and machine learning algorithms – are opening up new ways of addressing this dilemma.   This research introduced novel and more effective ways to address the diversity–validity dilemma that are emerging in the big data revolution.  This research benefits hiring practices in three primary ways:

1. It identifies a unified framework for reducing group differences, one that incorporates techniques used in machine learning and Industrial Organizational Psychology. 
2. It outlines two techniques for reducing group differences that are well-suited to large data sets.
   • The first selectively removes predictors in a series of iterative steps. While selectively removing predictors is a strategy that has been used historically, we applied this strategy at scale and demonstrated how it can be used in a big data scenario. 
   • The second technique is a new approach that incorporates a group difference penalty into the machine learning model optimization to explicitly optimize the diversity–validity trade-off (called multipenalty optimization). 
3. It compares these two techniques in a real-world selection scenario and demonstrates that multipenalty optimization more effectively reduces demographic differences while maintaining predictive validity.

Implications for improving diversity hiring outcomes

Both mitigation techniques evaluated in the article effectively minimized group differences, while multipenalty optimization more effectively controlled for group differences while maintaining predictive validity. Being able to successfully replicate human evaluators and simultaneously minimize group differences using machine learning techniques can provide organizations fair and objective ways of assessing job applicants. This can be especially useful in high-volume personnel selection contexts.

While these techniques alone are powerful for improving fairness in hiring, they are even more effective when combined with other DEI strategies. For instance, sourcing candidates from more diverse talent pools, using interview questions designed to be inclusive, and training evaluators to only focus on job-relevant criteria can all reduce the chances of bias occurring in a selection process. 

Fair and Inclusive Hiring

At HireVue, we acknowledge the impact of work and believe that every person deserves the opportunity to connect to new jobs. We’ve seen firsthand how dismantling biases in the hiring process can change the lives of candidates, that is why we decided to make these techniques publicly available to allow other researchers and organizations to apply these techniques. This will benefit society and hopefully fuel future research into this important topic. We hope that more people will join us on this journey to connect talent to opportunity in 2023.

Read the full research paper at the Journal of Applied Psychology.

---

Authors:

Caleb Rottman, Ph.D., Senior Data Scientist; Cari Gardner, Ph.D., Senior IO Psychology Consultant; Josh Liff, Ph.D., Director of Assessment Psychometrics; Nathan Mondragon, Ph.D., Chief IO Psychologists; Lindsey Zuloaga, Ph.D., Chief Data Scientist