Does your audition process have accidental selection bias?
In recent, fascinating podcast episodes of Revisionist History, Malcolm Gladwell shows pretty clearly that the LSAT isn’t a great predictor of eventual success as a lawyer – but it is a very good predictor of whether you’ll get in a top-tier law school!
So when, in the episode, Antonin Scalia essentially says he’s only interested in picking clerks from a top-10 law school, he’s decided to select only clerks in the top 98th percentile of LSAT scores…even though the LSAT biases a particular kind of thinking that isn’t necessarily what Scalia wanted!
It got me wondering – does my audition process show any accidental selection bias? Are there times when I think I’m evaluating one thing but am in fact evaluating something completely different? And if that’s the case, is that a feature or a bug?
We should be clear about what we’re selecting for, why we are selecting for it, and ideally weed out any features we are accidentally selecting for – with a fair, equitable audition process that rewards what we seek to reward.