LW - How I build and run behavioral interviews by benkuhn

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How I build and run behavioral interviews, published by benkuhn on February 27, 2024 on LessWrong.
This is an adaptation of an internal doc I wrote for Wave.
I used to think that behavioral interviews were basically useless, because it was too easy for candidates to bullshit them and too hard for me to tell what was a good answer. I'd end up grading every candidate as a "weak yes" or "weak no" because I was never sure what bar I should hold them to.
I still think most behavioral interviews are like that, but after doing way too many behavioral interviews, I now think it's possible to escape that trap. Here are my tips and tricks for doing so!
Confidence level: doing this stuff worked better than not doing it, but I still feel like I could be a lot better at behavioral interviews, so please suggest improvements and/or do your own thing :)
Before the interview
Budget 2+ hours to build
That's how long I usually take to design and prepare a new type of interview. If I spend a couple hours thinking about what questions and follow-ups to ask, I'm much more likely to get a strong signal about which candidates performed well.
It might sounds ridiculous to spend 2 hours building a 1-hour interview that you'll only give 4 times. But it's worth it! Your most limited resource is time with candidates, so if you can spend more of your own time to use candidates' time better, that's worth it.
Think ahead about follow-ups and rubric
I spend most of those 2 hours trying to answer the following question: "what answers to these questions would distinguish a great candidate from a mediocre one, and how can I dig for that?" I find that if I wait until after the interview to evaluate candidates, I rarely have conviction about them, and fall back to grading them a "weak hire" or "weak no-hire."
To avoid this, write yourself a rubric of all the things you care about assessing, and what follow-up questions you'll ask to assess those things. This will help you deliver the interview consistently, but most importantly, you'll ask much better follow-up questions if you've thought about them beforehand. See the appendix for an example rubric.
Focus on a small number of skills
I usually focus on 1-3 related skills or traits.
To get a strong signal from a behavioral interview question I usually need around 15 minutes, which only leaves time to discuss a small number of scenarios. For example, for a head of technical recruiting, I decided to focus my interview on the cluster of related traits of being great at communication, representing our culture to candidates, and holding a high bar for job candidate experience.
You should coordinate with the rest of the folks on your interview loop to make sure that, collectively, you cover all the most important traits for the role.
During the interview
Kicking off
My formula for kicking off a behavioral question is "Tell me about a recent time when [X situation happened]. Just give me some brief high-level context on the situation, what the problem was,1 and how you addressed it. You can keep it high-level and I'll ask follow-up questions afterward."
I usually ask for a recent time to avoid having them pick the one time that paints them in the best possible light.
The second sentence (context/problem/solution) is important for helping the candidate keep their initial answer focused - otherwise, they are more likely to ramble for a long time and leave less time for you to…
Dig into details
Almost everyone will answer the initial behavioral interview prompt with something that sounds vaguely like it makes sense, even if they don't actually usually behave in the ways you're looking for. To figure out whether they're real or BSing you, the best way is to get them to tell you a lot of details about the situation - the more you get them to tell you, the harder it w...