Stress in Tech Interviews // a recent NCSU research paper

The North Carolina State University (NCSU) published their research on stress in tech interviews. You should not be surprised: it's not looking so good (Press Release).

Their main finding is that the typical on-site, realtime whiteboard coding challenges cause so much stress that they really test more how well you can perform in front of others under pressure and less about how good your command of the actual topic is.

Luckily, the authors give some suggestions ("guidances") on how to improve the tech interview process in the Discussion section [My personal elaboration under the quoted headlines]:

"Guidance I - Use retrospective think-aloud for accessing explanation"

Let people work on a solution and a way to present it by themselves, unobserved. Then le them explain their thinking to the interviewer as a second step.

Of course, "live" brainstormings / discussions are part of what a developer would do in a team, but the situation is not comparable. If you are an accepted team member and stand in front of a whiteboard you already psychologically belong and you are aware of the problem.

"Guidance II - Evaluate the kinds of stress necessary for position"

Ideally, the interview is evaluating a candidate on things relevant for a job. Some type of stress might be part of the job, other kinds not. "Performing" (as on a stage) to solve code problems is usually not a job requirement. Even an important technical presentation is usually prepared and rehearsed.

I could imagine that if you hire for a "consultant" developer that often has to function in unknown situations in front of strangers, maybe this style of interviews is more appropriate.

"Guidance III - Provide accessible alternatives"

Several suggestions by the authors to overall reduce stress in the process:

• A "warmup interview" - an opportunity to settle-in and learn about the format of the real interview, allowing to ask questions.

• Allow for a "free reset". Basically, a second chance (different problem) in case the candidate just freezes on the
  first one - I would imagine just knowing that there is such an option will calm down candidates, even if they never make use of it.

• Give "Partial program sketches" - Instead of starting from a literally blank whiteboard / paper, give some part of the solution and ask them to fill in the rest. Should work for architecture sketches as well.

• Provide "familiar affordances". Allow the candidates to use paper or a laptop if that works better for them.

"Guidance IV - Consider impacts on talent and diversity"

Missing out on candidates that are well able to solve the problem but not handle the stress, is the obvious problem causing the wrong kind of stress in interviews.

Furthermore, the authors point to research that shows women are more affected by performance and test anxiety and thus might be at a disadvantage regardless of their actual qualification for the position.

Other groups might be disadvantaged as they might be lacking access / resources to prepare for the stress of a typical whiteboard interview, whereas some candidates can prepare extensively.

What I really like about this paper is how it combines proper research with applicable suggestions. I still want to interview people on technical topics, but I am now way more aware on how my design of the process might lead to distorted results and unnecessary suffering on the candidate's side. It is also good to see that there seems to be a growing body of research on (tech) interviewing.

I definitely recommend to read the original paper, as they authors argue way more nuanced and provide interesting references for all their points:

Behroozi, Mahnaz, Shivani Shirolkar, Titus Barik, and Chris Parnin. 2020. “Does Stress Impact Technical Interview Performance?” In ACM ESEC/FSE ’20.