What's Wrong with the Tech Interview Process?

There seems to be a growing consensus that the interview process often adopted at tech companies for hiring technical staff, in particular developers, is broken. Almost a year ago, I featured it as a major problem in a post discussing the issues facing tech today and, suffice it to say, it hasn't been fixed in the time since.

One need not even look far to find it being discussed on Twitter or Hacker News of across a variety of blogs. The issues seem to boil down to three things:

1. Coding tests are arbitrary, needlessly difficult and disconnected from the skills actually required for the job.
2. The number of rounds and the time demands of interviewing are difficult to manage.
3. Hiring decisions often seem arbitrary and communication about why someone failed a stage are often poorly communicated, when they are communicated at all.

Let's take a closer look at each of these.

Coding Tests and Whiteboards

Let's start with coding tests and whiteboarding since it is probably the most common frustration that developers cite.

The primary frustration developers express around coding tests have to do with them being needlessly difficult and frequently arbitrary. Many people describe being asked about complex computer science concepts and algorithms that they may have studied in college (assuming they were a CS major) but haven't had the need to reference since (or if they did, they just look it up).

I shouldn’t have to rely on an algorithm question lottery to demonstrate my skills and abilities during an interview. Luck and chance should not be part of the tech interview process... It’s like testing your dentist’s skills based on their ability to balance a redox equation using the oxidation number method from their undergraduate chemistry studies.

• Sahat Yalkabov, F*** You, I Quit — Hiring Is Broken

Part of the problem with the code test or whiteboarding is that it doesn't seem to be relevant to the job it is designed to screen for and often seems included as a way of gatekeeping.

And just how have these folks determined this definitive fact that I am not technically strong enough? Care to guess? If you said, "a live coding test with a random brain-teaser type problem you'd find on codewars.com," you'd be right. Now don't get me wrong, I have also passed interviews with these sorts of tests. Hmm.. Isn't that odd?

• Ambrose Little Technical Interviewing is Broken, But We Can Fix It

To me, there are two major issues with coding tests. The first is the disconnect between these coding tests and the supposed goal for which they were designed, which is to determine the skill and knowledge of the interviewee. It seems to me that a body of work - whether it be their education, work at a company, contributions to open source or side projects - is more illustrative of someone's technical skills than a random code challenge brought completely out of context. There is nothing especially unique about the job of being a developer that requires answering random technical trivia, but it seems to be a practice that is unique to this role.

More importantly, the second problem is that the practice prioritizes random technical knowledge (something that can relatively easily be learned) over other aspects like personality fit, ability to work with others, work ethic, eagerness to learn (things that cannot be easily learned). You can be a brilliant coder and a crappy person but I can tell you which part of that equation will have more impact your success in a developer job.

Extensive Time Demands

Tech interviews are often marathons. Many roles require extensive rounds of interviews in addition to time-consuming "homework" assignments.

This doesn't seem to be exclusive to the large, household-name tech companies. Across the board you hear stories of tech companies treating the process as if it is designed to weed out candidates who simply cannot make it to the end of a grueling race.

More and more employers are adopting pointless, expensive and time-consuming extra processes to make their recruiting pipelines even slower and more off-putting to candidates.

• Liz Ryan, No, You're Not Crazy -- The Hiring Process Is Broken

This is compounded by a slow or completely broken communication process.

I've gone through half-day to full-day interviews called "running the gauntlet" only to wait a month for them to say no or nothing at all. It's a part of the process and we all get over it. (shrug) Still, it's frustrating when you have to use precious vacation time to go on interviews that lead nowhere.

• Milecia McG, The Crazy Job Search Process

Often the communication breakdown seems to break down for a candidate who isn't continuing in the process, which is certainly unfair to the candidate left hanging. However, not infrequently, the communication is poor or nonexistent even for candidates who are moving to the next stage but are often left to wait weeks before that is communicated.

This seems entirely self-defeating for companies who are hiring in at least two ways. The first is that it pointlessly limits your potential candidates to those whose finances and/or job allow them to survive the countless missed hours and days required. Second, it misses out on candidates who drop out of the process because they either get frustrated with the lack of communication and length of the process and the other are, of course, people who simply find a different job during a process that need not be this protracted.

Arbitrary Hiring Decisions

The biggest issue seems to flow from the prior two, which is that rejections can seem completely arbitrary - so much so that there is a site dedicated to sharing rejection stories filled with recognizable names in the developer community.

Even many people involved in the hiring process seem to recognize that it is flawed.

Part of the problem stems from the screening process (something the stories on rejected.us frequently validate), which, according to the study cited below looking at Y Combinator hiring, accounted for nearly half of applicant rejections.

Most companies reject a high percentage of applicants during a recruiter call (or resume screen). Across the 25 companies we interviewed, an average of 47% of applicants were rejected in this way (the rate at individual companies went as high as 80%, and as low as 0%). The recruiters doing this rejecting are non technical. All they can do is reject candidates who don’t match the profile they’ve been taught to look for.

• Ammon Bartram, Who Y Combinator Companies Want

As the study notes, in many cases, this is because the screener generally isn't technical and therefore must rely on profile information they have been supplied. That's not to say that the screener should be technical, I believe there is a ton of merit in including people trained in and focused on recruiting, but it means we need to do a better job of making sure they understand the profile and are supplied with the information they need to properly evaluate folks.

The other problem is back to the coding tests. Their seeming trivia-like nature seems to lead to unpredictable results. Basically, good candidates fail a good percentage of the time, as noted in the following research.

Roughly 25% of interviewees are consistent in their performance, and the rest are all over the place...

To me, looking at this data and then pretending that I had to make a hiring decision based on one interview outcome felt a lot like peering into some beautiful, lavishly appointed parlor through a keyhole. Sometimes you see a piece of art on the wall, sometimes you see the liquor selection, and sometimes you just see the back of the couch.

• Aline Lerner, Technical interview performance is kind of arbitrary. Here’s the data.

The combination of screening, testing and time can mean companies hiring processes are not necessarily optimized for finding the best candidates.

This Isn't an Easy Problem to Solve

Look, I get it. It takes time and effort to interview someone, and most of you just want to get back to building stuff. Coming up with a standard question lets you get away with doing more with less effort, and gives you a modicum of an ability for comparison across different candidates.

But really take a long look at whether this selects the right candidates.

• Zach Holman, Startup Interviewing is F****d

As Zach notes, this is a difficult problem to solve, which may explain why the system seems to be currently supported by a combination of "that's the way we've always done it" and "that's the way everyone else is doing it." This reasoning gives companies the excuses they need to avoid taking a hard look at their processes to see if they are actually achieving the goals they are supposedly designed to achieve. Most companies seem to believe that, at worst, this system gives them a ton of false negatives but not false positives.

It seems that companies would benefit from asking themselves:

• Does our code test or whiteboarding exercise really improve our ability to identify good candidates? Or are there more relevant exercises (a reasonable homework assignment related to the day-to-day job requirements) or criteria (project experience or open source work that can be judged en lieu of a quiz or homework) that might be more useful?
• Do we really need full day interview marathons or 5+ stages of the interview process? Are there stages we can cut out and still be equally effective? Are we losing good candidates just because they cannot afford to complete our process?
• Are our screeners given the information they need to properly screen people or are we potentially losing highly qualified candidates? Are our criteria too specific (ex. looking specifically for Angular when JavaScript or JavaScript framework experience would suffice)? Does our process seem to result in a high degree of false negatives?

These aren't easy questions to answer, for sure, and, obviously, lots of good candidates still manage to make it through. But I think of it this way: As developers, we've developed system of unit testing to help prevent us from deploying buggy code. These tests don't always check for issues that would cause the whole application to break down. Some might cause the software to work improperly some significant percent of the time or for some significant percentage of users depending on specific criteria. However, we wouldn't allow our developers to deploy code that failed these tests just because the application might work for some users some percentage of the time. Our interview and hiring system is currently failing a ton of unit tests. Let's fix it.