Death of the Coding Test: Interview Methods that Better Evaluate Candidate Competency

A little over 7 months ago, I wrote a blog that made a much larger impact in the developer community than I thought ever possible...

The feedback to my blog was way bigger than I could have ever imagined and the coolest thing about it all was that it elicited a varying degree of responses:

• Strong Agreement
• Strong Disagreement
• Hilarious Memes
• Vigorous Debate in the comments
• Insults directed at my writing ability 😅

And amongst all of the different responses and comments, one particular thought and question really stuck with me...

If we don't do Coding Tests, What interview approach should replace them in the developer interview process?

In this blog, I hope to answer that question. I'll share some interview approaches that I believe are better (or at least as good) at evaluating the competency of developers, engineers, and programmers for roles at your company or on your teams.

Note: I would love to hear other's thoughts on this as well, so please share your thoughts in the comments below!

Let's Set the Record Straight

Before I share my list of best interviewing approaches, I would like to make sure I convey my opinion on coding tests as succinctly as possible.

I believe that coding tests, as they are currently implemented in the tech industry, are not the best method for evaluating candidate competency or future "on-the-job" performance.

Coding tests take on several forms but essentially challenge the test taker to answer a question or a set of questions that are used to evaluate the takers understanding, knowledge, or competency on core Computer Science or programming fundamentals. Tests can consist of algorithm based challenges, tricky coding problems, and sometimes even questions to assess a test takers knowledge on Computer Science principles and/or programming design patterns.

Coding tests are somewhat standardized across the tech industry but I do not believe that they are a great gauge for evaluating the diverse sets of skills that candidates posses and do not consider the wide range of backgrounds that candidates bring to the interview process.

There are some uses for coding tests in the industry and I don't believe they should be completely thrown away but I do believe they are only truly useful for a smaller portion of the development population and should not be considered the industry standard.

What Do We Do Without Coding Tests?

Something Better!

One nice thing about me being in the tech industry for as long as I have been, is that I've been in A LOT of interviews. Interviews to find new jobs, interviews to join new projects and interviews needed for promotions.

That has given me a broad amount of experience as interviewee, as well as interviewer. And in my current role as Developer Relations Manager @ Asurion, I have the opportunity to talk with developers and engineers on a very regular basis and have learned a lot about how to best understand and evaluate the varied and broad skills of any particular programmer.

So here is my list of better alternatives to coding tests:

1. Take Home Task or Project Assignment

This approach gives the candidate a programming or coding task that could be considered a "take-home assignment". The interviewee would be tasked with solving a programming problem or project challenge that resembles something similar to what they would do in their day-to day in their respective role. In most cases, the candidate is given a timeframe in which to finish the assignment and is expected to return their solution to the interviewee once completed.

I have personally been evaluated in the interview process using this method and feel that it was a great way to measure my skills. In one case, I passed the project assignment with flying colors resulting in a job offer and in another case, I performed decently but not as well as other candidates in the process and was not extended an offer.

In both cases, I felt great about the results. Because the interviewer/company did a great job setting up the parameters and task to reflect what would be needed for the job, I felt it was a great reflection of my actual skill at accomplishing the given task.

I think this a great method for roles that might need to evaluate a candidate on a host of different skillsets; Coding skill, problem solving, critical thinking and time management.

This may not be the best approach to use if you are on a restrictive timeline for finding a candidate for a role. The reason being is that since you are asking people to complete a take-home assignment, you will need to give them the ability to find time in their schedule to do the assignment. For some, they may be able to complete the assignment quickly but for others, who may have busy schedules or a lot of home responsibilities, it may take a bit more time to fit completing the assignment into their schedule.

So here are a few more pros and cons for using this approach:

Pros

• Reduces Candidate Performance Anxiety: Candidates are given the ability to select the best timing and environment to accomplish the task. For those similar to myself, this can help them to perform closer to their actual competency level while reducing anxiety that would heavily affect their interview performance.

• Customizable Evaluation Approach: Interviewers can create a more bespoke experience for the assignment depending on the candidates skill level, position requirements or time constraints. Interviewers can add or remove objectives as needed to accommodate interviewee.

• Two-Stage Evaluation: This works great for evaluating a candidate on coding and programming competency and evaluating their communication and actual understanding of their code. If after receiving a candidate's returned assignment, if you believe they performed well enough to continue in the interview process, you can now use the take-home assignment in a future step of the interviewing process. You can now ask them questions directly about their code and how and why they made their decisions to better understand their critical thinking and communication style and skills.

Cons

• At Home Tech Requirement: One thing to be aware of is that not every candidate has the same tools to work with at home. Some may have the skills to perform on the job but not the hardware to perform the take-home assignment reasonably. This needs to be a consideration and should not remove a candidate from the interview process if they lack the personal tools or hardware to accomplish the task at home.

• Candidate Time Constraints: As I mentioned a bit earlier, not all candidates have equal time capacity in the interview process. Be aware that this method may not be great for candidates who only have a limited amount of time to participate in the interview process. Consider using a different approach if a candidate expresses that they may struggle finishing or setting aside the time for the take-home assignment.

2. Pair Programming Session

Another great way to evaluate a candidates ability and to see how they may perform in the working environment is to conduct a pair programming session in the interview process.

The pair programming session approach tasks the interviewee at solving a problem or tackling some task with the guidance and assistance of an interviewee. This approach can be done in-person or virtually and can easily be adapted to best match the candidates needs.

I have experienced pair programming-esque techniques being used all throughout different interview processes but I want to make sure I clarify something about how this should be implemented.

This is actual pair-programming NOT a proctored coding test.

I have participated in interviews where it was explained to me that there would be a pair programming session only to find that I was given a task and then was simply watched and questioned about my coding decisions as I accomplished the task.

That is not how pair programming works and not how pair programming should be implemented into the interview process. Pair programming at it's heart is a collaborative effort where two developers try to work together to complete some task.

In the interview process, this is exactly the same. The interviewer and interviewee work together to solve a task while learning from each other. In the interview process, this gives the interviewer a chance to really see how a candidate would work in pair programming environment.

A candidate should not feel as if they are trying to get the problem given to them "right" or "wrong" but should be encouraged to operate as realistically as possible. This can be difficult in the interview process but can be extremely effective on evaluating a candidate.

A huge key to making this approach effective is to make sure to pair program on a problem or task that reflects the actual job. It can be a wasted opportunity if not working on a task that reflects real on the job responsibilities.

So here are a few more pros and cons for using this pair programming approach:

Pros

• Code in Realtime: This gives the candidate an opportunity to code directly in front of the interviewer to get their skills evaluated. It can be sometimes hard to get an actual gauge of a candidates' coding proficiency without actually seeing them code.

• Low Pressure but Some Pressure: This a good opportunity to see how a candidate performs under some form of pressure in the interviewing process. As natural as you attempt to make the pair programming process, it will still feel like some what of a test to the interviewee. This can give you an understanding of how a candidate performs under pressure. Though I am an individual who can suffer from performance anxiety, I do think it is appropriate to push candidates especially when they are likely to encounter pressure situations on the job.

Cons

• Finding the Right Interviewer: The most difficult aspect of this approach is finding the right person to conduct the pair programming. The interviewer should be familiar with the pair programming process but, more importantly, should have the temperament to conduct this portion of the interview. They should have the ability to not only evaluate the candidate's skill but also the ability to work with and support the candidate. To get the best results, the candidate should feel as they are allies with the interviewer and tackling the task alongside them.

3. The "How Would You Solve..." Prompt

In this approach, the interviewer asks the candidate to try and solve a real or hypothetical problem that they would likely encounter on the team they would be joining. The interviewer gives the interviewee a prompt with the basic details of a problem that the team is trying to solve and asks the interviewee to walk them through how they would solve it.

It's extremely important to express at the very beginning of this process that the candidate can ask any questions to get more details on the problem. This makes a big difference because it lets the candidate know it's okay to communicate with the interviewer to get clarity.

One great thing you get by using this process is the ability to evaluate how the interviewee thinks though problems and how they come to conclusions. If the interviewee asks very few clarifying questions or if they ask a lot of clarifying questions, you can get perspective of their experience with dealing with technical, development, and general problems on the job.

Depending on the role that the individual is interviewing for, you can get an idea of if they have the critical thinking and experience needed for the role (and even if they may have more experience and knowledge than the role requires).

Similar to other interviewing approaches, I do not believe interviewers should be looking for the correct solution to the problem to be presented but how the interviewee got to the solution. We can sometimes detract from noticing the benefits of how a person came to their conclusion if we are too focused on them getting the "correct" solution. And as in most things in life, there is rarely a right solution to any problem but solutions with different advantages and disadvantages.

Personally, this is one of my favorite approaches in evaluating a candidate. Sometimes there are individuals who can perform really well in live coding and coding test scenarios but lack insight on how to get to a solution organically. They are very technical and studied individuals who perform well under pressure but can lack critical thinking skills that really matter on the job or in their role.

So let's look at a few more pros and cons for the prompt approach:

Pros

• See Their Thinking: As I mentioned before, this method is a great way to get into the mind of the interviewee. Instead of looking at the results of a test or even some static code where you might need to interpret intention or how they might have came to some conclusion, you get to see how a particular candidate thinks. In the long run, you want a great critical thinker on your team, not just a good coder.

• Gauge Future Team Impact: One benefit of getting a chance to hear how someone solves a problem is that you can sometimes identify how someone will impact your team. Something they may say or share may indicate that they are able to bring something new or needed to your team that would have never been identifiable through code.

Cons

• Be Wary of "Good" Talkers: There are particular individuals who excel at verbal communication and can even at times deceive the most adept of interviewers with clever words. It's important to follow-up and ask deeper questions to determine if someone knows what they are actually talking about or just knows how to speak in a way that seems to indicate understanding.

• Best When Used with Other Approaches: This could likely be said for any interview approach, but this rings even more true for this method. In an interview process it is always a good idea to get multiple performance indicators to help you better understand a candidate's possible future performance. This approach can help identify strong critical thinking skills but is not always the best way to evaluate more technicals skills like coding ability and understanding.

4. Personal Project Code Review

Using this approach, an interviewer asks to review a candidate's personal projects, open-source code or any other technical work that they have done outside of being employed in a traditional sense.

This allows candidates to self-select the technical measure on which they will be evaluated and gives an interviewer an idea of what the candidate believes is representative of their skillset. The assumption is that whatever work or code a candidate shares with the interviewer is likely work or code they are particularly proud of and something they would stand by.

When used, this approach should be implemented as a supplemental tool or an optional approach in the interview process. The truth is that not every great developer or programmer codes in their free time or has interest in working on personal projects outside of their normal daily work on the job.

Just because a candidate does not have a personal project or code to share does not mean they are any less of a programmer than an individual who does.

Another thing to note is that it is crucial to not only evaluate their code without them present and then make a determination about their skills. It is important to review their code and then discuss with them the how and why behind their code.

There may be instances where an individual makes, what seems to you, a terrible coding decision but there may be a very specific reason that actually is very clever and not intuitive at first glance. Ask questions about the code, the thinking around the code, and find ways to evaluate their skills in their entirety.

Here are a few more pros and cons for the personal code review approach:

Pros

• Can be used as Substitute to Other Technical Evaluations: If a candidate has a substantial amount of personal project code that is available, there are some circumstances where you could use this approach in lieu of another (E.G. Personal Project Evaluation as substitute for Take-Home Assignment). I would recommend making this approach optional in early stages of the interview process as it may allow you to "skip" other parts of the interview process, thus shortening the overall interview experience.

• See a Different Coding Style: From my experience, people tend to code differently in their personal projects than they do on their job. This gives an interviewer some insight into the candidate's personal coding style and competency when the only person who will truly evaluate their code is the candidate themselves. It also gives you a glimpse into their general documentation style as they will likely only document what they deem to be important to document.

Cons

• Might not be their Code: One thing to be aware of is that when viewing a candidate's personal project is that not all the code may be their code. In our development culture with the existence of StackOverflows and Dev.to-like sites, there is a chance that someone copy-pasted their project to "look good". It is extremely important to ask the interviewee what made them make their decisions and to hear their perspective and thinking around the highest quality and lowest quality sections of their code.

• Project Code can be Dated: If code in a project was created long enough in the past, it can present two problems; the code reflects their skill when they were at a lower level OR the code reflects their skill at a level they are not currently at anymore. It is important to have context on when the code was written to understand if the code represents their current skill level or a skill level benchmark of the past.

5. The "Take a Ticket" Approach

In a similar fashion of the "Take-Home Assignment" approach, this method involves giving the candidate a problem to solve or a feature to implement. How this differs is that this would happen during the actual interview, may it be virtual or in-person, and the candidate is given a set amount of time to work on the problem.

To make this approach the most effective, it is a good idea to have the interviewee to solve a problem or implement a feature that is "real" or directly reflects what would be on the job. It's also important to give the candidate access to the same or similar resources they would have on the job and not to "test" them on knowledge that wouldn't matter in a real on-the-job scenario.

To make the "ticket" they would have to solve as real as possible, I would encourage either giving them access to real code (that is not proprietary, running live or would not effect your business advantage in any way) or to make a "dummy" repository with older or slightly modified real code that they could work with. Give them a specific "ticket" that would likely need to be resolved in the real-world and let them code away.

If the interview was happening in person, you could even place them at a desk in your office to help simulate their future work environment as closely as possible.

In this approach, I would recommend going for a more proctored approach where you let the candidate code but do not directly interact with them unless they ask a question relevant to the ticket. You could also, let them work on their own without an interviewer present but you might miss the opportunity to answer any crucial questions they may have or view how the candidate codes and works.

All and all, the goal is to mimic what they would actually be doing on-the-job as closely as possible and to give the opportunity to operate in the method that best suits them.

Let's now look at the pros and cons for the "Take a Ticket" approach:

Pros

• Mimics Real Work Experience: This is a great way to gauge how a candidate might operate in a real-world work environment. If set up correctly and if the parameters are communicated effectively, you might see a side of the interviewee that you may have never seen in a "traditional" interview process.

• Helps Candidate Evaluate if Work Environment is a Good Fit: One thing I have not mentioned much in this blog is that it's also important for the candidate to evaluate if the job or role they are interviewing for is a good fit for them. This can be a great opportunity to help the candidate feel out what the job might be like and if it's a good match for them.

Cons

• More Effort to Implement: Though this method can pay dividends in the long-term, it does take some work to implement correctly. You have to set up the work environment, the code base/tickets, need a solid interviewer to proctor and need to set aside the time in the interview process to make it happen. Given the upfront effort it may not be the best fit for everyone.

• Be Wary of Performance Anxiety: Just be aware that some candidates may perform poorly in the proctored format of this approach. It may be useful to think through an alternative implementation if the interviewer notices the candidate is suffering from an episode of performance anxiety. Try your best to take that into consideration.

---

Conclusion

If there is one thing I'd like you to walk away with in this blog is that there are lots of solid alternatives to the traditional Coding Test that has become standard in our industry.

Just like with any interview method, no method on its own is perfect for every scenario and for every position/role that you may be interviewing candidates for. In my personal opinion, to get a more full understanding of the competency of a candidate, it's best to implement two or more of these methods in the interview process.

I'm not a fan of very long multi-stage interview processes BUT I do believe it is very difficult to evaluate a candidates technical ability or potential through one individual technical screening. And in some cases, using just one approach can create false positive and negative indicators that may not allow you to select the best candidate for the job.

As I mentioned in my previous blog, I think using a blanket approach for evaluating candidates, no matter the approach, may be more efficient but shows a lack of care for candidates and increases the likelihood of hiring the wrong person for roles you are attempting to fill.

I truly believe the effort you put into creating an interview process and the consideration you put into approaches you use to evaluate candidates, will determine not only the quality of candidates you hire but will have direct impact on their sentiment of your company and teams as they go through the interview experience.

Coding tests may be the standard but there are better alternatives that we can consider for interviewing and I encourage anyone who is in the position to impact the process to take some time to consider some of the alternatives I have shared above.

---

Are there any other alternative interviewing methods to coding tests that you have found to be effective?

Do you think any of the approaches I've shared are not as effective as they seem?

I Would love to hear your thoughts!

---

Thanks for Reading!

Bradston Henry

---

Photo Credits (as they appear):
Cover Photo by Pixabay
Photo by Tim Gouw
Photo by Any Lane
Photo by Christina Morillo
Photo by ThisIsEngineering
Photo by Pavel Danilyuk
Photo by ThisIsEngineering
Photo by Tima Miroshnichenko

---

Follow me on my Socials:

https://linktr.ee/bradstondev