Introduction
Have you ever started an interview process with a company, and had them send you a timed coding challenge as a first round process? Do the questions feel like challenges that the developers of the company would not be faced with on a daily basis? If this has happened to you, and you're feeling frustrated, you're well within your rights to feel that way!
Remember These?
Timed Coding Challenges are largely incorrectly implemented. There are a number of factors here that play into this, including:
- Involving questions without any sort of context as to how it relates to the day-to-day challenges that the company's dev team faces.
- Stuffing a lot of questions into a time frame that makes it very hard to provide good answers in a stress-free environment.
- Writing incredibly long user scenarios that are time drains.
- Not utilizing the results of the timed challenge outside of moving candidates to a next step (more on this later).
Alright Jon, what are you getting at?
I'm writing this in hopes of changing some minds on the right way to use this tool. Personally, I don't think timed challenges (or any long take-home challenge of any kind) should ever be used, but that doesn't make for a very interesting article. Instead, I'll offer some ideas on better implementation of this process and how companies can turn it into a stronger screening tool.
Let's get to the do's and don't's!
Don't: Send arbitrary challenges that your dev team wouldn't encounter in their day-to-day.
Do: Select a challenge that your team has found a solution for, and offer it as a challenge (or something close to it) at this point.
You may begin
Examine a challenge your dev team solved that has been in a stable condition for a long time. Perhaps you have some logic around filtering the response of an API. Maybe you have some interesting conditionals and approaches you use to rendering data in different forms. Asking someone to do this as a challenge would be a great way to see how they approach something from their vantage point. It will also make it interesting and easier for whoever is reviewing the code to formulate opinions on the candidate quality!
Contrast this with asking more algorithmically focused question, like doing efficient counts of matching pairs in an array. Does this question fit with the day-to-day mindset and work of your engineers? If it does, then by all means, go for it. Otherwise, it might be harder to get the person reviewing this code engaged in that process and could cause them to reject quality candidates because of an insufficient screening process. Consider both sides of the coin!
Don't: Overload the candidate with many questions because of a perceived large time limit.
Do: Focus on asking quality, engaging questions that can be completed with time to spare!
Tick Tock Tick Tock
I and many of my peers have felt the pain of a 5 question, 90 minute timed exercise where 1 or 2 questions required a significant portion of time to read and think about before attempting an answer. Truth is, there's just no value to be found in having a large question pool given to your candidates. Copy and pasting questions from popular coding practice websites and calling it a day, for example, is very disingenuous and often how these situations arise.
Instead, if you're going the route of a timed interview, why not pare down the question pool to one or two questions, and give a candidate just as much time to complete them as if you had given them five questions? What do you have to lose by doing this? Better yet -- do your current engineers work in an environment where they are rigorously timed like this? If the answer is "no", why are you exposing candidates to an unrealistic environment through which you evaluate them by? This sets people up to fail, rather than giving them every opportunity to succeed.
Consider a simpler warm-up question (this one can be generic and straightforward), followed by a tailored question that we discussed previously. Those 90 minutes you were giving them for 5 questions? Give them the same amount of time here, if you're timing things. That way, you significantly reduce the pressure a candidate feels and you'll be getting work that they've had time to think about, review, and optimize!
Don't: Define every single detail of your question(s) down to the last character.
Do: Get to the point quickly!
Stories on stories on stories on...
I won't linger on this one for too long. Your coding challenges do not need a full blown narrative attached to them regardless of what the ask is. One or two sentences of context are fine, and then clearly defined expected outputs with sample inputs capture everything you need.
Don't: Forget all about this coding challenge once you move a candidate to a next step.
Do: Craft a challenge that can be iterated on so you can continue to use it throughout any further technical assessments in your interview process.
Collaboration is awesome!
I can say with certainty that the majority of timed coding challenges I have completed have been discarded once I move to the next stage in that process. Outside of off-hand references to the challenge, it simply would not feature in any way, shape, or form.
I am often left asking myself why any company would make candidates invest that much time in a challenge, only for it to be ignored as that candidates progresses through the process. Wouldn't it be easier to craft a challenge that a candidate could carry with them the whole way through? Think of the benefits!
- You can structure your entire technical assessment of a candidate around how they built something from scratch with just a few guidelines and some assets. Simplification for a low cost.
- You get a really in-depth look into a candidate's problem-solving process through a focused, well-defined challenge.
- Your candidates get the benefit of learning what it's like to collaborate with the existing engineers, and vice versa.
- You'll be doing candidates a solid by helping them to practice and refine their problem solving processes in different settings!
Conclusion
Hopefully this article gave you some ideas if you are able to influence the interview process at your company. Think a bit about what resources you have, what realistic challenges you can present to your candidates, and everything you want to learn about someone (and everything you want someone to learn about you!) before sending that timed challenge. You only stand to gain a better reputation, positive feedback, and higher-quality candidates by refining your process!
Top comments (10)
May be worth noting that such coding challenges are also a self-reinforcing pattern. The more people the process flunks, the more your people who passed it are likely to consider it valuable.
Three reasons behind that:
The higher the failure rate, the more it makes some feel like those who pass are "the best of the best."
The stronger that feeling, the more current employees may find their success at it feels validating, reinforcing a sense of self-worth.
Like fraternity members who had to go through hazing to get initiated, there can even be a sense of righteous injustice that new members wouldn't have to pass through the same level of Hell to join their ranks.
If you're seriously considering changing your process, that reinforcement and bias may result in serious push-back against the changes.
well said
Definitely can see how those biases will keep people from being okay with making these changes.
Your example rarely if ever occurs, as someone either catches that sort of thing in a code review, or the lists they work with will never be large enough to matter.
Competitive programming is an incredibly dubious proposition and I caution against anyone having that sort of mindset.
I don't agree that not being able to come up with the best way fast somehow means the different way will become a problem later. I'm not sure how you came to that conclusion, but I would need you to provide some extremely measurable examples of that before I'd concede the point.
Timing people and expecting them to produce their best work is dubious at best, incredibly presumptuous and unfair at worst.
I like this post and share your sentiments! Recently, I received via email a Technical Review question to do within 24 hours to be built in Java (server-client application). I believe the review was pretty generic as the role didn't even ask for Java as a skill.
Ugh that sounds maddening. I wouldn't blame you if you didn't take the time to do it.
Knowing a candidate can do a job is important so coding should be apart of screening. However, if you are going to do an automated coding challenge as part of your hiring process, there should be a couple of ground rules:
don't limit the candidate with things like "you can't use the Internet" -- being able to find a solution without reinventing the wheel is important
don't place a time limit -- that doesn't mean don't time the challenge, just have a conversation about why it took whatever amont of time. Did you use TDD? Did you struggle with a particular section?
assuming you have unit tests in the challenge that validate the solution, don't limit access to the test inputs and don't take away the simplest of debugging tools like writing out to the ui
don't assume that the way the challenge is written will be interpreted by the candidate correctly. It may not be as clear as you think and there should be two-way communication throughout the challenge.
understand that coding is not the extent of a dev job and communication and willingness to ask questions are equally, if not more, important.
I'd rather hire a candidate who will asks questions and demonstrates critical thinking over the rockstar that can hack out the fastest solution without tests. We canteach better coding, it's much harder to teach teamwork and communication IMHO.
Good points! Yeah -- coding being part of the screening at some point isn't the worst. I should clarify that I think that take-home challenges are not so great, and that I think doing the coding in a pairing session over a hangouts or phone call with a shared code env is what I prefer.
My point is that an interview setting is not the place to uncover that knowledge. That is far better suited for teams that are working collaboratively together already.