I had the pleasure of designing and evaluating these tests for the last few years and I can confirm that Mistake 1 and Mistake 2 are the most common reasons for failing these tests. Failing here is an instant fail.
But I would also be very careful with Mistake 3! Not every company wants a solution to their tests publicly available. So if git is in clear violation of 1 or 2, then do not use git at all!
Luckily there are now free private repos on github so I suggest to always make it private and invite the reviewer – or ask when in doubt.
I also agree that it is absolutely OK to have questions about the test. This can actually be part of the test! Never hesitate to ask questions! This is also true for the following technical interview of course.
To Mistake 7 I would add that next to the instructions you should also note your motivation on why you solved it the way you did. Explain why a certain technology was chosen and why some alternatives where not.
I have evaluated over a hundred technical tests in the last 2.5 years and it helps a lot if we get a short – and I mean short – summary of your thoughts during development.
I did test for several companies on both sides of the game.
I never had the case that companies forbid to open source their tasks.
Do you know the reason why they would do that?
The only point I can see so that people don't copy that solution but yeah in the question round after you would invite the candidate it would be pretty easy to see that this is not his or her code ;)
A test can be quite long lived. Assume you are looking for 10 software engineers (of same position, but different levels). Then you probably have to interview 100 candidates over the course of several months.
Even if I would design a test for each level (working-student, junior, intermediate, senior, principal ... Too much!) copying is still a possibility, e.g. choise of technology.
An openly constructed test that tests for skills, mindset and approach is much better suited in these situations. You learn much more too and you have an intro into the technical interview as well, as you will start asking about the test a lot – hopefully leading into a meaningful conversation.
This also allows you to actually judge the level of the candidate instead of hoping he applied for the right level.
A good test does not require a specific solution nor a specific level of knowledge but probes and challenges a candidate on all levels. It is also only the foreplay to the technical interview.
Valid points but our tests don't say anything about how to implement it.
I would never say in a test using this lib and this framework.
Usually, the candidate has their freedom. So if someone would come with the exact solution it would be very obvious.
Like for a full stack dev using exactly the stack would be amazing because you have so many stacks, libs, frameworks out there.
Yes its most of the time never about the actual implementation. It about will the candidate fit into the company/team and what is his mindset.
A big point for me is also will he/she function in a team.
We all know the rockstars that write "the best" code but nobody except them understands what is happening because he or she refuses to play well with the team. Also, other things that are more about soft skills than the actual coding skills.
I see the point that for some companies it can be a problem that the solution for the task is somewhere on GitHub.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
I can absolutely agree.
I had the pleasure of designing and evaluating these tests for the last few years and I can confirm that Mistake 1 and Mistake 2 are the most common reasons for failing these tests. Failing here is an instant fail.
But I would also be very careful with Mistake 3! Not every company wants a solution to their tests publicly available. So if git is in clear violation of 1 or 2, then do not use git at all!
Luckily there are now free private repos on github so I suggest to always make it private and invite the reviewer – or ask when in doubt.
I also agree that it is absolutely OK to have questions about the test. This can actually be part of the test! Never hesitate to ask questions! This is also true for the following technical interview of course.
To Mistake 7 I would add that next to the instructions you should also note your motivation on why you solved it the way you did. Explain why a certain technology was chosen and why some alternatives where not.
I have evaluated over a hundred technical tests in the last 2.5 years and it helps a lot if we get a short – and I mean short – summary of your thoughts during development.
If you are sending the task and require it to be private you could initialize a private repo on your company Github for them.
Yes, this is also a valid solution I have seen. Not every company has github or uses git though. There are other valid alternatives.
Of course, use whatever version control system you want.
I mean this are general advises and companies will have there own requirements and you can replace git with svn and github with bitbucket.
Thanks for the great comment!
I did test for several companies on both sides of the game.
I never had the case that companies forbid to open source their tasks.
Do you know the reason why they would do that?
The only point I can see so that people don't copy that solution but yeah in the question round after you would invite the candidate it would be pretty easy to see that this is not his or her code ;)
Yes. It is mainly to prevent copying solutions but also to prevent other companies from copying the test.
We explicitly asked for zip files and not a (public) git repo.
Interesting. We change the test depending on the level of the candidate.
Do you only interview one per level/position?
A test can be quite long lived. Assume you are looking for 10 software engineers (of same position, but different levels). Then you probably have to interview 100 candidates over the course of several months.
Even if I would design a test for each level (working-student, junior, intermediate, senior, principal ... Too much!) copying is still a possibility, e.g. choise of technology.
An openly constructed test that tests for skills, mindset and approach is much better suited in these situations. You learn much more too and you have an intro into the technical interview as well, as you will start asking about the test a lot – hopefully leading into a meaningful conversation.
This also allows you to actually judge the level of the candidate instead of hoping he applied for the right level.
A good test does not require a specific solution nor a specific level of knowledge but probes and challenges a candidate on all levels. It is also only the foreplay to the technical interview.
Valid points but our tests don't say anything about how to implement it.
I would never say in a test using this lib and this framework.
Usually, the candidate has their freedom. So if someone would come with the exact solution it would be very obvious.
Like for a full stack dev using exactly the stack would be amazing because you have so many stacks, libs, frameworks out there.
Yes its most of the time never about the actual implementation. It about will the candidate fit into the company/team and what is his mindset.
A big point for me is also will he/she function in a team.
We all know the rockstars that write "the best" code but nobody except them understands what is happening because he or she refuses to play well with the team. Also, other things that are more about soft skills than the actual coding skills.
I see the point that for some companies it can be a problem that the solution for the task is somewhere on GitHub.