DEV Community

neoan
neoan

Posted on

Why I hate coding challenges in the hiring process

The concept

We need to ensure that people have the skills they claim to have. I get that and accept that.

But how relevant are these coding golf challenges really?

The investment

After experiencing assessments ranging from 5 minute multiple choice quizzes to 2.5 hour coding interviews, it seems to me that the commitment expected from a candidate keeps growing and growing.
And we should think about that for a second:

Is it really okay to ask a job-seeker to invest half an hour for a phone interview, 2.5 hours for a coding challenge, 1 hour for psychological online tests, ignoring the additional hour the candidate presumably put into writing a cover letter and learning about the position/company?

Isn't there an underlying impertinence in saying: before I risk investing one single hour for a face-2-face interview, I expect you to risk investing about 15% of a work week? And never mind the fact that it could potentially be the candidate who decides not to go forward after finally seeing the company from the inside. I mean, somebody must have understood the psychological principle of reciprocity.

But I digress. Let's get back to the topic.

EDIT: I was made aware in the comments (check it out if you are a thrill-seeker ;-) ) that the above paragraphs can be interpreted as "a candidate shouldn't be expected to show effort and invest time". As a clarification: I am referring to the trend of these procedures taking more and more untargeted time, and don't want to imply that a candidate shouldn't expect a vetting process (see The concept).

The conditions

So welcome to this online code editor you have never seen before after signing off that you will not use your beloved IDE. Why? Because apparently working in a familiar surrounding that you will actually be using on the job is not interesting to the employer. Otherwise you could "cheat" by utilizing code-completion, syntax warnings, testing and all the other tools that would normally help you to produce the outcome the employer expects. Test-driven development? No, that's boring. Hiding the existing tests and not informing you about the errors is obviously a way more realistic scenario. Well, there's always logging for such cases, right? Think again! With hackerrank, for example, I haven't found any method of getting anything logged unless I run all test scenarios, losing valuable time as their system compiles my code each time (JS, mind you).

The time

So yes, these tests are timed. Nearly always. That alone wouldn't be an issue, but it does influence the way you are writing code. You want to get done, so variable naming is something you don't care about and your functions are all of a sudden 30 lines minimum. And different files aren't a thing anyway, right? The result is really ugly code. Code that works, but code you would reject in any real life scenario. Also, code that isn't necessarily performant.
But worry not! Remember these tests you can't look at? Well, some of them test for execution time or memory and will give you feedback like: "Your code didn't return in the allowed time-frame". Ah, silly you! You though you are going to save a couple of minutes but instead will now spend additional minutes refactoring the code until the test is happy, without knowing what performance you have to target nor any kind of benchmarks at your disposal. Stressed yet, are we?

The challenges

Given an array of integers, your functions should return the sum of the indexes of the array elements that make up the sum of the two previous and two following array elements of the array element with the value of that sum.

Ready? Go! I kid you not. This is how your challenges are often formulated. Besides the fact that you will end up reading these tasks many times over and question your English comprehension skills along the way, it is absolutely beyond me how such scenarios are intended to be realistic in any kind of real-life scenario.

"They are not supposed to be. They are intended to show your problem solving skills", you say? Congratulations, that is what those tests claim. But let's think about that for a second: A good developer doesn't jump into these kind of things like that. The important question for such a task would be "Hold on, what are you trying to achieve?"

I cannot emphasize this enough! A good developer understands the requirements and helps the customer or manager finding a solution. That is problem solving: How to best achieve the actual need. Given the challenge from above, one would have to ask: "I see. What do you need the sum for? What do these indexes represent and how does the array come about?"

The tragedy

Have you been advised to train on Leetcode & similar? So have many. And while these systems are fun and do enhance your skills, I question how relevant performance in such scenarios is. I go as far as to say: not only does good performance in these challenges say little about your potential as developer (other than it indeed proofs knowing the targeted programming languages), it already produced candidates that are specifically trained for such unrealistic scenarios only. In the end, I have no use for people who can write functions that "remove the smallest array element(s) until the array is empty" or "bubble sort all characters of the alphabet until the concatination includes 'hamburger'". Not if these candidates fail to write something as common as a secure login component or sending out a notification email on form submission. Where are we going with this?

Top comments (63)

Collapse
 
mb34 profile image
MB34

I once went to a PHP Developer interview and was asked to write an application, don't remember what it was supposed to do, but the thing is, they handed me a pen and a notepad, clearly expecting me to write this application by hand.

I gave them a puzzled look and asked if they had a computer to do this, perhaps using even Notepad. They declined.

I handed them the notepad and said that I don't think I'd like working there and walked out the door.

Collapse
 
nombrekeff profile image
Keff

xD I wonder how many people did the same thing

Collapse
 
sroehrl profile image
neoan

Lol

Collapse
 
ahferroin7 profile image
Austin S. Hemmelgarn

Yeah, coding challenges as part of the interview process are a huge red flag IMO. They show that either:

  • HR has more influence over the hiring process than the department that the interviewee will be working in and doesn't understand what the department actually needs.

or:

  • The management of the department itself doesn't actually understand how real-world development works.

Both are generally signs of endemic issues with how the company is managed that will cause other problems down the road.


Now, there is a realistic way you can do a coding-based interview. You give the interviewee a (properly isolated) real development system as they would be using, give them their choice of editor, let them use the internet and the documentation, give them an actual problem to solve instead of something that's better designed to test their understanding of their native language than their coding skills, and then (and this is the important part) watch them while the work through it.

The process is more important than the result in cases like this. I don't care if you can regurgitate a dozen variants of fizzbuzz in any of half a dozen languages. I care that you understand how to effectively utilize documentation and online resources to figure out the correct way to solve a problem.

Collapse
 
codemouse92 profile image
Jason C. McDonald

Separate from my other response in this thread, I want to add something...

Yeah, coding challenges as part of the interview process are a huge red flag IMO.

I think code golfing challenges specifically are the red flag. I find it helpful if a company has some coding "challenge" task, either before or during the interview, which is (a) unique to them, and (b) reflects the work you'll actually be doing.

As an applicant, this helped me twice to realize I didn't have the experience for the job I was applying for! The coding challenges in both cases used the technology the job required, in a manner similar to how I'd be using it on the job. I was able to save myself and the hiring manager a few hours of effort, through those challenges alone.

A good coding challenge can be a real asset (as I comment separately). A bad one, including ALL code golfing challenges, is useless at best.

Collapse
 
ahferroin7 profile image
Austin S. Hemmelgarn

Agreed. I was definitely being a bit over-generic in my statement there. If done right, a coding challenge can be a good part of the screening process. The problem is that most places don't do them right.

Collapse
 
sroehrl profile image
neoan

Interesting. What do you think about having the candidate look into the used codebase and explain the process of understanding what is going on?

I assume it depends on various factors if that is suitable, but I had the chance of trying that out and found it had an excellent side effect: you basically get a code review from an external view. This can be a win as the existing team tends to create a "bubble" and inconsistencies, missing docblocks etc become more apparent. At the same time, you gain a pretty solid understanding of the candidates ability to "pick up work"

Thread Thread
 
ahferroin7 profile image
Austin S. Hemmelgarn

If it's a case where you can do that without needing multiple layers of NDA's just for the interview, then yes, I think that's a good option as well. It's kind of dependent though on how close to BCP for whatever language/platform you're using you are, as being too far just makes it too difficult for the candidate to pick up anything.

Thread Thread
 
sroehrl profile image
neoan • Edited

Sure, and there's probably an even longer list of cases where it's also not appropriate neither of us are thinking about. That is what I implied when I said it probably isn't a possibility in many cases.

Thread Thread
 
codemouse92 profile image
Jason C. McDonald • Edited

It certainly makes sense for a team around an open source project to relate screening challenges to the existing code base! That can be an amazing technique...as y'all said, if done right.

There is a fundamental difference between reading/explaining code and writing code, though, and that should be accounted for. I've met plenty of people who could do one, but not the other.

And then you have the companies whose code base is already a horrific pile of spaghetti, and showing any of their code in an interview would reasonably send candidates screaming into the night. ;) Or do you ask then: how do we make this function better?

Thread Thread
 
sroehrl profile image
neoan • Edited

And then you have the companies whose code base is already a horrific pile of spaghetti,...

Lol. Yes, but my point is: then that company needs such feedback.

There is a fundamental difference between reading/explaining code and writing code, though, and that should be accounted for.

True. So I guess you want to see both capabilities.

Collapse
 
sroehrl profile image
neoan • Edited

watch them while the work through it

Could not agree more! This is the best way to see how they go about their work. The only problem is that this sometimes causes stress they are not used to and therefore do not show an accurate representation of their actual behavior.

fizzbuzz

And yes, what's up with that? I see that a lot recently. What happened to good 'ol foobar?

Collapse
 
ahferroin7 profile image
Austin S. Hemmelgarn

fizzbuzz

And yes, what's up with that? I see that a lot recently. What happened to good 'ol fooba

Fizzbuzz is a general class of coding challenges that's pretty popular because it's a) easy to understand, b) trivial to verify without even needing to run the code, and c) actually shows some basic understanding of the language in question (namely control flow and IO) while still managing to be completely useless otherwise.

The general form of the problem statement is:

Write a program that iterates over a sequence of numbers (possibly a fixed sequence, but usually taken as input) and:

  • If the number is a multiple of X print 'fizz'
  • If the number is a multiple of Y print 'buzz'
  • If the number is a multiple of both X and Y print 'fizzbuzz'
Thread Thread
 
codemouse92 profile image
Jason C. McDonald • Edited

I actually remember when that trend started. It precipitated off an article that made the rounds on LinkedIn many years ago, about hirees who could not actually code, but could fake their way through the interview.

The author made the point that you should at minimum verify the applicant's claimed skills with something that would be trivial to write during any interview, such as a fizz-buzz.

Unfortunately, as with Ben Franklin and Daylight Savings Time, the idea was taken too literally! Hiring managers took the suggestion and ran with it, implementing literal fizz-buzz challenges in interviews, contrary to the author's actual point.

So, disaster #1: Non-coders learned how to fizz-buzz, so as to fool interviewers.

Many other hiring managers did get the point, and came up with unique, simple challenges that could fit within the interview. Unfortunately, this part of the disaster was on the would-be applicants. Modern code golfing was born, obstensibly to help programmers prep for interviews, but...

**Disaster #2: Code golfing sites inadvertently nullified the point of the coding challenges.

Unaware that their weeding technique was now pointless, companies started formalizing code golfing challenges and offloading them to the front of the process, to save time during interviews. The same sites that allowed anyone to practice these challenges saw an opportunity to cash in at the expense of companies.

Disaster #3: The process devolved from something useful to an exercise in futility that screened nothing, and ultimately helped no one (except the fakers).

The trend was now firmly established, and no one seemed to remember the original point. Fizz-buzzing and code golfing became a normal part of the fake programmer's acting repertoire.

Collapse
 
tomekbuszewski profile image
Tomek Buszewski

I have a different problem during interviews. I panic. Like, a lot. Like, forgetting basics. Once I forgot how to iterate through an array. Or what does pop do. Problem is, I am stressed when evaluated. I don't really know why, but I do. When I am hired, I can easily do pair programming or code sitting next to a manager, no problem at all. But during the interview, damn, I can't even spell my name right.

Collapse
 
sroehrl profile image
neoan • Edited

Yes, that is more common that you would think. I think addressing that in the interview is most of the time the best move here. Sure, it will not always work, but most of the time you'll change the setting and trigger empathy. More importantly, if you notice that the person or the people interviewing you react encouraging, you often overcome the nervousness.

Collapse
 
tomekbuszewski profile image
Tomek Buszewski

Yeah, recently it was like that:

> i am kinda nervous on the coding part, sorry for that
>> you need to be performant under stress

But, some interviewers are being actually helpful, true.

When I interview people, I never ask them to write any code. I ask for code samples before the meeting, so I already know how they work in general.

Collapse
 
codemouse92 profile image
Jason C. McDonald • Edited

First off, yes, code golfing is not a good indicator of coding skill. I do coding challenges as part of hiring at MousePaw Media, and it's a critical piece of the process, but the challenge is actually a real world problem with multiple viable solutions.

Done right, it's a useful tool. Done wrong, it's nothing but a waste of time.

So, the bulk of your article is right on the money.


BUT, I need to address one thing you said, though, that fundamentally spoils the whole point you were trying to make...

Isn't there an underlying impertinence in saying: before I risk investing one single hour for a face-2-face interview, I expect you to risk investing about 15% of a work week?

No, there's an underlying impertinence in expecting a hiring manager, who has to screen through hundreds of candidates, to make an hour for you merely on your say so. We don't know you. We have absolutely no reason to believe you're worth interviewing. To pitch a fit about having to prove your qualifications is, to put it honestly and objectively, extraordinarily myopic.

I've interviewed for several jobs with over 300 candidates each (after resume weeding). Is it reasonable for the hiring manager to interview all 300 candidates, at 1-2 hours each? Don't be absurd. You're applying for maybe two dozen jobs a week tops, so you're putting in maybe 12-24 hours of work if each one took two hours of effort on your part. But, wanting to spare yourself that inconvenience, you're literally asking each those hiring managers to put in hundreds of hours of extra work to spare you a couple hours each.

To be quite blunt, regardless of who you are: you are not that important. That's not personal; it's true of virtually all of us. Unless you're literally Guido van Rossum or Bjarne Stroustrup, you're going to have to bite the bullet.

And never mind the fact that it could potentially be the candidate who decides not to go forward after finally seeing the company from the inside.

The interview process is two-way. Hiring managers expect that as a legit possibility with every single candidate. Our answering your questions and taking that risk is how we repay you for your effort up to that point. (If the company doesn't, they're not worth it.) If you walk or we walk, we're both out the time.

I mean, somebody must have understood the psychological principle of reciprocity.

And, to be honest, I don't think you do. You're wanting the person with the already large workload to spare you a little extra effort. You're approaching the hiring process with only one ruling mindset: "what's in it for me?" I would never want to hire someone with that attitude, especially when I've worked so hard to hire for and build a team of mutual trust and support.

If someone is not willing to prove they're a viable candidate, they're not a viable candidate...and that attitude spills over into the workplace. If they'll be so myopic with a hiring manager, I'd bet cash it'll be a matter of weeks or months before they do the same thing to the DevOps, the QA Engineer, the UX designer, the technical writer...

So, while coding challenge policies themselves could stand to be seriously improved, the expectation for you to prove your candidacy through such an activity is entirely reasonable.

Collapse
 
sroehrl profile image
neoan

Let's start by saying that this post is about how candidates aren't vetted effectively and not about that they shouldn't be vetted.

It's not that I don't see where you are coming from (although I think you misjudge my attitude massively), but let's put some things into perspective:

First, you seem to assume that I describe that problem only from the perspective of the candidate. Given the context, I can see how you can come to that conclusion, but in order to address some assumptions you made about me, I have to ask: What makes you think I have no experience in how the hiring process looks like from the perspective of the employer? What makes you think I never vetted candidates, worked hand in hand with HR departments and invested time to find the right team member?

I don't mean that confrontational, I just want to make you aware that a lot of what you write is based on things I do acknowledge on one side, but simply aren't as one-sided as you make them sound:

Is it reasonable for the hiring manager to interview all 300 candidates, at 1-2 hours each?

No, of course it isn't. I don't think you even CAN read my lines as "Instead, just talk to anybody sending you a resume", as you imply here.

You're wanting the person with the already large workload to spare you a little extra effort.

So here is where I start to feel a little attacked. Let's look at what you are saying here: I am the arrogant one for "demanding" to spare me some time while it seems to be a given for you that the candidate's workload cannot be compared to the hiring manager? I find that interesting. Not to say distopian. Again, I am not saying that candidates shouldn't be willing to put effort into the process, but at the end of the day it is a give-and-take. If YOUR attitude starts with your time being more valuable per definition, then I can promise you that you will end up "hiring down" instead of with the best possible candidate for the position you seek to fill. It is statistically likely that the ideal candidate sits at another company for 60-80 hours, potentially with similar responsibilities than you. If that isn't even conceivable for you, than you will never talk to that (here hypothetical) person.

And that all wouldn't have bothered me as we don't know each other and I also sometimes make wrong assumptions regarding how something is meant, but then you go on cementing that one-sidedness by going all in:

I would never want to hire someone with that attitude, especially when I've worked so hard to hire for and build a team of mutual trust and support.

Wow! So you would never hire someone with which attitude, precisely? The attitude you project on people who dare to have feedback to your hiring mechanism? You seriously use the words "mutual trust" when you mean:

"If you don't show me that you want it by complying without asking questions, and without knowing much about the working conditions, team or company, then it lacks evidence of an obedient laborer who - god forbid - might wonder what's in it for her/himself"?

Because the candidate has to take your word for it when you say "...especially when I've worked so hard...", but you would never trust the candidate on her/his word?

Yes, I see how that attitude is something you would want to prevent to

spill[s] over into the workplace

That would surely be devastating in a team of mutual trust.


But fight mode aside: Of course the candidates have to "prove they are worth your time". I certainly don't hold the position that it isn't adequate to have a reasonable investment and even agree on the general notion that the "work" of forming a new role must be divided on things like reasonable effort. As such, of course the hiring manager shouldn't spend days interviewing people that could have deemed the wrong match based on effective filtering methods (so back to the topic: let's find effective methods, not a "class war")

Collapse
 
codemouse92 profile image
Jason C. McDonald • Edited

If YOUR attitude starts with your time being more valuable per definition, then I can promise you that you will end up "hiring down" instead of with the best possible candidate for the position you seek to fill.

Not "more valuable", but "stretched thinner." There's a profound difference. If a hiring manager is putting in hundreds of hours, it's not unreasonable for them to expect each of their candidates to put in a tiny fraction of that time. It's no different from saying "I've covered the entire meal, and used all my available funds to do so, would you mind taking care of the tip?"

No, of course it isn't. I don't think you even CAN read my lines as "Instead, just talk to anybody sending you a resume", as you imply here.

See your own post:

Is it really okay to ask a job-seeker to invest half an hour for a phone interview, 2.5 hours for a coding challenge, 1 hour for psychological online tests...Isn't there an underlying impertinence in saying: before I risk investing one single hour for a face-2-face interview, I expect you to risk investing about 15% of a work week?

That literally states that you (apparently) believe it unreasonable for a candidate to have to go through a phone interview and a coding challenge before being granted a final interview. There's no other way to read it.

What's further:

Wow! So you would never hire someone with which attitude, precisely? The attitude you project on people who dare to have feedback to your hiring mechanism?

No, I would never hire someone with the attitude I explicitly described:

...only one ruling mindset: "what's in it for me?"

A successful team needs to be mutually supportive. Each person supports and is supported. They recognize their own needs, but also the needs of others, and keep both in perspective.

For someone to have the attitude of "WELL! How unreasonable of that hiring manager, for wanting me to do this coding challenge before I get a final interview!" they must first fail to recognize the position the hiring manager is in. That indicates the person has clear difficulties empathizing with another individual.

Because the candidate has to take your word for it when you say "...especially when I've worked so hard...", but you would never trust the candidate on her/his word?

First, of course I wouldn't expect that! Ergo...

The interview process is two-way. Hiring managers expect [a candidate walking] as a legit possibility with every single candidate. Our answering your questions and taking that risk is how we repay you for your effort up to that point. (If the company doesn't, they're not worth it.)

In other words, you're interviewing us as much as we're interviewing you. It's the principle of reciprocity.

And, in any case, it's quite common that there's a "probation" period after hiring. The candidate may still turn around and walk out (with pay). So I expect to prove myself trustworthy, both in the interview and afterwards.

Interestingly, though, if it turns out the candidate is untrustworthy, guess who's out both the pay and the time of multiple employees involved in onboarding? So, as great as the risk is for the candidate, the risk is greater for the employer. Don't underestimate that.

Second, I would be taking the candidate's word for a lot already. One has to maintain a wise balance. If you say you improved the security of the email servers at your school, chances are, I won't be able to verify that. I'll just assume you're telling the truth...so long as what I can verify matches what you're saying. So, I verify the little I can, and believe the rest based on word alone.

If I hire someone without verifying their technical ability, I am disrespecting my entire team by saddling them with a peer who they will have to entirely carry. I owe it to my existing staff to verify essentials. Period.

Furthermore...

What makes you think I have no experience in how the hiring process looks like from the perspective of the employer? What makes you think I never vetted candidates, worked hand in hand with HR departments and invested time to find the right team member?

As a simple analogy, if you were to tell me that it's unreasonable to ask food service workers to wash their hands before returning to work, it'd be a safe assumption you'd never worked in any public health field. Your statements, as originally written, implied a lack of understand of what a hiring manager's workload is like. Again, your original post...

Is it really okay to ask a job-seeker to invest half an hour for a phone interview, 2.5 hours for a coding challenge, 1 hour for psychological online tests...Isn't there an underlying impertinence in saying: before I risk investing one single hour for a face-2-face interview, I expect you to risk investing about 15% of a work week?

I'm willing to believe you didn't put that in the way you intended.

I certainly don't hold the position that it isn't adequate to have a reasonable investment and even agree on the general notion that the "work" of forming a new role must be divided on things like reasonable effort.

I'm glad you don't. Just understand that your opening "digression" summarily contradicted that, and that is what I took on. I cannot apologize for misunderstanding, because I only took your words at face value.

Thread Thread
 
sroehrl profile image
neoan

Not "more valuable", but "stretched thinner." There's a profound difference.

Because the hiring manager needs to allocate enough time to state his/her case on dev.to? But joke aside: Again, I think you base that on particular assumptions that translate to either what kind of candidate(s) you personally seek(ed) for or how you think that candidate spends the days. Because let's face it: if the candidate is at home looking for a job than it is more than logical that this person would do anything (including spending all of the available time) to seek employment. And if such a candidate is not willing to invest a large enough time then scepticism is appropriate. But if the candidate reaches out to seek better opportunities, things can look very differently. Do you know how often it happens that highly skilled people are contacted by head-hunters and if they show interest are presented with these procedures? So I guess I simply account for a different hypothetical candidate.

If a hiring manager is putting in hundreds of hours, it's not unreasonable for them to expect each of their candidates to put in a tiny fraction of that time.

So I let that slip the last time as I thought it's more of a generalisation, but let's address it: Why does this hiring manager spend hundreds of hours? Your example was based on 300 applications and I assume that the scenario has an HR department responsible for actual job postings on various platforms etc. So we basically have:

  • role definition
  • proof of requirement / reasonability
  • budgeting

as topics our hiring manager is potentially involved in prior to screening resumes? And then your mentioned 300 resumes on the table. So, if you don't mind me asking, what do I forget that would take hundreds of hours I am not accounting for? So far, it sounds like 40-60h. Not that that wouldn't be a time investment, but I'd look at this from a different angle: If the total amount of hours a company spends on recruiting really is in the hundreds (across multiple employees, of course), it's even more reason to look into the process.

About how I formulated what I find a reasonable demand:

See your own post: ...

Forgive me if I will not join a discussion of how I meant something after being presented with a text you pasted together from my post. The article as a whole is written with sarcasm and humour and taking things out of context is a rhetorical strategy one can use in debates to convince an audience, if you must, but not convincing for the actual author. I do take your word for it that the way you portray it is the way you perceived it. And there is no apology needed or expected for that. But there is really nothing beyond "That's not what I meant" I can add to that either.

A successful team needs to be mutually supportive.

Agreed. Fully! But pretending that the candidate has any kind of loyalty towards a team he/she doesn't know yet would be unrealistic (maybe even delusional). The candidate that doesn't attend an interview with the primary objective to find out "what's in it for him/her" probably doesn't even exists - and if such an exception comes along, I'd be highly interested in the primary motives. I just assume that we can openly talk about this here rather than playing this societal game where we have to pretend that there is any kind of personal attachment towards a (for the candidate at that point) soulless corporation. The human factor first has to grow.

As for most of what you say regarding the risk of hiring a person in the first place as well as the interview process itself I have nothing to disagree with. Again, it's not that I don't know this process (hint: maybe read the very first sentence of the post again). I really don't know how to formulate it any differently: I do not propose not vetting the candidate carefully and appropriately. I simply question how that t(r)ends to be done.

And yes, for the purpose of this post I do explore the topic from the perspective of the candidate. But there might be a future "Why you won't find your unicorn" version from a perspective you will more likely identify with.

Again, this is not about a class war employer vs. employee. I think in the end there is probably more that we agree upon than not (referring to the ideals of a mutually beneficial, invested and loyal team throughout the hierarchy).

Thread Thread
 
codemouse92 profile image
Jason C. McDonald • Edited

Why does this hiring manager spend hundreds of hours? ... So, if you don't mind me asking, what do I forget that would take hundreds of hours I am not accounting for? So far, it sounds like 40-60h.

Yes, 40-60 hours is reasonable, and what I would expect. That's a full work-week, and then some, assuming they have absolutely nothing further to do. God help the hiring manager (like me) who does things OTHER than hiring and HR.

The 300 hours I mentioned was, if you casually skim through again, what it WOULD take if a hiring manager was to take your (reportedly sarcastic) advice at face value and interview all 300 candidates who looked good on paper. Which is absurd.

I can appreciate if that wasn't what you meant, but it is what you said. I didn't need to "paste together" something, as you so strangely accused me of. I copy-pasted your paragraph so I couldn't be accused of misquoting.

I appreciate satire and sarcasm, having written plenty of it over the years. But the truth is, an author is entirely responsible for how that sarcasm is taken. If a reader indeed reads all of what you said, can quote it back to you, and misses your meaning, that responsibility falls entirely on you, the author, for being unclear.

I do not propose not vetting the candidate carefully and appropriately. I simply question how that t(r)ends to be done.

I'm glad you were not serious when you spent several paragraphs roasting hiring managers for expecting coding challenges before a final interview. Unfortunately, that message wasn't quite clear as originally put, as I think you already know given the abruptness of your defensive reaction. In the future, just go "Ah, I think I was unclear, what I meant was X" and move on.

It's not an attack on you. There are just too many entitled people who really believe that the whole hiring system is unfair because they're not being pandered to, and would take the opening of your article at face value as justification for their 'tudes. You and I both have a responsibility to be mindful of that in writing sarcasm, satire, and irony.

Thread Thread
 
sroehrl profile image
neoan

The 300 hours I mentioned was, if you casually skim through again, what it WOULD take if a hiring manager was to take your (reportedly sarcastic) advice at face value ...

Not only skimmed through it, but read the complete thread to make sure. You repeatedly make it sound like these hundreds of hours are a given, using it as an argument to make your point. I mean, apparently we are experts in misjudging each others words, but you made arguments like "If I pay for the meal you can at least pay the tip" and similar that all underlined that impression.

God help the hiring manager (like me) who does things OTHER than hiring and HR.

Yes, by now I have fully internalized that your position is that you are stretched thin and have a lot on your table (which I honestly don't doubt for a second, just to make that clear), while the candidate could be so kind to pause Netflix for a while to fill out some tests (and here is where I beg to differ). And again, if that's your hypothetical candidate, then that's probably a self-fulfilling prophecy.

I didn't meant to accuse you of misquoting, but of taking things out of context.

But the truth is, an author is entirely responsible for how that sarcasm is taken.

Ah, so we find ourselves in a value-difference. Uff, I don't think I want to get into that discussion. But no, I disagree and have a different opinion.

I'm glad you were not serious when you spent several paragraphs roasting hiring managers

So, about that: Yes, why did you feel so triggered in the first place? I didn't even use the word "hiring manager" once in my original post. My post is structured like this:

The concept

Here I say that vetting is necessary

The investment

Here I say that the effort seems to have a trend of growing.
(And all the two of us talk about is within two short paragraphs in this section)

The conditions

Here I talk about unrealistic conditions of hakerrank & co

The time

Here I open up the discussion to the necessity of time-based assessment vs. the negative effects it has on quality

The challenges

This part is in regards to the use(full/less)ness of common coding challenges.

The tragedy

Here I claim that training for such challenges distorts what those tests are intended to evaluate.

So, I basically inserted a question regarding how much we can ask a candidate to invest in a post that is about coding challenges and that set you off? Let's see:

Unfortunately, that message wasn't quite clear as originally put, as I think you already know given the abruptness of your defensive reaction. In the future, just go "Ah, I think I was unclear, what I meant was X" and move on.

So here we get to the point, don't we? This is not a class-war as I misinterpreted. This is about your ego. And silly me said "No apology needed" when you actually expected me to apologize.

And you know what, I probably would if I shared your believe that
"author is entirely responsible for how that sarcasm is taken". But I don't. I think that the majority here "got it right" and you came in from an angle that didn't allow you to understand where I'm coming from. But tell you what - pinky promise: Should we ever meet in real life, let's have a beer (I pay for the beer, you the tip ;-) ). I think that we might have more in common than this thread would indicate.

Thread Thread
 
codemouse92 profile image
Jason C. McDonald • Edited

Well, you can mischaracterize this as an ego issue if that makes you feel better. I was content at it being about your point stated mismatching your point intended, once that became evident, but I suppose that's not comfortable for everyone.

Everyone comes into an article with a unique perspective. The field of communication (my academic background) is all about learning to account for that. No one can read your mind, only you can do that, so if someone misses your point, it's your job to non-defensively clarify, and possibly to tweak your original post to remove the ambiguity if it seems appropriate to do so. Getting angry at someone for not being able to read what you meant instead of what you said is unreasonable.

As to...

Yes, why did you feel so triggered in the first place?

One can grow tired of hearing the same fallacies repeated over and over again, and can confront them, without being 'triggered'. As a career mentor, I have to deal with misconceptions among young developers about hiring on a regular basis. It's a topic I am passionate about, because I see the results so often, my own company notwithstanding.

But as I said, if you'd rather trade a discussion about ideas and how they're expressed for one about how terrible and sensitive a person you imagine I am, I cannot stop you.

I do hope someday you learn to take responsibility for your role in communication. I will grant that it isn't always an easy one to learn — I myself learned how over years of mistakes as a professional writer — but as such a lesson presently runs contrary to your beliefs, I see no point in continuing this conversation. You have a right to learn things the hard way, so I'll let you get back to it, without me interfering any further in that.

To that aim, if I miscommunicated what I meant about 300-600 hours being abnormal, I apologize for not being clearer.

In the meantime, we're at an impasse, so we'll just leave this here.

Thread Thread
 
sroehrl profile image
neoan

I am sorry, really expected you to have the last word and wanted to leave it at that as well, but I am baffled.

But as I said, if you'd rather trade a discussion about ideas and how they're expressed for one about how terrible and sensitive a person you imagine I am, I cannot stop you.

Really? My intend, even from the way you took it, is obviously to start a discussion about how we as a developer community can improve the hiring process. Even if I fully emerge into your perspective, one might still wonder why we are talking about hiring managers and attitudes below a post that isn't about that.

As for the personal level: I really don't know you and have nothing against you as a person. I match my tone to the level of arrogance I experience, but that does not mean that any personal grudges derive from it. I have no intend to "express how terrible and sensitive of a person I imagine you to be". You might think I am naive (actually pretty sure you do based on how you address me), but I am not disliking people based on having a different opinion and the willingness to express them strongly. After all, I am not blind towards the fact that there is some irony in this being about my ego as well.

And a quick note about your passion and mentorship: I haven't seen this topic addressed in your posts (but only skimmed your profile, so forgive me). I think it would be of interest to many what you can share about misconceptions regarding hiring and job-searching

That said, you may have the last word now. Promised. I will resist.

Thread Thread
 
codemouse92 profile image
Jason C. McDonald • Edited

Really? My intend, even from the way you took it, is obviously to start a discussion about how we as a developer community can improve the hiring process.

I really believe you did, right from the start! I was addressing a single section in which you were expressing a message that mismatched the rest of your point. I tried hard to keep on the idea, and I apologize if I made it about anything else.

I re-read my posts, and I'm fairly sure I avoided saying anything about your ego, mate. That said...

I have no intend to "express how terrible and sensitive of a person I imagine you to be"

Um, right here:

So here we get to the point, don't we? This is not a class-war as I misinterpreted. This is about your ego.

...and here...

So, about that: Yes, why did you feel so triggered in the first place?

That's when it left ideas and went into characterizing me as being sensitive and egotistical, rather than me just addressing a topic I'm passionate about based on my actual experience.

And perhaps I do owe you a citation of my credentials: I've been doing hiring successfully for a small, diverse team for nearly a decade. My processes have been vetted by university career services departments, and proven by the actual results of hiring. Like I'd said, I don't know your experience, but your statement seemed to suggest that you didn't understand what a hiring manager's workload tended to be like. Armchair experts on this topic are a dime a dozen, and you'd made no mention in your post of doing this before (unless I missed it? Correct me here!). One of the core principles of public speaking is to cite your credibility up front; that applies here.

But again, I believe that one section did not adequately express your opinion on the topic, and I'm well aware it mismatched the rest of your post.

I've been on the flip side of this before, by the way. I once got schooled in how Java actually compiled by one of the engineers responsible. I wanted to be able to dismiss him as "arrogant", but the fact was, he had knowledge I didn't. Between the two of us, we were able to figure out how to disambiguate some build vocabulary, which I later put into a post.

By the way,

I am not disliking people based on having a different opinion and the willingness to express them strongly. After all, I am not blind towards the fact that there is some irony in this being about my ego as well.

I can appreciate that you don't altogether dismiss it, especially as you haven't yet crossed the Code of Conduct line (even if that was a close shave one post ago, see above.) Things get heated. I wouldn't be surprised if I used less-than-ideal phrasing a few times myself, as much as I tried not to.

I think it would be of interest to many what you can share about misconceptions regarding hiring and job-searching.

I haven't written a lot about hiring in my DEV posts. Most of my work in this area is on a more individual basis as a mentor, although I've been involved in many conversations on DEV about good and bad hiring practice. I plan to someday write an article about it, but at this time, I've got bigger fish to fry.

My last point on the topic of responsibility of meaning: imagine a reader who is feeling put out because he was vetted for coding skills he claimed he had, and was found wanting because he couldn't code at all. He's oscillating between the idea that not being selected was on him, for not having the essential minimum skills before applying, and the idea that the hiring manager was just cruel and unreasonable. He happens across your post and reads that one section. How do you think he'd react? Would he thoughtfully consider how he can prepare for job applications, or would he just conclude that "any coding interview that expects me to verify my skills before the final interview is unreasonable!" Read your own post with a jaded eye, and you may see what I mean.

In any case, I digress.

I don't need to have the last word; in fact, your last post has only made me want to discuss different vetting methods with you! But I don't believe it's worth doing that under all that noise. I'll respond a second time, and we can focus on that instead. After all, it IS the bulk of your article's point.

Thread Thread
 
sroehrl profile image
neoan

I don't need to have the last word; in fact, your last post has only made me want to discuss different vetting methods with you!

Glad you're having fun as well.

I will use the new thread you started for that (probably a little later as I have things to attend to). Just quickly some remarks/questions on language to end this thread:

This is not a class-war as I misinterpreted. This is about your ego.

Yes, sorry. I probably should have said "our egos" right there. The actual epiphany here was to realize we manoeuvred ourselves in something I would formulate differently in a bar (hence the beer reference). It was regarding a male trait where two can realize that all the evolution and intelligence we gained doesn't spare us from sometimes finding ourselves in what is often referenced as a comparison of certain extremities.

So, about that: Yes, why did you feel so triggered in the first place?

I am aware of the connotation the word "triggered" recently gained in society. And I think we loose a physically accurate word describing "cause and effect" if we just use it in that "new, negative" way. "A caused B" is simply not the same as "A triggered B". So yes, I meant that something I wrote is directly responsible for you reacting in a certain way. But beyond that, there is no (neither negative nor positive) connotation I wanted to convey. I will always "trigger" webhooks, won't I?

Thread Thread
 
codemouse92 profile image
Jason C. McDonald • Edited

Connotation is weird, innit? It's the one thing that kills me every time. I'm extremely literal in verbal communication, a residual side-effect of a traumatic brain injury in 2008, so I will often completely miss the connotation of a word or phrase, since I'm selecting it purely for its denotative meaning.

For whatever reason, this doesn't happen quite the same way for me in written communication. But that struggle with connotation in verbal communication has given me empathy for differences in interpretation. I think "triggered" as you intended it certainly ties into this; I've been exposed to so many complainers unduly whining about the hiring process that hearing those same phrases in a different context is associated with the same ideas, ergo my (incorrect) assumption you were merely a disgruntled candidate who wound up on the wrong side of a coding challenge (paired with the lack of anything to indicate the opposite, ergo that whole "cite your credibility" thing.

I'm not sure our conversation was really all that "males in a bar" flavored, however. I've had similar conversations with people of all genders and backgrounds. The essential point is that we were, as you said, on opposite extreme positions. Unfortunately, in a public forum, backing down from a position you feel passionate about invites observers to conclude you have nothing valid to say; if we'd had this conversation in private, I'd have honestly just quit after your initial rebuttal. In the end, I think we both made our points, so even if we can't entirely agree, our cases are established well enough for the intelligent observer to draw a sensible conclusion for themselves.

But, yeah, "triggered" is an unfortunate victim of language evolution, and that means we cannot effectively use it in its original context in random conversation. So what do we use instead? Maybe "Yes, why did you feel so strongly to that point in the first place?" (I'm sure there are other alternatives, besides.)

At the risk of sounding overly P.C. (which I'm seldom accused of...if anything, diplomacy is often lost to me!), connotation is inescapable, and we have to factor it in. If you ever get the chance, I definitely recommend picking up any courses or books you can on intercultural communication! It's been incredibly helpful in my career, especially in writing and public speaking. I'm certain you would find a lot of enjoyment and practical insight from it; but then, every person breathing could benefit from communication studies.

Of course, on the note of knowing connotation and taking full responsibility for "misunderstandings" (I hate that term) of one's own writing, I wouldn't advise anything I wouldn't do. I will often go back and clarify my own articles, sometimes well after the fact, even when I have reason to believe my point was deliberately misinterpreted by an angry commentor. The fantastic upshot of that is that it reduces the mental load involved in reading my work. I sincerely believe that aspect of my writing is a major reason I got a book contract.

Thread Thread
 
sroehrl profile image
neoan

That might be a generational issue as well. I noticed that the use of smilies and emojis help putting the right tone into written communication. But I seem to be too old to get the hang of it yet too young to not being expected to "read" it.

As for triggered: I feel like the final word on that isn't out yet. There is always a "point of no return" when certain words or expressions tip over and cannot be used anymore without being understood in a certain way. I am not declaring the fight for the term "triggered" as over, yet. I started to be used in a psychological realm to prevent exactly that: to carry blame. "I am triggered by..." does not mean "It is your fault that I feel...". It simply declared the emotional effect something had on a particular individual. And it was a suitable use. Then one side added the connotation if blame and the other side made fun of the use in general. And all of a sudden that makes it derogatory in all contexts? Let's hope that's a fluke.

As for communication in general: I actually do invest some time in the topic and agree everyone should. But as you noticed, not only is language a living, constantly changing beast, we also live in a world where most of the people we communicate with use English, but didn't grow up with it (me included). This causes many local differences in forms of expression, between-the-lines interpretations and tone. It's something I always try to take into account, and therefore cannot go with you regarding your standard of "writing so everyone understands what you mean". At the end of the day, you have to aim for a percentage. If 60% get what you mean, there is room for improvement. But even if 99% of the readers put things together how you intended them to be understood, you will still "fail" to get your point across towards some.

And the feedback always feels distorted. If your book (congrats, BTW) gets sold 1 00 000 times and only 1% feel misrepresented/misunderstood, you will hear from that 1%. You won't get feedback from the ones nodding it off, and you probably won't get much feedback from the ones who loved it. But that 1%. These 1000 individuals will way more likely get vocal.

Thread Thread
 
codemouse92 profile image
Jason C. McDonald

As for communication in general: I actually do invest some time in the topic and agree everyone should. But as you noticed, not only is language a living, constantly changing beast, we also live in a world where most of the people we communicate with use English, but didn't grow up with it (me included).

Well, and that's just it. Communication emphasizes paralinguistics — aspects of communication beyond written/spoken language — so heavily, it really helps deobfuscate some of the weirder things about language, such as connotation. Intercultural communication, in particular, focuses heavily on differences in cultural and societal norms and expectations, where those come from, and how those influence communication.

It's something I always try to take into account, and therefore cannot go with you regarding your standard of "writing so everyone understands what you mean". At the end of the day, you have to aim for a percentage. If 60% get what you mean, there is room for improvement. But even if 99% of the readers put things together how you intended them to be understood, you will still "fail" to get your point across towards some.

Yes, you'll always lose someone, no doubt. The trouble comes in recognizing when we wrote something that's at a notable risk of being misunderstood. Wit, humor, and sarcasm are especially high risk in this regard, such that many authors will call out sections of satire outright. In other cases, you can "cushion the blow" to reduce the risk, especially if the context is missed.

(Thanks for the edit, by the way!)

If I were to take a shot at writing those two paragraphs we've been debating about, it would probably look something like the following. (You do NOT have to rewrite it; your edit is just peachy! This is just my demonstrating what I mean.)

After experiencing assessments ranging from 5 minute multiple choice quizzes to 2.5 hour coding interviews, it seems to me that the commitment expected from a candidate keeps growing and growing.
And we should think about that for a second:

How much value are you really getting out of piling on pre-interview expectations: a half an hour for a phone interview, 2.5 hours for a coding challenge, 1 hour for psychological online tests, ignoring the additional hour the candidate presumably put into writing a cover letter and learning about the position/company.

If most of that is doing little to effectively screen the candidate, isn't there an underlying impertinence in saying: before I risk investing one single hour for a face-2-face interview, I expect you to risk investing about 15% of a work week, most of which won't really tell me anything useful about you? And never mind the fact that it could potentially be the candidate who decides not to go forward after finally seeing the company from the inside. I mean, somebody must have understood the psychological principle of reciprocity.

See what I mean? Most of it is untouched, but a few words add clarity: your issue is the usefulness and return on those practices, not the practices in and of themselves. You're not dismissing phone interviews and coding challenges, but rather, you're challenging how they're done!

Anyhow, moving on...

And the feedback always feels distorted. ... But that 1%. These 1000 individuals will way more likely get vocal.

Definitely true. Usually you only get 2% of your audience giving feedback: the people who either love it so much they'd keep a copy under their pillow, or hate it and wish you'd go jump in a lake. ;) I'm exaggerating, obviously, but yes, there's a bias.

To that aim, however, one has to remember that for every person who WILL speak up about their dislike, there are probably a dozen or more who didn't say anything and just moved on. That's why I try to take feedback seriously (although not necessarily to heart): the one superfan speaks for a horde of moderate fans, the one harsh critic speaks for the horde of moderate critics.

There is a point where you have to ignore any particular critic, but only once you've seriously evaluated whether there's anything of merit in their criticism. Being a professional author really helps you develop a tough skin.

Collapse
 
jdforsythe profile image
Jeremy Forsythe • Edited

I put candidates through coding challenges on paper. Hate me if you want, but if you can't iterate an array with a for loop on paper, I don't want you on my team. This should be muscle memory.

The truth is that I will overlook simple errors that would be caught by an IDE. What I'm looking for is how quickly you start writing. The problems I pose are extremely simple (e.g. I start with "write a function to return the area of a circle" and get slightly more challenging from there). If you can't do that on paper, you should look for a different profession.

Collapse
 
sroehrl profile image
neoan

If we're talking pseudo code, I see paper as valuable. But it is annoying. The reason has a lot to do with the style of coding. When I code, I sometimes decide to move functionality into own functions, delete or add lines etc. That just doesn't work the same way on paper. It completely destroys "the flow". And having watched many developers coding, I learned that such behavior is quite individual and additionally varies depending on IDE behavior, language, followed code style. So no, overall not a paper fan.

Collapse
 
codemouse92 profile image
Jason C. McDonald

Yeah, I have used "on paper" a couple of times, but it really is incompatible with flow.

An analogous (and much more insightful) task is to have them flowchart a described algorithm on paper.

Thread Thread
 
jdforsythe profile image
Jeremy Forsythe

I probably should have been more clear. These are functions of 10 lines of code or less (some are 1). I don't care if you can write a bubble sort or Dykstra. It's completely useless. I just care if you can carefully read simple instructions and quickly write a loop or a couple lines of code to solve it. I don't even care what language it's in. In fact, I have prepared sheets with function definitions already written in the language they said they liked best on the phone interview to make them more comfortable with it.

Thread Thread
 
jdforsythe profile image
Jeremy Forsythe

Let's be honest - if you can't write a one liner on paper to solve the area of a circle given a radius, you have some work to do before you're ready for this line of work.

Thread Thread
 
sroehrl profile image
neoan

Well, I don't necessarily disagree that one should know that. But what for? The way you describe it sounds more like you boiled it down to a pure math question.

So not sure if I agree about that being an indicator of whether or not that makes you ready for that line of work.

Thread Thread
 
jdforsythe profile image
Jeremy Forsythe

That one is a softball question to get them loosened up. No it doesn't indicate they're ready for the job but can certainly indicate they're not

Thread Thread
 
codemouse92 profile image
Jason C. McDonald

Exactly. It's easy to forget that half the purpose of screening is to weed out people who aren't right. Such a question may not be able to quantifiably prove any one candidate is qualified, but we can determine several candidates aren't.

Collapse
 
jdforsythe profile image
Jeremy Forsythe

The functions I'm talking about are simple enough that moving code to another function is overkill. None of them require more than 10 lines of code. I'd be scared if they did that.

Collapse
 
codemouse92 profile image
Jason C. McDonald • Edited

So, carrying on the earlier thread in a hopefully less heated fashion...

Here's how I do hiring, with excellent results! (Many of my policies today were influenced by earlier mistakes in hiring; I now seldom mis-hire, based on outcomes.)

I hire entirely for a programming internship that requires some pre-existing coding skills. I expect that technical skill level will vary wildly, although there is a minimum.

  1. Application phase. Core expectations are made clear, and the applicant must explicitly acknowledge them in the application. Portfolio of code is requested, although there are situations where this may be impossible, and thus waived. I also look for references, and in some cases, unofficial transcripts.

  2. Initial interview (30 min - 1 hr). Learn more about the applicant, discuss the internship and how it will fit into their academic and career goals. This helps them decide if we're right for them.

  3. A coding challenge. They get a full week to complete this, in any language they want! The prompt is a simple, real-world sort of problem, not some sort of a code golfing fizz buzz. I like to give them time initially, so they can really put their best foot forward, without nerves and tight time constraints getting in the way. They can use their favorite tools and technologies, and submit when they're ready. (The actual challenge only takes about 30 minutes to 3 hours to complete well, so the week is a HUGE amount of time.

  4. I review the coding challenge in depth before the final interview, looking for specific things. I can't go into that here for obvious reasons, but my emphasis is on their creative problem solving and good use of their tools of choice. (This also helps me assess how to best tailor the internship to the applicant; it isn't just for screening.

  5. The final interview. I do have them do some live coding here, relating to that coding challenge, which allows me to verify that they indeed can code (and hadn't just outsourced the coding challenge to Taiwan, as some people actually do!). I always account for nerves here — and nerves are actually good, since they indicate the person is aware of their not being The Second Coming™!

Especially because this is for an internship, I also provide constructive feedback to almost all applicants, even if they aren't hired. This allows them to know what they can improve before the next job or internship they apply for elsewhere.

We've gotten praise from many interns after the fact about our hiring processes. The Career Services departments at two universities even use us as an example for other employers!

I'm running out the door right now, but I'm happy to expound on, explain, or justify any of the above. I'm curious what your opinion is in light of your article.

Collapse
 
sroehrl profile image
neoan

Finally have some time to get to this. I apologise for the delay. Let me start by saying that this approach seems very time intensive for both sides and therefore probably not realistic for many companies.

But I have a lot of questions:

Regarding 1.

When you say references in the context of internships, what do you mean? I assume interns usually have little to no previous experience?

Regarding 2.

Is this interview usually in person or over the phone? And you mention that the prospect can learn about the company, but what do you want to learn about the candidate in this interview?

About 3.

This sounds very interesting. Here are the things that come to mind: how do you ensure the result is based on the candidate's work alone? I tutor on wyzant on the side and the amount of students that basically want to pay someone to do their homework is relatively high. And then there is the majority of students who are eager to learn while we work through the assignments, but probably would have some trouble reproducing the complete assignment on their own. So I am torn. On one side, the ability to "google for" help is an important skill we usually don't test for, but effectively will show the long term abilities required to keep up with emerging technologies. On the other side, I would want to know if a candidate can gather information or work through a process completely on their own. While you do mention that in 5., I assume that you will find yourself having invested quite some time for a "poser" sometimes.

About 4.

I respect that dedication. And maybe on a side note for readers: your GitHub account is always more important than the portfolio you polished to hell. Nothing gives me a better understanding of your abilities and practices then your code.
I usually do this in the resume screening phase, but frankly I have never hired interns so I am not sure what I would get out of it in this scenario. Additionally, I work in a field where proprietary work is common. So more often enough people (including myself) cannot share the work they are most proud of.

So I guess this approach is overall fair, but frankly sounds like the opposite issue to what I am describing in the post (and explains your reaction even better): You invest a lot of time with a relatively high risk. In this case it's just you who spends an unreasonable amount of time without knowing "what's in it for you"

I might lack some important details, but overall, I feel like this approach is not scalable. It might work well for you and your candidates/target group, but I am sceptical of this approach being something that can

a) provide companies with an accurate assessment of a candidate's skills while

b) give the candidate a fair chance to show abilities and

c) design the process so both sides have a reasonable amount of time and risk investment

Collapse
 
codemouse92 profile image
Jason C. McDonald

You invest a lot of time with a relatively high risk. In this case it's just you who spends an unreasonable amount of time without knowing "what's in it for you"

Um, ha, that's assuming I go through all these steps with each person regardless. They don't necessarily make it all the way.

When you say references in the context of internships, what do you mean? I assume interns usually have little to no previous experience?

That's correct. I want to know about their work ethic and their character. Often, references are current/past professors or teachers, or supervisors at prior jobs (usually not tech jobs), and the like. Not having references is not a deal breaker, but it certainly helps.

Is this interview usually in person or over the phone? And you mention that the prospect can learn about the company, but what do you want to learn about the candidate in this interview?

We're 100% remote, so this is over video chat. I'm finding out about their career goals, what they want out of the internship, their work/study habits, and generally whether our program will be a good fit for them.

How do you ensure the result is based on the candidate's work alone?

Well aware of this problem. The obvious first step is to check for copy/pasted code from online, but that doesn't guarantee they didn't outsource it. We discuss it and have the candidate work on it during the final interview, and that always routs out the rest.

I will have to answer the rest later, but understand: I've been putting this into practice for years, with a high hiring accuracy.

Thread Thread
 
sroehrl profile image
neoan

... with a high hiring accuracy.

Sorry to switch gears here, but this little sentence opened up a world for me. Are you saying that you are applying long-term metrics to measure "hiring success"? If so, how does that look like? How do you evaluate e.g. that you made the right choice? And after which timeframe do you come to such conclusions?

Thread Thread
 
codemouse92 profile image
Jason C. McDonald • Edited

You must remember, this is an internship, a training program. Whether someone thrives in and successfully completes the program, or washes out (variety of reasons), is a significant indicator. I'm also the internship supervisor, so I'm the one responsible for mentoring, tailoring the program to individual needs, answering questions, and helping the intern succeed.

It is worth noting that, of our 12 program graduates, 9 are in mid-level coding jobs, 1 is in the army, and 1 in graduate school. Only 1 is not in the industry because of the geographic area he wants to stay in.

The percentage of how many hirees actually graudate from the program has gone up significantly as we've refined our hiring process (although our hiring rate really hasn't dropped). I've even checked my assumptions inversely: we've hired some candidates despite some red flags from the hiring process, and they've always wound up washing out due to serious problems.

You also need to know that we have a very diverse team from many backgrounds, cultures, and fields of interest. I encourage interns to be involved in project management and standards development, to share their opinions openly, and to challenge even my ideas! So, their success really is dependent on forming the core soft skills and work habits necessary to thrive in software as a whole.


Regarding the rest of your original response:

Your GitHub account is always more important than the portfolio you polished to hell. Nothing gives me a better understanding of your abilities and practices then your code.

I agree! I allow applicants to use their GitHub as their "portfolio" if they wish; and many do.

There are times when applicants have no portfolio, and I account for that as well.

You invest a lot of time with a relatively high risk. In this case it's just you who spends an unreasonable amount of time without knowing "what's in it for you"

Oh, I know plenty of "what's in it for" MousePaw Media. We have a number of projects going, and I also consider where an intern will best fit. There are times there's a mismatch between goals and interests that becomes quite apparent during the initial interview, wherein the applicant won't continue. We're not right for everybody - we do have actual projects that have to get done too.

We're not an easy internship program to get into. Two of the local universities advised on our program, and one of the Associate Professors (whose job history includes many years at Microsoft) called our program one of the most robust he's seen. He refers students to us directly.

I might lack some important details, but overall, I feel like this approach is not scalable.

So far, it's scaled well. 30+ interested parties, 14 pending applications (as of writing), and it's working well for a fairly reasonable effort from me.

It might work well for you and your candidates/target group, but I am sceptical of this approach being something that can

a) provide companies with an accurate assessment of a candidate's skills...

Again, note the outcomes.

b) give the candidate a fair chance to show abilities and

We've been able to spot, not just present ability, but raw and undeveloped talent beyond that. Remember, those same applicants are now in mid-level full-time software development positions at other firms, and very much enjoying their work from all reports.

c) design the process so both sides have a reasonable amount of time and risk investment

I deliberately take a bit more 'time risk' on, so as to provide constructive (and legally appropriate) feedback to applicants. Often, this is the first tech position these applicants have applied for. We're forgiving of many reasonable faux-pas, and help applicants understand how to improve before they interview elsewhere.

All of that combined is a major reason why the Career Services departments at two of the universities we work with use us as an example for other internship programs.

Thread Thread
 
sroehrl profile image
neoan

Just to clarify: by "scalable" I meant "applicable to a wider range of companies". While a lot of what you write sounds great, it is a very specific and unusual scenario. In most settings, interns are the exception, not the rule.

So while I believe you that this might be the ideal approach for your organization, I remain sceptical of how transferable such an approach is.

And I don't say that to criticize, I say that as I am searching for solutions that can be applied in the broader context.

Unfortunately, the classic career "You come as a junior and stay for the senior positions" seems to be outdated. Most devs make their move after about 3-5 years.
Certainly, there is some blaming on the employer side to do, but there are many other reasons as well. This poses two issues: it's hard to have employees stay with you. And it is costly to fill positions. Especially in the higher roles, it sometimes takes up to a year to have the full output, as complex onboarding and in-depth understanding of code-base and processes are necessary. This makes "wrong decisions" often not immediately apparent.

It sounds to me (and forgive me with not being familiar with MousePawMedia), that you kind of place yourself between the industry and universities in order to buffer this risk. So I assume both the academic institutions as well as the private sector have an interest in "funneling resources" through you in order to bridge the gap.
And if I am right with this assumption, it makes sense that your approach is vastly different.

Thread Thread
 
codemouse92 profile image
Jason C. McDonald • Edited

Unfortunately, the classic career "You come as a junior and stay for the senior positions" seems to be outdated.

Agreed, and my approach has nothing to do with it. They leave us by design and move into a mid-level position elsewhere.

This poses two issues: it's hard to have employees stay with you.

It's a year long program. Our goal is for them to move on to paid positions at other firms; it's actually how we're structured.

And it is costly to fill positions. Especially in the higher roles, it sometimes takes up to a year to have the full output, as complex onboarding and in-depth understanding of code-base and processes are necessary. This makes "wrong decisions" often not immediately apparent.

The program is a year, but a number of interns have stayed on longer to keep working on the projects. So, I actually have observed the full output and the in-depth understanding. (Don't get hung up on "intern" — this program goes significantly deeper than standard internships.)

I think you're confusing my motivation and evidence of success. Just because I am hiring interns does not mean my approach is only applicable interns. I account for the lack of experience by deliberately going a lot easier on candidates than I (or anyone) would be. There are other companies that successfully use similar practices to what I describe.

You do have to be careful of confirmation bias. I'm reading "I don't think coding challenges work, therefore the fact they work for you must be an unusual circumstance."

As to myself, I can only speak to my own experience, and I can share that of the hiring managers (etc) I know...although you know neither of us could confirm or deny it second-hand.

So I assume both the academic institutions as well as the private sector have an interest in "funneling resources" through you in order to bridge the gap.

The private sector hardly knows I exist. I run the program for the sake of the students, to help them gain the all-important soft skills necessary to thrive in the coding field. The Career Services know this, which is why they work with me.

And if I am right with this assumption, it makes sense that your approach is vastly different.

Except it isn't all that different...based on talking to hiring managers, former interns who have gone on to other jobs, career services faculty, and my own experiences being a candidate. I've observed four types of companies:

  1. No screening.

  2. Traditional whiteboarding.

  3. Leetcoding screening. (The topic)

  4. What we do.

Thread Thread
 
sroehrl profile image
neoan

I'm reading "I don't think coding challenges work, therefore the fact they work for you must be an unusual circumstance."

No, apparently I haven't put that in the right context: I didn't refer to coding challenges when mentioning scepticism. I also don't think coding challenges aren't working in general, it's the code golfing challenges I have issues with.

The private sector hardly knows I exist. I run the program for the sake of the students,...

Well, that might be true, but I am looking at what you are saying from a different perspective. What I am hearing is: "students that ran through our program end up being very employable and capable". So from the perspective of companies, this translates to: "students vetted and 'formed' by MousePawMedia are low-risk, capable employees".

So maybe it makes sense for you to form relationships with the industry as well. You more or less offer a service that circumvents the very problem I am reporting here. And that service has a value you might be able to capitalize on without selling your ideals and while still fulfilling your main goals.

Thread Thread
 
codemouse92 profile image
Jason C. McDonald • Edited

Ah, I understand now.

I still can't believe we're so unusual that our hiring practices couldn't possibly work elsewhere. I'm able to determine the skill level of applicants with the coding challenge, and that has proved accurate. It could be made more restrictive, or more tied to a particular language or domain, without taking additional time from either party.

In any case, 20 minutes of me (the hiring manager) reviewing the coding challenge gives me a fairly accurate snapshot of the applicant's present skill, and that always either saves me a final interview (2 hours saved) or advises the final interview, making it far more reliably insightful.

So from the perspective of companies, this translates to: "students vetted and 'formed' by MousePawMedia are low-risk, capable employees".

Ah, yes, that is indeed what I'm hoping our company represents to employers, especially in terms of the letters of recommendation we provide to program graduates. It's a win-win for our internship graduates and their future empoyers.

So maybe it makes sense for you to form relationships with the industry as well. You more or less offer a service that circumvents the very problem I am reporting here. And that service has a value you might be able to capitalize on without selling your ideals and while still fulfilling your main goals.

Ahh, I understand you now! Thank you for that. I would need like to form stronger connections between MousePaw Media and the industry.

That said, I'd still understand the need for vetting, such as how we do it, even for our own internship graduates applying elsewhere. Someone may be fluent in (say) C++ and algorithms, but that doesn't mean they're good at (say) API testing in Python. In my mind, half the beauty of a coding challenge such as I describe is "are you a good fit for this particular role?"

Collapse
 
teejip profile image
Christiaan Pretorius

Coding challenges seem to favor a specific personality type not real world coding skills. It also tends to give successful candidates a false sense of confidence in their coding skills. Too much emphasis is placed on an interaction that really only tests a certain type of stress reaction. These challenges test ones ability to solve a well defined problem with a relatively simple solution deterministically. Real world problems are not well defined and the solution usually lies in cooperation.

Collapse
 
codemouse92 profile image
Jason C. McDonald • Edited

This. I encountered a candidate whose entire experience was wrapped up in code golfing, which had me concered: can he even write real code? Golfing and real coding are entirely different practices. It's like expecting to complete in the PGA World Cup because you're under par at the local Family Putt Mini Golf, to abuse the analogy. ;)

Collapse
 
sroehrl profile image
neoan

Well put

Collapse
 
alexparra profile image
Alex Parra

Personally going through this process.
I too think most of the challenges I’ve been presented have little to do with real on-the-job work.
By far, the very best I’ve seen so far has been by Woven (via Qualified) where the two problems presented do in fact mimic real case you might face on-the-job, (investigate a bug with little info on a small code base that could be just about any app and communicate with your peers about the findings + finish developing a feature that is half way and again could be part of just about any app).
And don’t get me started when the position is frontend and the tests are completely unrelated.
As you I do understand the goal of testing problem solving skills but most of the scenarios seem so off real life that it’s unreal.
Maybe we’re wrong...

Collapse
 
sroehrl profile image
neoan

Yes, it's fair to mention the good examples and I would have had an example as well. Unfortunately, I cannot recall the name. But they had a 2 hour screen sharing session where you checked out actual applications and debugged them.

Collapse
 
jouo profile image
Jashua

Nice post :)

I personally dislike Leetcode, probably even hate it, I can understand its intended purpose but I would rather invest my time learning real programming skills

I feel that most people disagree with that, whenever I mention it in websites like Reddit I get downvoted a lot

Collapse
 
sroehrl profile image
neoan

Well, Reddit downvotes are often related to hitting a point.

Collapse
 
codemouse92 profile image
Jason C. McDonald

Or hitting a nerve. Completely benign statements can get dogpiled for that reason.

Collapse
 
smadani profile image
Sina Madani

It doesn't even prove you know the target language: all of these coding interview questions involve writing everything in one file, a couple of functions, and using only arrays and integers/strings. Not how well you know the language, its features, abstractions, data structures, standard library etc. Just whether you can write a for loop (and only the kind involving i= 0; i < length; i++ type, not anything fancy or even understanding how the for construct works).
On these coding platforms like algoexpert, they claim to support 7 languages (all with imperative constructs), for all problems. What does that tell you? That all of these problems involve very basic imperative constructs - they're not going to write an idiomatic solution within each language, because there's no need.

Collapse
 
nombrekeff profile image
Keff • Edited

Lol what a funny post, I don't know if it was intended but It kinda made me laugh a bit xD

And I totally agree, I don't get the point of these tests, and have always hated them (for interviews).

Collapse
 
sroehrl profile image
neoan

Was intended to be humourous, yes ;-)

Nonetheless I mean what I say

Collapse
 
cheetah100 profile image
Peter Harrison

There is I think a tendency for software developers to signal their superior intelligence. The job interview is a way to invent tests which exist simply to communicate how wonderfully intelligent they are. I won't be crude, but this is just a member measurement exercise for intellect.

Okay, so what are we aiming for when we give developers a test? Are we trying to find out if they know the syntax? That they have memorized the API to the point they don't need completion? Do they know how to do binary shifting? Generally no.

We are not trying to test the basic language skills as they should have already been excluded if they don't have these skills. We are not trying to perform some kind of intelligence test. Some companies give tests that are meant to test intelligence, but there is an underground market in training to pass these tests. So again, no.

Instead we need to focus on higher level understanding and coding practice in everyday coding. We want to see if a developer asked to modify an application can recognize and correct poor design or anti patterns. Such a test must respect the value of time for a developer, and so must not be too complex. Providing an existing project of small size with a explicit objective means the task can be completed quickly by a competent developer.

This is not a pass fail type of test, but rather a exercise to flesh out whether the developer picks up on common development mistakes and will refactor as they go.

I wrote a article covering the test I wrote. This is good not only for interviews but also as a training exercise.

The Test:
dev.to/cheetah100/java-developer-l...

The Solution:
dev.to/cheetah100/java-developer-l...

Collapse
 
jpaulin profile image
Jukka Paulin

I am in midst of revolutionizing this. Going to make virtual chalkboard, with a proper screeching sounds when drawing. Anyone in with this? Just kidding. I absolutely love the original article. Testing culture has a few things that could be done much better, including transferring ability metrics more transparently without them being explicitly carved out by somebody.