Evaluation of developers is an important topic for anyone involved in software development. Many developers care a great deal about career growth or raises at the very least. Managers need to be able to justify decisions around promotions and raises.
Ideally, developers would be rewarded based on the amount of value they provide to a business. It’s a simple concept: you make the company more money, the company pays you more money.
The complexity lies in the fact that the nature of software development does not make it easy to measure the value a single developer brings. If you have a product that has hundreds of features and makes $10 million a year, how do you know how much of that value was attributed to a single feature? Even if you could do that, the feature was most likely worked on by multiple developers. How do you attribute the contribution of each?
There isn’t a good answer to that question. It just isn’t practical to attempt to make those kinds of measurements. What we are left with are subjective judgement calls by managers.
The risk with this approach is that the criteria are vague from a developer’s point of view. They can’t really know what their manager is thinking. There is now an incentive to focus on learning to sell themselves to their manager instead of focusing on becoming a better developer.
An attractive solution to this problem are quantitative metrics. If we can’t measure the exact value a developer brings to the company, maybe we can find something we can measure easily that will be a good indicator of value created?
The most naive measure here is counting the lines of code a developer writes. This may make sense to those who aren’t developers, at least at first. Developers write code… so maybe more code means more value?
The problem here is that high quality code often means fewer lines of code. A really simple example is: Say you wanted to display 100 products on your e-commerce website. You could do something like this:
displayProduct(1)
displayProduct(2)
displayProduct(3)
….97 lines later…
displayProduct(100)100 products, 100 lines of code!
No decent developer would do this.
They would instead make it 2 lines of code:
for productNumber = 1, productNumber <= 100, productNumber++:
displayProduct(productNumber)If you paid your developers per line of code, you would reward the inefficient developers.
An analogy to this is writing essays, novels, blog posts, etc. Would you judge a writer solely on the number of words written? Probably not. There are a minimum number of words needed to get a complex point across, but those points get lost when a writer clutters their work with useless sentences. I know I’ve read books that would have been great 3 page essays, but ended up being really tedious books.
So the lines of code metric doesn’t work. The notion of a quantifiable metric for evaluating developers is still attractive though.
Some may argue that creating many code branches is the mark of a great developer. Yet I once worked with a developer who would create code branches to hide the fact that he wasn’t very productive.
Some may argue that you can measure the number of code commits a developer makes. But that just makes it more likely to get lots of really tiny commits that have little to no meaningful changes in them. Or a developer can easily game that by undoing and redoing the same few sets of changes to bring up commit counts.
Maybe number of bugs fixed is the answer? Software bugs can be complicated though. By tying raises and promotions to bug fixes, you create the incentive for a developer to fix apparent issues while ignoring underlying causes. It would be like using duct tape to fix a broken leg. Worst case, you would reward developers who don’t take extra care to prevent bugs in the first place.
The real problem with quantifiable metrics like these is that they change the incentive for a developer. While creating business value is the goal, it can be hard for a developer to make it apparent that they are creating business value. It would be much MUCH easier to just optimize for other quantifiable metrics regardless of whether those metrics bring value to the business or not. This is far from desirable for anyone.
We are still left with the challenging question: how do you know if a developer is doing a good job?
In my opinion, the best measures are all subjective:
- Are they good team members?
- Can they solve problems?
- Can they write good code?
- Are they eager to learn?
- Can you trust them?
There are others, but the point is that every measure of a good developer is qualitative. That requires managers to communicate frequently with each developer that reports to them. Managers must create a culture where developers can be confident that they will rewarded for doing the right thing, not looking like they are doing the right thing.
Quantitative measures are attractive because they are easy. But nothing about management is easy. If it was, anyone could do it.
Do you have an evaluation method that you think is unique? I would love to hear from you!
This post was originally published on blog.professorbeekums.com
Top comments (11)
As we cannot evaluate the business impact of a developer directly, I like to assess the reliability of them.
Evaluation of story point and commitment to delivery is an important aspect to consider a developer as great.
We should be able to count on a great developer word. Delays due to development not able to deliver can have an enormous impact on the road maps, so on the business.
I am not telling developer should kill himself to give. I am more into someone I can trust to tell me if there are delays and then we can act.
I also like the approach from Mike Seavers, Director of Engineering at Riot Games, describe in the articles Debugging Title: Part I, II, III
What do you think about these approaches?
I agree that a developer should be extremely communicative of any kind of delay. I also think it's pretty bad if a 2 day task blows up into 2 weeks with no reason on why, though there's also a management failure there for allowing it to get to that point without communication.
I do think whether delivery commitments are actually the most important thing is something a team should discuss. Trying to make a deadline for a conference or investor meeting? Yeah, the deadline matters a lot.
Taking some downtime to clean up technical debt or laying down some foundational work of a new system? Quality is the important part here. It's important not to make the incentive time based rather than quality based (so long as the time is reasonable). Creating arbitrary time lines when they aren't necessary will encourage developers to cut corners elsewhere. If a dev has an epiphany at the last minute which will save us enormous amounts of time going forward, I would probably be up for extending the scope of that work. I would not want to have created an incentive for the dev to not speak up about it because they want a "completed" task as a notch on their belt.
The article about debugging titles seems really interesting. I think it's good that someone is trying to be more analytical about this. I've only skimmed it, but I do think there is still a lot of contention with the attributes though. People leadership can include both 1-1 mentoring and leading a whole team. Someone can be good at one without being good at the other. How does the attribute become affected? I like the effort, but I think it is extremely difficult to remove subjectivity from titles.
Yeah I really love what the riot game says above all how he conducted it as an experiment engineering.riotgames.com/news/deb... find some problems after feedbacks and then tries to fix them engineering.riotgames.com/news/deb...
That follows Deming's Philosophy of PDCA (by the way I start a talk "Agile Conversations" on this here youtube.com/watch?v=Cgs0F70c-I4)
About performance appraisal Deming makes this warning youtube.com/watch?v=2F1rhik2_X0
So I like what you said also:
"Quality is the important part here. It's important not to make the incentive time based rather than quality based (so long as the time is reasonable). "
First, give them complete responsibility for a section of code. If they can't make it work after several sprints adding significant new features, they can't code and cannot work effectively work with the team to complete tasks. Developers that cannot code will start something then will give reasons why they cannot work on it then another developer has to complete it for them. Many times they mention best practices and other management speak to cover the fact that they can't code.
Second, after they finish a feature, review their code, structures, and variable names. Bad names, structure, and messy code is a red flag. If they copy code from stack overflow but don't rename variables to match the repo they are coding in, another red flag. Proper indention is also important.
Third, Developers that are good typically type fast once they have worked in a block of code for a while. They can explain what they setup and why. They can diagram what was built.
Fourth, passion isn't a good indicator, but persistence is. The best developers never give up. Have developers explain their most difficult technical issues they faced during the interview. Determine when and why they give up.
Fifth, It is okay if a developer doesn't know something, but if they make the same mistake over and over, be careful. Developers must be able to learn and learn fast. If they don't, they will fall behind and quit/get let go. Most of what we know will be obsolete in 3 years.
Sixth, they can debug and know the debug tools inside and out. They want to know why something works the way it does. Working together with them debugging a few issues is one way to measure their debug skills but it doesn't always work depending on personality.
Seventh, they work well with the team and help wherever and whenever needed. They can joke about the development process. Ask them what was something fun that happened on their last project.
Great post, for me the key is "Are they eager to learn?" In my team I don't really care if people aren't great developers on day one, it's about month 3, 6, 12, etc. Are they learning, are they improving? Ultimately none of us are perfect developers who know everything, we're all continuing to learn.
I adhere to the Spolsky: "Good developers are Smart and Get Things done."
I further adhere to the Yegge: "Great developers are Done, Get things Smart"
Basically can you say yes to this question with respect to the person?
If so, do your best to keep them. If not, work to develop where they're weak.
That encompasses:
The first thing I look for in people is whether they are passionate about it. Of course there is no simple yes/no answer, it always varies from person to person. Second I would always give them time to evolve and do not judge them pretty quickly. Each person takes his/her own time. A good way is to ask them showcase what they have done in a short talk/presentation. In this way you can both judge them in many angles without making it awkward for them.
I believe peer discussions or even manager/developer discussions should happen at least once a month. In that way , mistakes are corrected quickly and expectations are set at the beginning itself.
IMO, ultimately it is subjective. A good programmer might be a poor commutator, or vice versa, etc.
Confidential Peer Evaluations can help managers get a sense of what their team thinks of each other.
We do reviews every +/- 4 months. Means the compensation increases are not a impressive each time but give quick feedback to engineer and allows for course correction sooner.
How often do you do performance evaluations? Where I work we do performance appraisals once a year, and it really feels like that isn't often enough. I'm curious what others do.
I think official ones are once a year for comp changes. Good managers should be doing a weekly one on one which includes more frequent feedback. This is a good opportunity to take a step back from the day to day and reflect. It's like a retrospective, but for 2 people. I also think real time feedback is also the best feedback. Anytime someone does something exceptional or something that really needs to be improved, let them know immediately.