Developer Compensation: Stack Overflow Doesn't Stack Rank

Article hero image

Are Developers Good Negotiators?

Developers come from all walks of life, and have many unique interests, passions, and hobbies. Often the only thing that developers have in common is their love for programming. It follows that some are good negotiators; others get the double digit percentage finance rate at the dealership when they go in to buy that new car.

How Does Your Company Determine Compensation?

When you hire developers, how do you decide on their salary? Do you allow for negotiations to take place? Is there a strategy in place where you offer a low value, expecting the candidate to counter with a higher number? Are you pleased when they don't counter, and you get good talent for cheap? The thing is, developers are the linchpin in your tech company. They make or break your products - quite literally, in fact. They're worth a lot of money. You should be paying them what they're worth as one of many strategies to keep them happy. If you're low-balling developers with a salary strategy that rewards negotiation skills, you're probably underpaying them while overpaying the developers who are good negotiators (but maybe not amazing coders). Your underpaid talent might not feel comfortable asking for a raise to the income that they are worth. Can you guess what happens then? They leave your company for another that values them correctly. The result is that your department has high turnover, lots of churn, and high costs around replacing the fleeting talent.

Stack Ranking: Upsetting Developers Everywhere

One of the most common ways to compensate employees is by employing a stack ranking system. There are varying approaches to stack ranking, but a typical implementation of a stack rank system is as follows:

  • A company employs a ranking system, often a scale of 1 to 5, to assess employees.
  • These numbers often come with generic and impersonal descriptions. The scores themselves are bucketed with percentages that limit the number of employees that may receive a given grade:
    • 1: Exceptional (10% of employees)
    • 2: Highly Effective (20%)
    • 3: Consistently Strong (40%)
    • 4: Partially Effective (20%)
    • 5: Not Effective (10%)
  • Managers are given a fixed $X in raises to hand out for annual reviews.
  • The $X budget for raises is distributed per the grading, with the 1's getting the biggest chunks, and 5's often getting nothing.
    • Note that this is how stack ranking is used to control budget costs. You always know exactly how much money is being given out in raises.
  • The 4's are warned that they're on the way to being fired if they don't shape up.
  • Typically, the 5's get fired or put on a performance plan.

This system - also known by the endearing terms "rank and yank," "forced distribution," and "grading on a curve" - is popular because it control costs, both in terms of annual raises and also under-performing employees. It serves as a system that forces the bottom 10% (or whatever the bucket is set at) out of your company regularly. This is not a bad thing in-and-of itself, assuming that the replacements hired are any better. Of course, this is where one of the major problems becomes evident.

Why Stack Ranking Sucks

Here's the thing: someone always has to be a 5. This system is built on the false assumption that there's always someone who is Not Effective on your team. It institutionalizes the idea of mandatory mediocrity. It is easy to see how ridiculous this concept is when you apply it to objects instead of people. For example, let's review 5 cars with stack ranking: a Ferrari, a Lamborghini, a Maserati, a Porsche, and Rolls Royce. Which one is the exceptional one? Which two are under-performing and mediocre? In this all-star set, they're all great, but stack ranking demands that one is worse than the others by a large margin. What happens if you employ good hiring practices and recruit a team of 10 amazing developers? What if they're all 2's and 3's, but you have to give out 4's and 5's? You end up having a difficult annual review where you find yourself apologizing and telling your developer how great you think they are. Because you really do think that. But your words are hollow, and their raise and review are the actions that speak louder; at this company they are thought of as mediocre, because someone has to be. Have you ever had a review where the actions of your manager didn't match their words? You're being told what an all-star and amazing player you are on the team, how important and awesome you are, and how everything you touch turns to gold, but your review says "you're average" and that big fat 3 rating is searing itself into your brain. You're wondering "if my boss thinks I'm so great, why is my rating average?" That's what stack ranking gets you. This review probably upset you, and now you're contemplating your options. Not a great outcome for you or the company. Stack ranking also stifles the desire of your developers to try new things, take on new roles with more responsibility, and take risks to grow their careers. This is because a 1 or 2 player won't want to take on the risk of joining a new team, or getting a new boss, who might rank them as a 3 or lower compared to their more seasoned colleagues. Indeed, the smart play is to stay right where they are, and reap the benefits of being on the good end of this ridiculous bell curve. My biggest concern with stack ranking is the fact that compensation is relative. Your assessed performance depends entirely upon the performance of your peers, as subjectively assessed by your manager. You might have been a 1 if it weren't for that person who happened to claim it. Now the best you can be is a 2. But if they didn't work there, perhaps you'd be a 1. Doesn't seem fair (or even rational), does it?

My Personal Experience With Stack Ranking

I have had a few jobs in the past that employed stack ranking. At one of them, developers picked 3 projects to be assessed on, and then both they and their manager ranked their performance on the projects from 1 to 5. One of the projects that I picked was something that I had built from scratch with a team of 4 people. The thing we built had made the company millions of dollars that year, and I was the lead on it. Naturally, I gave myself a 1 on it. My manager gave me a 3. I asked him how I could possibly be average at the thing which I created, was the most experienced with at the company, and which had led to millions in additional revenue. His reply was that I was awesome and not to worry too much about the grade. It was very confusing; what he said didn't match what he wrote. The review continued, and I ended up being given a 3 out of 5. I got a small raise. This was all conveyed to me as my boss happily told me that I was awesome, to keep up the great work, and that to keep it between me and him but I got the biggest raise on the team. The idea of having the biggest raise made me feel less wronged... Right up until I found out that it was a lie. My team and I were at lunch a few days later when one person bragged that he got the biggest raise on the team. Another immediately said "what? I was told that I did." I began to laugh as I realized what my manager had done. He had told us all that we received the biggest raise, and to keep it to ourselves. Perhaps as damage control for the pain that the mediocre grades had inflicted, but unfortunately for him we talked to each other. The jig was up, and now most of us were madder than we would have been had he said nothing to us at all. A side note: leaving that job was one of my favorite resignations. When I quit, my boss was distraught, but I said (paraphrased) "hey [boss], don't worry about it! I'm a 3 out of 5, so you should have no issues hiring any other average developer to take up the work I was doing." The look on his face told me all I needed to know: he had now realized for himself why stack ranking sucks.

How to Properly Review & Compensate Developers

The key to happy developers is fair compensation. Fair compensation is all about transparency. At Stack Overflow, we have a transparent system for assessing employee skills and compensation, which is lovingly called Be More Awesome. There is no magic in employee compensation here, and all developers know exactly what they're getting paid, why, and what they could get paid in the future. There are no negotiations, no bell curves, and no quotas. Be More Awesome (BMA) is pretty simple. It is meant to measure skills; we use performance as evidence of skill. A person may rate highly on their BMA for a skill, even if they haven’t used it in the previous year. There are 4 possible grades that can be awarded for each skill:

  • B: Could be more awesome. This is a good thing to work on over the next year -- and your manager will help. This can also be applied for new employees who haven't been in the company long enough to demonstrate that they deserve a higher grade.
  • A: Does as expected, at our high Stack standards. Completely, utterly able to accomplish what is needed.
  • A+: Does more than your team expects, even at our high level. Exceptional and noticeable skills.
  • A+++: Widely recognized level of amazingness. Does and teaches. When people think of this skill, they think of you (or would if they knew you). This will be rare, even on our amazing team.

You might notice that there is no grade below a B. We don’t have C's or lower because we believe that everyone that works here is awesome. If one of our developers were doing C, D, or F level work, we would already be working closely with them to correct it - prior to reviews. It's also not uncommon for people who are earlier in their programming career to receive a lot a B's. This doesn't mean they're doing poorly at all. It just means they're closer to the beginning of their career than the middle and there's a lot of opportunity for them to grow. The actual skills that we assess each year change, but the 2016 BMA chart for developers currently looks like this:

2016 Developer BMA

There are descriptions that explain what each of these categories are, and they are available for all employees to review at any time. Once a developer is assessed on a BMA, their letter grades get converted into a numeric score by using a formula that is also published internally for all developers to review. This formula outputs a numeric score between 0.00 and 5.99 (with 5.99 being the best grade), which is then rounded down to the nearest whole number. In short, a developer can receive a score of 0, 1, 2, 3, 4, or 5. Next we assess the years of programming experience. This is a value that falls in a range from 0 to 25. Naturally, this goes up by 1 at every annual review. The score and years of experience are then looked up on a chart that has years of experience on the X axis and score on the Y axis, and details salary amounts at each cell. It's worth redundantly noting that this chart is published internally also and can be reviewed by all developers at any time. Unsurprisingly, the cell that your score and experience points to is exactly what you get paid.

Make Compensation Transparent

There are no secrets or magic in our compensation system. All aspects of it are published internally for all developers to review at any time. They also get input into the changes to the BMA skills each year, well in advance of their annual review. They know the formula that we use to calculate salary. Most importantly, their compensation doesn't depend on the performance of anyone else. Everybody can be a 5 in our system and everybody can be a 0. Above all else, our system is fair and evaluates individual performance, not team performance. If you want happy developers and low turnover, I highly encourage you to try adopting such a system yourself. If your company is unwilling to do so, perhaps evaluate why. Are there secrets and magic in the compensation system that you don't want your employees to know about? Why do you value these hidden metrics? Do your employees feel valued? A happy developer is a productive developer, and while a fair system does not allow you to easily control salary costs in terms of budget (because everybody can be a 5), it does help increase job satisfaction, lower turnover, and maintain a relationship of trust with managers. And as I've written about before, if you don't have the trust of your employees, you will fail. EDIT (7/27/2016): We have published a salary calculator based on our internally transparent compensation numbers! Take a look. PS - hate stack ranking but love Stack Overflow? Come and work with us! Or discover our listings for developers using C#, WordPress, AWS and more at Stack Overflow Jobs. Follow David Haney on Twitter at @haneycodes

Login with your stackoverflow.com account to take part in the discussion.