Is the Stacked Ranking of Employees Actually Bad?


Vikram Bath

Vikram Bath is the pseudonym of a former business school professor living in the United States with his wife, daughter, and dog. (Dog pictured.) His current interests include amateur philosophy of science, business, and economics. Tweet at him at @vikrambath1.

Related Post Roulette

98 Responses

  1. Avatar Patrick says:

    Without some sort of concrete guidelines, managers (being human) are unable to really judge what constitutes extraordinary, ordinary, and unsatisfactory performance.

    This sentence has some problems.

    I mean, presumably if you’re managing some people who are doing a task, you should have some domain experience. One would hope enough domain experience that you can rate performance.

    If you need a checklist, you’re a blithering incompetent. You might be a good manager of some other problem domain, but not this one. You know the one-guaranteed-for-certain-underperforming member of your group is?

    Mirror, please! Mirror! Hurry up, the delay is ruining my pithy observational moment!

    Now, there are *aspects* to your organization that are outside of your control, and in many of those cases they might also be out of your domain expertise… and to the extent that those two things are true, it’s good for there to be guidelines in place to tell you what those factors are and how they relate to your group.

    But “keep two, fire one”, as a company-wide policy, is effectively “you’re probably a terrible manager”. I don’t care if Jack came up with it, it’s still stupid.

    Unless all your employees are relatively interchangeable cogs, anyway.Report

    • Avatar Vikram Bath in reply to Patrick says:

      what constitutes extraordinary, ordinary, and unsatisfactory performance

      This could perhaps be modified to “what ought to be labeled extraordinary, ordinary, and unsatisfactory performance so that those things have constant meanings throughout the organization.” The idea is to avoid one department saying *all* of their people are of course extraordinary.Report

  2. Avatar Jaybird says:

    Stacked ranking is one of those things that needs to be used sparingly and only when there is a systemic problem.

    Using it year after year after year? Ugh. This ain’t baseball.Report

    • Avatar Chris in reply to Jaybird says:

      I’m sorry, but you are performing below the level of a replacement programmer. We’re going to have to send you down to the minors, where you’ll be working on Windows 8.Report

      • Avatar Mike Schilling in reply to Chris says:

        It takes a lot of very smart people working very hard to create anything that dreadful. Mediocre programmers would have merely produced a bloated and less stable version of Windows 7.Report

      • Avatar Chris in reply to Chris says:

        I assume that would have been called Vista II.Report

      • Avatar Mike Schilling in reply to Chris says:

        I have never figured out why Microsoft switches between numbers, years, words, and cryptic two-letter names.Report

      • Avatar Patrick in reply to Chris says:

        Everybody forgot about Windows Me.

        If they were numbers, you’d remember Windows 6 as a ball of crap.Report

      • Avatar Chris in reply to Chris says:

        I remember buying a laptop with ME, using it for 2 weeks, and then replacing it with 2000. When I think about it, 2000 Professional may have been my favorite Windows version.Report

      • Avatar Will Truman in reply to Chris says:

        Windows 2000 was absolutely awesome.

        Leaving XP behind makes me very sad.Report

      • Avatar Vikram Bath in reply to Chris says:

        I didn’t get why everyone liked XP so much. I assume it must be because the consumer lines had such awful kernels and XP was the first stable one a lot of people ever used. I liked NT 4.0.Report

      • Avatar Will Truman in reply to Chris says:

        Windows 2000 was the perfect combination of resource management, stability, and capability. Windows XP was a more bloated, less efficient Windows 2000 with better driver support.

        I actually think Windows 7 is just dandy. The problem is that a lot of my older computers cannot handle it. The jump in system requirements between XP and Vista/7 is huuuuuuuuge.Report

      • Avatar Vikram Bath in reply to Chris says:

        I just want to note again that I have a philosophical distaste for your polygacomputing. I found myself much happier when I switched over to being loyal to a single well-specced machine.Report

      • Avatar Mike Schilling in reply to Chris says:

        I assume it must be because the consumer lines had such awful kernels and XP was the first stable one a lot of people ever used.

        I think that’s it exactly. I never used 2000 (went straight from 4.0 to XP at work and 95 to XP at home), and having a common stable OS was a huge plus.Report

      • Avatar Will Truman in reply to Chris says:

        I really tied going that route. I mean, tried *really hard*. Well, not really hard. And instead of one computer, I was going to reduce the primary station from four to three. I was going to retire two of them and bring in only one replacement.

        Then I realized that all it takes is a couple of parts, which, hey, I have right over here, and I could have five! Five!! (Excluding the two NAS systems, I mean. And the laptops.)

        XP’s retirement complicates these plans. I have a plan for the fourth. But the fifth isn’t good enough to run Windows 7. I don’t need two Linux machines. Though some of the laptops will likely be going over to Linux. I should be able to take parts from a couple of non-functional ones, at which point I will have a new one!

        That I will try like heck to figure out something to do with…Report

      • Avatar Rod Engelsman in reply to Chris says:

        It seems like every other release of Windows was a POS. Win95 was crap, but 98 was pretty decent. ME sucked balls, but the XP at least ended up pretty good. Vista sucked, then Win 7 was very good in my opinion. I haven’t tried 8 but reportedly it’s a confusing mess that can’t decide whether to be fish or fowl (desktop or tablet oriented).

        So I guess stick to the odd-numbered releases?Report

      • Avatar BlaiseP in reply to Chris says:

        The last release of Windows for which I had any respect at all was NT. With the advent of Java, I purposely began to avoid developing for Windows altogether. They’re all horrible. The APIs are just rotten. There’s no consistency to any of it. It’s just a sad, stinky bag of shit.

        It’s not that I hate Microsoft. I sorta pity them, truth to tell. They were dragged down by their own lack of standards. Standing on the main bridge between hardware and software, they were in a perfect position to make and enforce some reasonable standards. Not only didn’t they enforce them on anyone, they wouldn’t abide by their own standards, constantly working under the covers, giving themselves little advantages.

        Microsoft never cared about my problems as a developer and an architect. I just quit dealing with their shit.Report

      • Avatar Jim Heffman in reply to Chris says:

        Look up “OS-tan”.Report

  3. Avatar greginak says:

    Tod’s post from this morning was thoughtful and interesting. Pat’s post about the NSA was good, so i’m afraid there is no room here for this post Vik. Third best is like being the second loser or something like that. This isn’t the kind of blog that can accommodate posts that aren’t better than the others. Good luck in your future posts.

    This in no way gives you a strong incentive to go all Tonya Harding on fellow co-workers or fellow posters. Well actually it does but never mind that part.Report

  4. Avatar Will Truman says:

    I completely get the problem that stacking is supposed to solve. I’ve worked at places that were far too sluggish to identify the good and the bad. Even good managers can fall prey to it. Some managers who are great in other respects are just bad at that (and sometimes what makes them great managers elsewise is precisely what makes them bad at that).

    But in addition to the toxic environment it creates, it is often going to flat-out fire the wrong people. At the very least, I’d give managers the opportunity to basically explain why the people at the bottom might deserve to keep their jobs and/or what steps are being made to address their shortcomings, whatever they are. A lot of the same managers who don’t want to let particular people go might at the same time have a hard time filing a report defending them. And if they get transferred and the next manager doesn’t mince words, it would reflect poorly on the manager and I’d think that they’d know that.

    Of course, that assumes that there is a genuine problem that needs fixing. Sometimes there isn’t. When that’s the case, congratulations: you’ve created a problem that will need fixing.Report

    • I’ve worked at places that were far too sluggish to identify the good and the bad. Even good managers can fall prey to it.

      To be honest, I would be surprised if it weren’t “especially good managers” rather than “even good managers”. As Scott Adams suggests in the last link I put in, the skill of firing well is anti-social behavior.

      Frankly, this is part of the reason that outside consultants are hired for deciding who to fire. And, yes, that’s exactly from the plot of Office Space.

      Of course, that assumes that there is a genuine problem that needs fixing. Sometimes there isn’t. When that’s the case, congratulations: you’ve created a problem that will need fixing.

      Yes, this. I think it’s likely that firing the bottom can work well for a bit, but if it’s going to be a yearly process, you’re going to be cutting into flesh needlessly. Thus my suggestion that it be done here and there and not on any announced schedule.Report

  5. Avatar Mike Schilling says:

    Firing is vital pour encourager les autres. (And if you can’t read French, pack up your desk.)Report

    • This would seem to support that view: Link

      (I cheated with Google Translate. Does that count as reading French?)Report

      • Avatar Mike Schilling in reply to Vikram Bath says:

        Vikram, I’ve enjoyed blogging with you, but we’re going to be moving in a different direction.

        Actually, what that link says is that people hired to be researchers don’t teach as well as people hired to be teachers, as it contrasts both tenured and non-tenured but tenure-track faculty with adjuncts. That makes perfect sense in terms of incentives; since tenure-track faculty are evaluated far more on research than on how well they teach, they invest their time accordingly.Report

  6. Avatar Morat20 says:

    Mathematically, stacked ranking — at least the strict versions — can’t work.

    It assumes a normal distribution, centered on some ‘median worker’ — but that rests on several false assumptions. Yes, the distribution of workers is going to be a normal one — but the median point isn’t necessarily where managers assume it is.

    First, hiring is a selective process — you can’t slap a normal distribution for a random workforce onto a selected sample, because the sample is automatically biased towards the high end! Each and every manager is — for the most part — attempting to hire the best possible candidates. Yes, they will be wrong. But there’s a big filter there, and it effects the outcome.

    Secondly, there is not an infinite pool of talent — you cannot toss low performers and expect to hire median or better ones reliably. Even if you’re really successful, all you do is move your median higher along the workforce — meaning YOUR average worker is more talented than the average applicant. Which means your applicants will, on average, be worse than your regular workers — and if you shift it enough, you’ll be firing people where you are unlikely to find better talent and the churn this causes is worse for productivity than the ‘bad workers’ you were firing.

    Lastly, it’s all based on…bad metrics. If you can’t trust your management enough to take their honest opinions on their employees talent and productivity, to the point where you’re straightjacketing them, whatever analysis metrics you have for performance and talent are going to be bad. Which means your “normal distribution” is baseless — you have a distribution, but it’s a purely artificial one unlikely to be closely related to the metrics it’s claiming.

    Modern metrics aren’t as bad as IBM’s k-line system (it took longer than it should have to realize that rating coders by how MUCH code they produced was a bad, bad, bad incentive that rewarded poor coders and penalized good ones), but how to measure how ‘good’ a programmer is is…pretty darn squishy.

    Did he get his job done on time? Does that mean he’s good or underworked? Did he miss his milestones? Was that because he was bad, or because he got stuck waiting on other dependencies — or because his management had unrealistic expectations or because requirements changed?

    You either trust your management or you don’t. A stacked system might work once or twice, if you’re trying to prune the workforce, but it’s a ridiculous long-term performance tool.Report

    • Avatar Mike Schilling in reply to Morat20 says:

      it took longer than it should have to realize that rating coders by how MUCH code they produced was a bad, bad, bad incentive that rewarded poor coders and penalized good ones


      • “I know that the current method you use to assess programmer productivity includes accounting for errors. But we have found that QA is spending too much time counting the errors when filling out their reports. So, going forward, we’re going to base productivity only on lines of code.”


        You’d be surprised to hear that this did not, in fact, save QA any time.Report

      • Avatar Mike Schilling in reply to Mike Schilling says:

        When I was working at BART (many, many years ago), I was writing a gigantic YACC parser, and the metric was lines of C produced. I killed.Report

      • Avatar Troublesome Frog in reply to Mike Schilling says:

        Another great metric with respect to software QA: using “number of bugs reported” as a metric for test team members. I’d love to see that “number of bugs reported and fixed.”

        BUG 662: “Program crashes when I click OK.”
        BUG 663: “Program crashes when I click OK while wearing a hat.”
        BUG 664: “Something kind of weird happened. Once. Can’t reproduce. Not sure if I was wearing the hat or not.”Report

    • Avatar Vikram Bath in reply to Morat20 says:

      > It assumes a normal distribution

      This isn’t strictly true. It assumes some distribution of worker abilities though. @morat20 ‘s point about potentially hiring worse workers than you are letting go of still holds though.

      I think a stacked ranking advocate would reply that while this might be true, the idea is to churn through the available workforce in search of those who are worth keeping for the long haul, even if that means you let go of some really good people on the way there.Report

      • Avatar Mike Schilling in reply to Vikram Bath says:

        I can even see it as logical if you’re well-known and/or successful enough to have your pick of new grads. You hire only the well-above-average and let go the ones that don’t live up to expectations. Note that I said “logical”, not a good idea. You still have the over-competitiveness, the morale issues, the constant loss of institutional memory, and the incentive for anyone who’s not a mono- and ego-maniac to find something less precarious.Report

      • Avatar Morat20 in reply to Vikram Bath says:

        But that doesn’t work. You have to assume you have enough applicants each year to — theoretically — pick better hires.

        Except if you’re deliberately skewing your own median towards the high end (by systematically firing off your bottom end and thus shifting the middle of the curve to the right), then your applicant pool will get worse and worse compared to your current staff.

        More and more applicants would be ‘below’ your standards as you shifted your ‘average’ higher up the applicant pool.

        That’s even assuming you have somehow improved your hiring process, since obviously you hired the ‘low performers’ in the first place.

        That doesn’t even get into lost experience and expertise. It generally takes several months for an employee to get his feet under him and hit peak productivity. (Heck, it often wastes just a month getting through mandatory training, basic orientation and HR stuff, and learning the ropes). Which means there are hidden costs to firing that aren’t being accounted for.Report

      • Avatar Mike Schilling in reply to Vikram Bath says:

        There’s a new pool of college kids every year. Some of them will perform as expected, some not. You keep culling the ones who don’t to make room for the new crop, so (in theory) you’re keeping all the good performers and gradually replacing the others.

        That doesn’t even get into lost experience and expertise. It generally takes several months for an employee to get his feet under him and hit peak productivity

        Absolutely. That’s another reason it works only in theory.Report

      • Avatar BlaiseP in reply to Vikram Bath says:

        I like working with n00bs. They haven’t yet acquired bad habits. They’re still flexible. What I want to see in a new hire is pretty simple. Three virtues:

        1. Ability to rapidly master new skills.
        2. Ability to emulate existing best practices.

        The third virtue is tricky. Some people work best alone, without distractions. Others achieve more when working with others. A fresh n00b must always be paired off with more senior talent, where the first two virtues are given a chance to appear. N00bs must always have mentors.

        After some weeks, the mentor and n00b meet with management to horse out where the n00bs particular talents appear, and they will appear if there are any. I call this meeting the Sorting Out. Nothing particularly negative is to be said at the Sorting Out. Its purpose is to determine where this n00b belongs in the pack, where his particular skills would be most useful and profitably applied.

        Some people shouldn’t be coders at all. They should be testers. Some should be business analysts. Others should be DBAs. Others should be project leaders, I’ve seen more than one n00b step into the role of project lead, simply because they demonstrate leadership potential. It’s a tough job, wading through a pile of poorly-written spec, dealing with clients, dealing with the administrivia, liberating the more-technical people from things they’re no good at and the n00b is better at,

        After the Sorting Out, the n00b is given a different mentor, one within his area of demonstrated skills. Everyone’s good at something. The key is finding it.Report

      • Except if you’re deliberately skewing your own median towards the high end (by systematically firing off your bottom end and thus shifting the middle of the curve to the right), then your applicant pool will get worse and worse compared to your current staff.

        This argument appears to concede though that the system does succeed in moving the median worker to the high end in the first place so that firing good workers becomes a risk.Report

      • Avatar Mike Schilling in reply to Vikram Bath says:

        I call this meeting the Sorting Out.

        You should make them wear a hat.Report

      • Avatar morat20 in reply to Vikram Bath says:

        This argument appears to concede though that the system does succeed in moving the median worker to the high end in the first place so that firing good workers becomes a risk.

        Yes, for the sake of argument I assumed it did — then pointed out that even IF you started with a normal distribution of skill and talent (which you don’t, but okay) and even IF your stacked rankings allowed you to fire the low end, you’d quickly move to a position wherein your ‘average’ employee — and even your ‘low’ employees were further along the curve, and thus your hiring pool would look worse and worse.

        In short, statistically you’d soon be firing people you can’t replace, because the median applicant is worse.Report

    • Avatar J@m3z Aitch in reply to Morat20 says:

      Adding on to Morat’s point, on which group are we basing our distribution and mean? It’s one thing if it’s our whole corporation and there’s a reasonably large n. But if we do it for each ptoject team, we’re implicitly assuming each project team’s distribution at least roughly matches the corporate distribution. But if the teams are relatively small that assumption won’t hold, even employees were randomly distributed among teams. As a result, we might be firing the bittom person on team 1, even if s/he is superior to, say, the bottom three on team 2 and you’d do better to fire a couple people on that and transfer him/her to it.Report

      • I agree. This is the straitjacket mentioned. I would suspect that an advocate would acknowledge this deficiency in any given year but argue that over a sufficient number of years it would all even out across teams.

        Additionally, they would point to the corrosive effects of the alternative of not firing anyone.Report

      • Avatar Mike Schilling in reply to J@m3z Aitch says:

        “This is the company’s biggest priority, so we’re putting together a team of our best people. Too bad we’ll lose 10% of them next July, but c’est la vie.”Report

      • Avatar J@m3z Aitch in reply to J@m3z Aitch says:

        argue that over a sufficient number of years it would all even out across teams.

        Well, if by “even out” you mean each team has someone fired who’d be better placed on another team, sure. 😉

        The problem is it can never really even out because there’s a boundar problem. You can fire someone who’s better than the worst, but you can’t fire someone who’s worse than the worst, so the firing necessarily skews up into folks who shouldn’t be fired.

        The whole thing smacks of pop management ideas promted by people who are kind of clever, but didn’t pay much attention in their stats classes.Report

      • Avatar Mike Schilling in reply to J@m3z Aitch says:

        people who are kind of clever, but didn’t pay much attention in their stats classes.

        “Managers” is much less typing.Report

      • Mike, I worry that you’ve never had a good manager. I have, so I know they exist. Rare beasties, though, I believe.

        But could you or I really do better? I know without a doubt, from experience, I would be a lousy manager. So for me there’s a whole pots and kettles and yes, indeed, I have walked a mile in their shoes awareness that I think many employees who’ve never managed lack.Report

      • Avatar BlaiseP in reply to J@m3z Aitch says:

        I’m with J@m3z on this one. Management is just another skill set which can be mastered, if you’re willing to first be a servant to those you manage. The best managerial technique I’ve ever seen was taught by the US Marine Corps, who consider the Servant Leader the key to their many successes.

        Even in the Army, officers eat only after enlisted men have been served.

        A good leader is only as good as his followers. Humans want to be led effectively. They want to belong to something greater than themselves. They want to achieve great things. They want to succeed and prosper. None of this is possible without the auspices of a Servant Leader.Report

      • Avatar Mike Schilling in reply to J@m3z Aitch says:

        Mike, I worry that you’ve never had a good manager. I have, so I know they exist. Rare beasties, though, I believe

        My experience matches yours. I feel OK about generalizing based on the vast majority, though.Report

      • Mike–That’s fair, but you glossed over the really important part of my question.

        Blaise–“Servant leaders.” Indeed. The value of that is difficult, if even possible, to overstate.Report

      • Avatar BlaiseP in reply to J@m3z Aitch says:

        As long as you understand you’re likely to be a lousy manager, you are on your way to being a good manager.

        As I’ve been saying, I just finished up babysitting my friend’s business. Finally back in my own place with my own cat in my own Bissinger’s chocolate box. Tough, tough few weeks. 24/7 kind of operation.

        So I’m sitting with the drivers at the conference table. I flat tell them, “I haven’t driven a mile on these roads. You know this job and I don’t. But I have put most of our clients into Google Earth — and you — and most of our destinations — see here — (and pulled up Google Earth, showed them all the push pins in the map).”

        “And what’s more”, I told them “I’m not going to dispatch anyone without some validation from the two senior drivers. So every day, each of you will be given the full itinerary of all the drivers, also in Google Calendar. If something goes sideways, we’ll all be able to look at the itinerary and adapt.”

        “And another thing, I want everyone to either come into the office, every day, or call in, so we can discuss the next day’s runs. There will be no bossing around here. I am your servant. I am the Idiot Office Boy. You are the drivers. But you will all sing off the same sheet of music or there will be trouble for us all.”

        Everything worked out — with the exception of one driver, who had been previously fired, once, and rehired, supposedly because she was needed. She was nothing but trouble. So I was obliged to make her call in after every run, so she could “work well under supervision.”Report

      • Avatar Mike Schilling in reply to J@m3z Aitch says:

        When I’ve managed, I’ve almost always had good people, which makes it a snap. You make sure they have what they need and stay out of the way. Managing problems employees is much harder, and, no I’m not very good at it.Report

      • Avatar J@m3z Aitch in reply to J@m3z Aitch says:

        Well, in your favor, not many people even manage good employees well.Report

      • Avatar Lyle in reply to J@m3z Aitch says:

        I agree also, to do it fairly it needs to be done over groups of 100 or more. This of course does consume management time as the various groups compete to rank. If a team is truly outstanding then the other managers in the larger business group should be aware of it.Report

    • Avatar Jim Heffman in reply to Morat20 says:

      Stacked ranking also has trouble dealing with interdisciplinary teams and sequenced work. People generally do not start from a blank sheet of paper and carry a project all the way to boxes-on-the-shelves entirely on their own. And if the guy who was supposed to write the print driver doesn’t get that job done, it doesn’t matter if you’re Einstein with a C compiler–you will *not* complete the assignment, and if your stack ranking is based on assignments completed then that’s a problem.Report

      • Avatar morat20 in reply to Jim Heffman says:

        There’s also small teams to consider — the smaller a team, the less room there is for slackers and poor performers, and the more obvious it is who isn’t up to snuff.

        You work in a shop of 100 people? Well, 1 or 2 really poor performers might get lost in the noise. Your team is 5 people, one of whom also wears a ‘project lead/manager’ hat? Yeah, you know if someone’s not making the cut and you’ve already gotten rid of them.Report

  7. Avatar Rod Engelsman says:

    I’ve never worked in a stacked-ranking environment, so I can’t add any personal insights to this other than to compare to other systems.

    For example, the Navy uses (or at least used) an evaluation system where supervisors (generally your immediate superior) graded you on several characteristics (rating knowledge, military bearing, etc.) on a scale of 0 to 4. This suffered enormously from the “Lake Woebegone” syndrome, or something akin to grade inflation. A 3.5 was considered barely adequate and a 3.0 was what you gave an extreme fuck-up. Private Manning to this day, probably would still get at least a 2.5 or so if that’s any indication. As a consequence there was little actual differentiation on the basis of actual performance.

    I can’t speak for the office staff, but as a driver I’m ranked by fairly objective performance criteria. You’re either (unranked), Bronze, Silver, Gold, Platinum, or Diamond. There’s an average-miles-driven-per-week criteria as a base qualification for each level and cut-offs for Service Failures (mostly, but not limited to, picking up or delivering late) and Preventable Accidents. But it’s not stacked in that there’s no preset number of drivers in each category. You get rewarded for being at a higher level with a performance bonus paid quarterly. (As a Platinum Driver–yay!–it amounts to about an extra weekly check.) Firing offenses are mostly either too many service failures or certain categories of accidents. So basically, do your job, work hard, and don’t fuck up and you’re golden.Report

    • the “Lake Woebegone” syndrome, or something akin to grade inflation

      Yep, this is exactly what the stacked rankers fear happening. In what you are describing, they aren’t really making use of the full five points on the scale.

      It sounds like you’re pretty happy with the system you’re evaluated by as a driver. Usually, it’s worth examining how people will ultimately try to subvert the system. E.g., jacking up the vehicle and letting the miles roll on by Ferris-Bueller style.Report

      • Avatar Rod Engelsman in reply to Vikram Bath says:

        Regarding the Navy system… I think this is a particular problem with the military that may or may not be so relevant to private industry. In the military it’s pretty explicitly understood that part of your job description is developing your subordinates. So if you have a sailor or soldier that you give a poor evaluation, in some sense that reflects poorly on your how you’re doing your job as a leader as well.

        I think a lot of what happened, for example, with Nidal Hissan leading up to that Fort Hood shooting can be chalked up to that mentality. His superiors were loathe to seriously downgrade his performance reviews because that would in turn downgrade their performance reviews, whether that was fair or not.


        Yeah, I think it’s a pretty good system. You get a sense of where you stand relative to your peers while not really being in direct competition with them. And I want my fellow drivers to do well. Losing a customer because some other driver in the company was a slacker hurts me too. Customers aren’t and shouldn’t distinguish one driver at my company from another; all they see is the name of the company on the truck and if they have had a bad experience with another driver it reflects poorly on me regardless of how good I am.

        At the same time, if due to competition or a weak economy, cuts have to made, being ranked toward the top is some nice job security.Report

      • I think it’s worth noting that at least in the long term, even such a system as you describe is still competitive. If everyone improved to the point that everyone qualified as a diamond driver, new criteria would be set. (And the same would happen if traffic increased and everyone became an unranked driver.)Report

      • Avatar Kim in reply to Vikram Bath says:

        maybe not. If it’s to incentivize good behavior, and increase profit, then you’d be okay with continuing to pay people a bit “extra”. And there’s always the n00b.Report

    • Avatar Mike Schilling in reply to Rod Engelsman says:

      I remember that from The Caine Mutiny. The captain before Queeg saw Willie Keith not taking his job seriously, and threatened him with an Above Average performance report instead of an Excellent.Report

    • Avatar Patrick in reply to Rod Engelsman says:

      This suffered enormously from the “Lake Woebegone” syndrome, or something akin to grade inflation. A 3.5 was considered barely adequate and a 3.0 was what you gave an extreme fuck-up.

      Again, if you look at a department and the average score is above 2.5, consistently… well, see this comment above. Either the department is awesome or the manager isn’t doing their goddamn job. If the department is awesome, you don’t even need to know how they do what they do, you’ll be able to tell by asking people who have to interact with that department.

      So do that. And if everybody says the department is awesome, kudos, you have a great manager who’s running a great department. And if everybody doesn’t say the department is awesome, you’ve got grade inflation and somebody who doesn’t manage well.

      Seriously, a good number of these management techniques seem (to me, anyway) to be a method for correcting for bad management by encouraging or discouraging turnover among line employees. We can’t fix our managers, so change up the guys and gals below them so that maybe things will get better. Stacked is the worst, look at the consequence for a good manager. If you’re good at hiring, and good at managing, you may very well have a great department with an abnormal worker distribution and the distribution is all good. Now you have to fire a good worker and only give kudos to two people?

      This is the dumbest fucking thing on the planet. It’s straightjacket management.

      If your manager can’t do the job, there’s your employee to fire, right there. Fire that person, or demote them, or transfer them. And if you wind up firing five managers of the same Veep in a row, well, you’ve probably identified another problem. Fire that Veep.

      Mangers make more money than line workers, ostensibly because they’re better at making strategic decisions. Mangers make decisions that can seriously mess up their line workers’ lives, or seriously make them better. We have a lot of terrible managers in our workforce.

      There. There’s your problem. Solve that problem.Report

      • Avatar Vikram Bath in reply to Patrick says:

        You do realize that if a manager knows in advance that that is the company’s philosophy, then he will fire one of his employees regardless of merit anyway just to prove that he’s good at making tough decisions. It’s touching your nose going the long way around your head, but it’s still touching your nose.Report

      • Avatar Patrick in reply to Patrick says:

        If you fire an employee regardless of merit, I predict that within two cycles of hire everybody in the company is going to think your department is wretched.

        Because the minute you start firing people willy-nilly, all the people who are good start looking for new jobs.Report

  8. Avatar BlaiseP says:

    Stack ranking works, but only in one set of circumstances: downsizing.Report

  9. Avatar Kazzy says:

    “Without some sort of concrete guidelines, managers (being human) are unable to really judge what constitutes extraordinary, ordinary, and unsatisfactory performance.”

    Couldn’t we… shouldn’t we… just hire better managers who are better able to judge?Report

    • Avatar Morat20 in reply to Kazzy says:

      They’re stack ranked too, but they’re aware of it and can game the system. One of the fun little tidbits from MS was that you NEVER wanted to work for a new manager, because he lacked the savvy and experience to protect his team when horsetrading rankings came around.Report

    • Avatar Troublesome Frog in reply to Kazzy says:

      There’s a good argument to be made that for any reasonably sized organization, institutional procedures should be designed to make the median employee successful, even at the expense of good ones being extra productive. A really flexible system can rock in the hands of talented managers, but the reality is that the average manger is… average.

      You can beat it by hiring carefully in a small company, but in a company of thousands, you’re battling the law of large numbers. Just get used to it and start handing out safety scissors and making sure that you keep the paste on the top shelf out of easy reach.Report

      • Avatar Patrick in reply to Troublesome Frog says:

        You can beat it by hiring carefully in a small company, but in a company of thousands, you’re battling the law of large numbers.

        Then the serious question needs to be asked: are you really getting a competitive advantage from the size of your company?Report

      • Avatar Troublesome Frog in reply to Troublesome Frog says:

        That’s a great question. My experience is that for pure software projects, I’m more than happy to go toe-to-toe against giant companies if I’m on a relatively small team of good workers who work well together. A team that’s small enough to self-manage is practically unbeatable as long as it’s full of people who are capable of working together and self-managing.

        I think that the model falls down elsewhere. A small team of good coders can compete with a big company on product quality. It can deliver a whuppin’ on product cost and time to market. But it probably can’t deliver nearly the same technical support or sales capabilities. But you can’t grow your tech support and sales team without stressing the engineering team, so eventually that team needs to grow.

        It’s super hard to grow engineering teams without losing that “handful of really great engineers” magic. That’s probably true for other industries as well, but it seems to be particularly thorny in tech land.Report

    • Avatar Vikram Bath in reply to Kazzy says:

      what constitutes extraordinary, ordinary, and unsatisfactory performance.”

      Couldn’t we… shouldn’t we… just hire better managers who are better able to judge?

      Part of the problem is consistency across the organization. And it’s very possible to find two good, wholesome people, put them into management positions that they are good at and have very different ideas about what is merits the label “extraordinary”. It’s not just about making the decision. (Of course, the problems mentioned in earlier comments about talent not being evenly distributed among teams still applies.)

      Keep in mind that performance evaluation is probably less than 10% of what a manger actually does anyway. The other 90% is probably just getting stuff done on time. You might not want to get rid of a manger who excels at that simply because they are too loyal to their team.Report

      • Avatar Patrick in reply to Vikram Bath says:

        Well, if you get stuff done on time, you can have under-performing people on your team. Hell, you can have them just for comic relief and team spirit.Report

      • Avatar morat20 in reply to Vikram Bath says:

        In my experience, getting things “done on time” means you’re under-worked.

        Estimating how much time anything takes in the software world is such a fun guessing game, especially as the software grows complex and you introduce dependencies. (X can’t get started until A is done, etc).Report

      • Avatar greginak in reply to Vikram Bath says:

        Or you can use Scotty’s rule from TOS/TNG. Always claim it will take you double the time to get something done then it really will. Then when you get it done in half the time you are a hero.Report

      • Avatar Badtux in reply to Vikram Bath says:

        @morate20, if your projects are consistently late, either a) the project was poorly spec’ed, or b) you have not properly planned the project execution, or c) the deadline was just a suggestion to begin with. I can assure you that if you have set a marketing deadline of April 20 so that you can get the ads all staged out by the end of January, my team will meet that deadline with a product that is at least MVP (Minimum Viable Product). It may not have every frill and feature on the MCL (Marketing Check List), but hey, time, features, reliability. Choose two.

        Managing dependencies is the biggest job of a project manager and making sure everybody understands what inputs they can expect from their upstream and what outputs their downstream expects is one of the major things a project manager is supposed to coordinate. What I *will* tell you is that 90% of the time, when person A says he can’t do anything because person B is blocking him, person A is lying. He can mock up the inputs and outputs of his module to get it done. Heck, I even did that with a network stream processing module once, I put injection points at the input and output that redirected the I/O to/from files instead. If you can’t mock up like this, you can’t do proper unit tests, and you’re going to have a failed project anyhow.Report

      • Avatar Troublesome Frog in reply to Vikram Bath says:

        In my experience, getting things “done on time” means you’re under-worked.

        This is an interesting way of looking at it, and it may be a good state of affairs as long as we know that “under-worked” means that you have enough extra time in the schedule to absorb surprises and not that you need more work. I’ve had this conversation with many managers.

        When anybody gives you an estimate, there are error bars around it. But even people who recognize that tend to think of the error function as something Gaussian. But there are infinitely more ways to end up with a surprise delay than there are ways to end up unexpectedly ahead of schedule, so it’s a heavily skewed distribution.

        When management says, “Can it be done by X date?” there’s an implicit probability attached to that. So yes, if you’re delivering project after project on time, you’ve probably selected a spot in that distribution that’s waaay past the median date. But that may be OK for the type of business you’re in. Alternately, it may be better to operate at the 80% confidence level as long as management understands that an average of 1 in 5 projects will go beyond that date even if estimates are dead nuts on.

        Most managers seem to think that if you’re running at anything less than 100% of potential output, you’re wasting resources. In software-land, this seems silly. A programmer is a tool that’s capable of making other tools. They should never be “idle” even if they’re not working directly on a product deadline. There’s always something productive they can do, so having some excess capacity isn’t really pure “waste.”Report

  10. Avatar Badtux says:

    As someone who understands elementary statistics I can see the problem with stacked ranking right away. It will improve your company’s average skill level only as insofar as the people getting hired are better than the people getting fired. The thing is, after a certain amount of time you’ve already fired all the weak engineers and are now firing engineers that would be considered above average in most of the industry.

    Replacing above-average engineers with engineers with a higher skill level than what you’re firing is hard enough in a normal company where the above-average engineer left to go to another opportunity that was more in line with his long-term career goals or higher pay or both. It is difficult to impossible once word gets around in the industry that even having above-average skills can get you fired as a “poor performer” at Microsoft. Why would I take my above-average skills to Microsoft and risk that kind of impact to my career, when there are thousands of other companies clamoring for my skills as an above-average engineer where I could spend the next fifteen years of my life without worrying about being fired as a “poor performer”?

    Yes, I actually had a manager at Microsoft try to recruit me to his team. I was, like, you gotta be kidding me. I don’t need that kind of stress in my life. I like the stress of creating kick-rear hardware and software on time and on budget, not the stress of worrying about my job. Not that I really have to worry, I’m at the point in my career where I can get a job by calling one of my former bosses at their current company and they’ll make a position for me if one doesn’t exist, but I also know that these former bosses aren’t going to fire me (I’m part of their stable of talent that they haul around the industry from startup to startup, duh). We might run out of money like at my last job, but running out of money is way different from being fired as a “poor performer”. Just sayin’.Report

    • Avatar Vikram Bath in reply to Badtux says:

      This was a study I did not include in the original article, indicating that almost all the improvements to be had are obtained in the first years of using the system.Report

      • Avatar Badtux in reply to Vikram Bath says:

        Also see this study, which basically says that the bell curve doesn’t apply to organizations that have highly competitive hiring processes to begin with. What you end up with is a “chopped off” bell curve, basically the right hand side of the curve, where the vast majority of people are clumped down at the bottom of performance and then you have a long trailing tail to the right of higher performers. At that point any gain in workforce quality by firing part of that big glop to the left of the curve is minimal, and the expense in terms of workforce morale, recruitment costs, knowledge retention, and willingness of workers to cooperate and innovate will outweighs any benefit. Especially if it detracts from attracting and retaining people in that tail to the right.Report

  11. This reminds me of when I was a TA and worked for the occasional professor who graded on a strict curve, usually as a way to “fight grade inflation.” Not necessarily a bad goal, but it’s easy for a professor to mandate and harder for the TA, who actually knows the students and helps them personally. I used to feel uneasy telling my students “good luck on the exam” because I knew if enough of them did really well, I’d still have to give a large number of them mediocre grades.

    In practice, the students’ performance usually approximated a bell curve anyway. Byt that may have reflected a bell-curve like bias in my grading, where I was implicitly comparing “excellent” essays with “good” and “mediocre” ones.Report

    • Avatar J@m3z Aitch in reply to Pierre Corneille says:

      I thought of this in terms of grading, too. In a course where there are regularly a hundred or couple hundred students this might be a reasonable way to fight grade inflation, since the classes are large enough that–barring some other intervening variable–the median performance and curve, ought to be the abought same from year to year. It can also obviate that intervening variable of different graders (TAs) from year to year.

      My own classes are small enough that the particular set of students does make a difference year to year, so grading one year’s group with the same strict curve as another year’s group would be utter nonsense. Additionally, we’re small enough that a change in recruitment efforts can cause a major change in an intro class’s median. This year we (thank god) mostly abandoned a recruitment strategy that had brought in, as one subset of our frosh classes, a specific group that significantly underperformed their classmates (and mostly failed out after one or two terms). Had I employed a strict curve in my frosh I would have had to pass some egregious non-performers the last two falls. Were I to try this term to hit last year’s disturbingly low-end skewed curve, I would have to fail more students than–I can already predict–would deserve it.Report

      • With some exceptions, most of the classes I TA’d for were large (100 to 200 students) ans split up into smaller sections of 30’ish students, with each TA being responsible for two sections, or about 60 students. And TA’s, as “section leaders,” usually had a lot of autonomy in how channeling how the class was taught. The students would go to a lecture 2 times a week and to section 1 time a week, wherein some magical thing called “discussion and critical thinking” was supposed to take place somehow. So it’s a large sample, but broken up into smaller samples. I’m not sure how that parses when it comes to curving.*

        Was poli-sci TA’ing classes like that when you were a grad student?

        *I’m also not sure how good that is for the undergraduates. It’s a pretty good funding opportunity, learning opportunity, and CV line for the grad students, but I’m not sure if in practice it’s anything more than just moving students through the education mill so they can get a slip of paper that qualifies them for a job a high school graduate can do perfectly competently. I suppose I’ve just gotten cynical a bit.Report

      • Avatar J@m3z Aitch in reply to J@m3z Aitch says:

        “section leaders,” usually had a lot of autonomy in how channeling how the class was taught

        Oh, lord, yes. I somehow (luck of the draw) only TAed a couple classes of this type, though. But one–Women and Politics–was taught by one of the worst teachers on earth, a person who rarely completed her sentences. Imagine a whole lecture composed predomimantly of half sentences, broken off in the middle, often with the words, “you know,” as a new thought hit her and she changed directions. It was horrible to sit through. My best evaluation said “I’m glad my TA was able to explain what she was saying.” In reality, I wasn’t, and couldn’t–I just helped explain the reading material. And since I was grading them, that worked out fine.

        My other great review from that class said, “I’m glad James was my TA and not some radical man-hating dyke.” I shared that with the other TA, who was in fact a lesbian, but of the log cabin type, and not at all hateful of men, or the “angry feminist” type. It was good for a laugh.Report

      • Ugh! I have a lot of war stories that I won’t relate because it would threadjack this post. But I did have at least one situation similar to the non-sentence completer.Report

      • Even large classes can have dramatically different mean levels of performance. You get different kinds of students registering when your class is 8 AM Mondays, Wednesdays, and Fridays than when it is Tuesdays and Thursdays at 5:30 PM.Report

      • Vikram,

        Good point. My statement assumed fairly random assortment, but factors such as you mention can bugger up that assumption.Report

  12. Avatar Damon says:

    It’s funny. I read that article and it took me a while to realize that MS’ stacking wasn’t the same as how my old company did it.

    We ranked ALL employees, periodically. It wasn’t used for compensation. It was used for layoffs. Everyone was given a ranking in the dept. The company deptartments met and ranked everyone in the company, etc. all the way up the chain. The loweset ranked folks were at the highest risk for getting fired.

    What MS did just seems bizzare.Report

    • Avatar Lyle in reply to Damon says:

      In general departments are told you have an x% raise budget and have to decide how to divide it up. If you used forced ranking here, but not for layoffs other than during downsizing, it could work. If someone gets 0% raises it should send a strong message if percentage increase in the overall salary budget is stated. I agree that during downsizings, or re-orgs the stacked ranking can be used to ease these folks out, in particular if they have gone a few years with 0% raises and have not gotten the soft message that implies.Report

  13. Avatar Jim Heffman says:

    It’s funny how people think “stacked ranking” is actually an attempt to fairly judge performance. It has more to do with staving off lawsuits accusing relatiatory or prejudicial employment activity (that is, “they fired me because I blew the whistle / was gay / wouldn’t screw the boss”)Report

    • Avatar morat20 in reply to Jim Heffman says:

      I call BS on that. Straight forward, absolute, BS. Discrimination lawsuits are practically impossible to win without your manager being an absolute idiot, and at-will employment is, well, more or less American law everywhere.

      If your supposition had even a smidgen of truth behind it, you’d find stacked employee rankings used most heavily in states with the most exemptions to at-will employment.

      Which is funny, because Silicon Valley is notoriously at-will (in fact, at-will employment is often described as one of the key factors for it’s huge success) and we’re talking software engineers here.Report

  14. Avatar roger says:

    Forced ranking is absurd and neglects basic psychology and anthropology. Take a look at hunter gatherers and you will find teams of egalitarians.

    Forced ranking leads to some extremely destructive behaviors:
    1) it undermines cooperation and selfless sacrifice necessary for teams to thrive
    2) it fosters hierarchical factions of employees who take care of each other in ways harmful to the corporation (good ol’ boy networks). Failure to be in a network is death.
    3) it completely misses the fact that the most important link sometimes comes from marginal players. Consider the story of the battle lost for the want of a nail. Good teams are not composed of just superheroes, but sometimes depend upon the essential contributions of a diverse group
    4). It destroys creativity and risk taking, as most bold ideas fail, and failure is a bad place to be in forced ranking.
    5) In addition change and creativity creates waves, and waves get you killed by other coalitions who will resist any change which lowers their relative fitness.
    6) it diverts the focus from customers, products and competitors to an internal hamster trail.

    That said, it is the responsibility of management to create institutions which eliminate free riders, weak links and bad apples. Indeed, employees and customers expect it.

    Microsoft is its institutions. Its institutional incentives and structures sucked, and not surprisingly, so have its products. I completely gave up on all things Microsoft years ago. Hasn’t everyone?Report

  15. Avatar Alan Scott says:

    If you only reward the best two people on a team and fire the worst, isn’t that a huge institutional disincentive to creating a team of brilliant people?

    This sounds like a structure that overencourages “safe”. One that would create nothing more than tepid improvements of existing products and belated clones of competitor’s successes.Report