The Value of Downvoting, or, How Hacker News Gets It Wrong
Paul Graham’s Hacker News is a great website to find interesting programming links and sane discussion. The site reflects a sort of post-Reddit sensibility; the design of HN was directly intended to address the shortcomings of programming.reddit.com from someone very much on the inside (Reddit was a Paul Graham Y Combinator startup). As such we studied it closely when building Stack Overflow.
It is true that discussion on Hacker News is more serious and less incendiary than the wild-west anything goes of programming.reddit.com. I’ve seen this firsthand, on blog articles I’ve written that have been posted to both sites. In What I’ve Learned from Hacker News, Paul explains:
It’s pretty clear now that the broken windows theory applies to community sites as well. The theory is that minor forms of bad behavior encourage worse ones: that a neighborhood with lots of graffiti and broken windows becomes one where robberies occur. I was living in New York when Giuliani introduced the reforms that made the broken windows theory famous, and the transformation was miraculous. And I was a Reddit user when the opposite happened there, and the transformation was equally dramatic.
I’m not criticizing Steve and Alexis. What happened to Reddit didn’t happen out of neglect. From the start they had a policy of censoring nothing except spam. Plus Reddit had different goals from Hacker News. Reddit was a startup, not a side project; its goal was to grow as fast as possible. Combine rapid growth and zero censorship, and the result is a free for all. But I don’t think they’d do much differently if they were doing it again. Measured by traffic, Reddit is much more successful than Hacker News.
But what happened to Reddit won’t inevitably happen to HN.
It’s a good read for anyone interested in building communities online. As you might imagine, I read it with particular interest since we’ve been running a full blown (and far larger than I would have predicted) programming community over the last 7 months.
Perhaps the most notable difference between Hacker News and Reddit is that it’s impossible to downvote anything on Hacker News. There exists one, and only one, form of vote: the upvote. So you can either upvote something, or do nothing at all. It’s an interesting design decision, but ultimately a bad one, in my opinion.
(update: Apparently it is possible to downvote comments, which I never realized. It is buried in the faq🙂
Why don’t I see down arrows?
There are no down arrows on submissions. They only appear on comments after users reach a certain karma threshold [ed: this is unstated for some bizarre reason, but it is currently 100].
(I apologize for my misunderstanding, but there’s no visible UI for downvoting, and I can’t recall ever seeing a single negative voted comment in all the times I’ve visited Hacker News! Also, I put these comments in parens to make them extra-LISPy so Paul Graham would see my corrections.)
Let me get some important caveats out of the way: we have to be careful in drawing comparisons between Hacker News and Stack Overflow, because they are fundamentally different sites. We’re a Q&A site with some accidental discussion, and Hacker News is a site that exists for the express purpose of discussion and link sharing. So to the extent that we have different missions and different goals, approaches that work for our site might not work for HN, and vice-versa.
On Hacker News, every post effectively starts at zero (technically, one implied upvote, which is your own, but we’ll call that zero), and can be upvoted indefinitely.
The advantage of this system is that nobody gets downvoted, but at a steep cost: we’ve lost half the potential information. If a post has zero upvotes, does that mean it’s bad? incorrect? uninteresting? mediocre? There’s no way to tell, because zero has multiple meanings.
If you add back in the negatives, suddenly the range is doubled. An evil or incorrect post is now different than a mediocre or uninteresting post, because it will have downvotes and a negative score.
But getting downvoted isn’t anyone’s idea of a good time. It’s tempting to disallow it entirely, to avoid this inevitable discussion:
Please do something else to discourage downvoting. Maybe increase the cost to the downvoter (there’s already a “declined” on force user to comment on downvoting).
This isn’t about points. It’s about participation. Downvoting should be reserved for nasty/offensive/stupid/poorly-thought-out/totally-off-base comments. If someone spends the time to make an honest effort to answer a question, but it’s not that great an answer, just don’t upvote them… Downvoting sends a message, “We disapprove. You spent your valuable time, but we don’t care.” It makes me think, why should I bother spending the time to write up answers for this forum?
I stopped posting on several usenet newsgroups because the major participants were just nasty and sarcastic. Don’t let this happen to Stack Overflow.
You could argue that the saner level of discussion on Hacker News is because downvotes are disallowed. I’m not so sure; I think it’s more attributable to the fact that Hacker News is relatively young, having launched in February 2007, and the small (but growing) size of the community.
In building Stack Overflow, we realized the intrinsic informational value of full range post scores. Downvotes give you the critically important ability to distinguish between the good, the bad, and the ugly. Without downvotes, how can you possibly tell the difference between a post that is harmless but uninteresting, and one that is actually wrong or harmful? Sure, it stings a bit to get downvoted. I’ve been downvoted myself on Stack Overflow. And each time, it makes me pause. But that’s good! That’s necessary! You have to believe there are potential consequences for every post you make — both good and bad. This is how things work on real playgrounds; why would we expect our web playgrounds to be any different?
The idea of a world where nobody can be downvoted strikes me as more than a little utopian. Is it realistic for users to expect to post in an environment where there are no penalties at all, no way for their peers to express disapproval or disagreement with their post? When you can’t leave a quiet, anonymous downvote, you’re more likely to post a snarky reply to express your displeasure. That’s why disallowing downvotes is actively harmful to community.
The problem isn’t downvotes, per se, but encouraging responsible downvoting. That’s why on Stack Overflow, we do it this way:
-
Upvotes add 10 reputation to the post author
- Downvotes remove 2 reputation from the post author, and 1 from your reputation
The trick here is that downvotes are mostly informational. The cost of a downvote to the users’ reputation (or karma in Slashdot/Reddit parlance) is quite low. It would take a whopping 5 downvotes to equal the effect of a single upvote. And, on top of that, downvotes cost you a tiny bit of reputation. The net effect is that you have to feel very strongly about something to downvote it. Downvotes are serious business, and not to be cast lightly. We designed our system around that maxim.
Does it work? I think the data itself tells the story. Here are the total number of votes cast on Stack Overflow through 3/7/2009:
| upvotes | 1,251,020 |
| downvotes | 122,141 |
On average, there are 10x as many upvotes cast as downvotes. That’s even more optimistic than math would predict (10 / 2 = 5x). That’s because we also do a few other things that help keep downvoting in check:
-
We limit total votes per day to 30 per user.
- You do not have the right to cast downvotes at all until you earn the equivalent of 10 upvotes, or 100 reputation.
The endless inflation of a system with no voting limits is something we learned early on. Instituting vote limits has many advantages besides reducing the inherent inflation. Even if you want to maliciously downvote someone out of revenge, you can only do -60 damage to that user’s reputation per day — while simultaneously reducing your own reputation by -30. And you’ll have to wait 24 hours to do it again, which is a nice de-facto timeout to potentially let cooler heads prevail.
I understand what Mr. Graham was aiming for in Hacker News. An environment where nobody has to feel the sting of a downvote is a laudable goal, and it’s certainly easy to implement. But is it real? Is it honest? The lack of a downvote removes far too much of the critical community feedback loop from the system. And in the longer term, that will do more to tear down your community than build it up.
9 Comments
Jeff, I don’t disagree with your debate about no downvotes is a lack of information. But the additional issue with Stack Overflow (and all Stack Exchange are the same really) is you can only see the net result until you have earned a substantial amount of rep.
If an answer has positive 10 net votes, what does that “10” mean? Without knowing the total upvotes and total downvotes, it could mean anything at all:
Continuing with 10 net as the example:
10 up = A good answer – information can probably be trusted
20 up and 10 down = Twice in favour as against – information is probably trust-able but use your own judgement as 10 downvotes as a 1/3 of the total voting is significant
200 up and 190 down = community is split, who knows if it’s good or bad information
Etc
I understand and agree with the choice of downvotes on questions and answers. However, in light of all this, it makes even less sense not allowing downvotes on comments. We can remove comments that are offensive, but factually wrong comments keep collecting upvotes from similarly misinformed people.
The problem with downvotes, IMO, is, that it doesn’t explain what’s wrong with the post at all. Does it contain wrong or outdated content, is it poorly researched, does it not cover the actual topic/question, is it confusing, is it a wall-of-text for a simple problem trying to explain to much and therefore overshooting the actual problem, is someone disagreeing due to a different take (pragmatic vs. originally-intended) on the same topic, does the post use poor wording (not everyone is a native English speaker), is there a personal difference with an other member who as a revenge downvotes multiple posts, …
Such downvotes occur almost on a daily basis. So how should a community improve at all if all they could do is just guessing? It is always said that quality wins over time, though in reality I tend to see that answers of high-rep users are upvoted regardless of the quality or research effort they’ve put into their answer. They are naturally given more credibility compared to a low-rep user, who just “has to earn his wings to play with the big-boys”. I have seen quite some question having had answers from low-rep users which content was correct, maybe limited down to the actual essentials, when a high-rep user came up, minutes later, produced an answer and the low-rep user shortly after received a down-vote. I can’t state whether the down-vote was produced by the high-rep user or by an other one, but this doesn’t change the actual outcome. After a couple of minutes the low-rep users post was gone leaving just the post of the high-rep user containing basically the same information as the low-rep user.
If the site aims at being as newbie-friendly as it states it wants to be, it should train high-rep users to assist low-rep users in producing more quality answers to start with and not just forcing those low-rep users out of the way. Instead of producing almost the same information in an own answer, the high-rep users could have given the low-rep users some tips on how to improve the post, what is missing in his opinion and stuff like that. This would also start some kind of mentoring or peer-review system as it is common in modern software development and raise the community aspect. Its a combined effort and not one where the strongest wins.
Any professional investor in the stock market knows that the ability to short a stock is absolutely essential for correct price discovery. The same logic applies elsewhere. Stack Exchange gets it right. Hacker News is a bubble, where if you ever have any dissenting opinions, you are soon shadowbanned, leading to bigger bubbles which get increasingly disconnected from the real world.
What Stack Exchange can optionally try is an investment-like vote accuracy rewarding system, although the current system seems to work sufficiently fine.
> The lack of a downvote removes far too much of the critical community feedback loop from the system.
…and the lack of explanation for the downvote also leaves out too much of that critical community feedback.
> I’ve been downvoted myself on Stack Overflow. And each time, it makes me pause. But that’s good! That’s necessary! You have to believe there are potential consequences for every post you make — both good and bad.
So when you were downvoted, how did you know what to change in the relevent posts – are you telepathic?
On this matter, Roman Vottner is correct – downvotes without explanations are practically useless. Why would I bother to edit my downvoted post if I don’t know why it was downvoted? How can I sensibly improve my post without that information?
As for some of the usual counter-arguments:
* possibly receiving a snarky reply which expresses one’s displeasure – downvoting users must identify themselves;
* possible retaliation against downvoters – only a moderator is allowed to see the meta-data (including the author) for each downvote and associated explanation.
In the absence of practically-useful feedback, Stack Overflow also gets it wrong…
The problem is, there are more and more down votes. At least on me. Before I didn’t receive any down votes, now it happens regularly, for no apparent reason. I didn’t change the way I write my posts, except for the fact that, having gained a bit of experience, I now post more complicated questions.
Criticize me as much as you want, call me an idiot if you feel like it, but do it to my face! People who anonymously vote me down without leaving a comment are just nasty cowards.
I am on SO for 14 years and have some 70k rep. I am infuriated every time I post a well-thought question, and people will downvote because they do not agree with the question, or think I should do something else. I welcome comments about alternatives, but when I ask for something, it is because I want to understand how to do this something – not to be told “I won’t tell you because I favor something else, take a downvote for even asking”
This is particularly visible in the Go community: ask a question that touches their elitist egos and bam – downvotes. This is by far the most toxic community I have ever seen in tech, which will push away every beginner. This is probably what that inbred community wants – but hey, there fortunately is Reddit where people will answer your question without expecting you to lay in front of them and beg.
It is a real shame the Go community does not see it as a problem.
I have had so many wonderful experiences in other places (Python, Typescript, CSS, regex, …) where the ones who want to help do help. I was happy to pay back when I could.
So no – the downvoting system is SO is terribly, horribly broken because it favors bullying.