community June 13, 2011

Optimizing For Pearls, Not Sand

In March 2010, we rebalanced our reputation system to favor answers. While we value good questions (and asking a great question is absolutely an art), we want to explicitly encourage people to provide the best possible answers. Without people interested in providing good answers, the questions are moot. We know that answers have more intrinsic…
Avatar for Jeff Atwood
Co-Founder (Former)

In March 2010, we rebalanced our reputation system to favor answers.

While we value good questions (and asking a great question is absolutely an art), we want to explicitly encourage people to provide the best possible answers. Without people interested in providing good answers, the questions are moot. We know that answers have more intrinsic value than questions, and the reputation balance should reflect that.

The question asker already enjoys a substantial benefit beyond reputation gain from upvotes on their question — namely, they get great answers to their question! Thus, the asker shouldn’t need as much reputation gain.

In November 2010, we began to actively block low quality questions, too:

We believe asking questions on our site is a privilege, not a right. If, after a few fair attempts, you haven’t been able to prove that your contributions to a particular Stack Exchange make it at least … not-worse … then we reserve the right to refuse your questions. If we don’t do our part to cull the bad questions, then we risk alienating the true experts who provide what really matters: the answers!

Last month we made voting more visible and added 10 additional “question-only” daily votes to encourage people to vote more on questions, so we can better discern their value.

Users intuit that answers are the real unit of work in any Q&A; system and tend to favor answers in their voting.

Continuing in that same vein, we have two more changes to formally announce today:

  1. We now limit users (and IP addresses) to a maximum of 6 questions per day and 50 questions per month.
  2. Downvotes on questions no longer cost the casting user 1 reputation, so they are effectively “free”.

Perhaps you’ve noticed a theme here. Incoming questions are a universal constant, all around us in countless billions. But answers — truly brilliant, amazing, correct answers — are as rare as pearls. Thus, questions are merely the sand that produces the pearl. If we have learned anything in the last three years, it is that you optimize for pearls, not sand.

Consider the question Does torture work well as an interrogation technique? on Skeptics. Is this a brilliant question? Is it even an original question? No, it’s just a mundane grain of sand question that could have been asked by anyone at any time. What makes it remarkable is the incredible answer on that question by Larian LeQuella with over 100 upvotes.

Sand, meet pearl.

That’s why we’re determined to keep question quality high, even at the cost of refusing a little sand. It’s true that you can’t have Q&A; without questions, but having the wrong sorts of questions is far more dangerous. The fastest way to kill any Q&A; site is to flood it with low-quality questions. I think Mark Trapp summed it up best in this meta answer:

To put it another way, when I go to a Stack Exchange home page, I see a list of questions. If most of those are terrible questions with little to no indication that I’d be wasting my time by reading them, the value proposition of visiting and participating is diminished: I have better things to do.

Compare that to answers on a specific question: I’ve made a conscious choice to look into what I think is an interesting question. I already made the decision that the question is worth my time. If I find the answers to be useless, I have a few different options, as an interested party, to register my displeasure, including writing my own answer. Being able to write your own answer is key: if your answer is good enough, it’ll rise above the junk answers and everyone will be better off for it.

There is no such action for question lists. I can’t say “these questions suck, show me this question I just thought up instead”: that’d be silly. So, it’s imperative the question list have a high signal-to-noise ratio, and removing the penalty for those users who do take the time to read a question and later find it to be useless so they can down-vote is conducive to that.

Fundamentally, answers can be filtered in ways that questions cannot. While there is a tension between having “enough” questions and a bunch of amazing, highly skilled answerers twiddling their thumbs waiting around for something to do, in the long run we’d much rather err on the side of having interesting and on-topic questions for these folks to sink their teeth into.

We feel that the world is awash in questions, but not answers. Answers are the real unit of work in any Q&A; system. Therefore, the only logical thing to do is to maximize the happiness and enjoyment of answerers. If this means aggressively downvoting or closing unworthy and uninteresting questions, so be it. Without a community of people willing to answer questions, it really doesn’t matter if there are questions at all, does it?

Podcast logo The Stack Overflow Podcast is a weekly conversation about working in software development, learning to code, and the art and culture of computer programming.

Related

community June 17, 2020

Saying thanks: testing a new Reactions feature

We’ve heard from our users that the inability to say “thank you” is frustrating—especially for new users who don’t have enough reputation to upvote or comment. Even when users gain these privileges, they still want to say “thanks.”
Avatar for Lisa Park
Product Designer
community February 25, 2020

Sharing our first quarter 2020 community roadmap

Editor’s note: You can discuss this post on Meta here: https://meta.stackexchange.com/questions/344178/the-q1-2020-community-roadmap-is-on-the-blog/344209 As we work to improve our relationship and communications with our users, we want to share with you our community roadmap each quarter, so once every three months. Even though we are halfway through this quarter, we wanted to share with you what we…
code-for-a-living May 15, 2020

The Overflow #21: The way forward

May 2020 Welcome to ISSUE #21 of The Overflow! Cheers! Have a beverage with our newsletter by developers, for developers, written and curated by the Stack Overflow team and Cassidy Williams at Netlify. Behold! Discussing privacy concerns in contract tracing apps, a lonely sample size of one, and email sender reputation. From the blog A…
promotion March 19, 2020

Coming together as a community to connect

Stack Overflow is a community at our center. And communities come together in times of need. Our normal work routines have been disrupted and may continue to change for a while. We’re all still defining the new “business as usual.” We’re all trying to figure out how to maintain team collaboration when lines of communication…
Avatar for Lori Colston
Sr. Product Marketing Manager