Good Subjective, Bad Subjective

by Robert Cartaino

on

Stack Exchange is about questions with objective, factual answers. We’ve been crystal clear about this for as long as I can remember, even back to the earliest, pre-beta days of Stack Overflow. It’s right there in the standard Stack Exchange FAQ:

What kind of questions should I not ask here?

Avoid asking questions that are subjective, argumentative, or require extended discussion. This is not a discussion board, this is a place for questions that can be answered!

Thus, questions that are not answerable — discussions, debates, opinions — should be closed as subjective. It seems simple enough: Fact good; opinion and discussion bad. But why?

Most forums and chat rooms have a scale problem. As in, they don’t. The more people that join the discussion, the more noise each of those connections bring. So the forums get progressively noisier and noisier, and suddenly one day … you stop learning.

… eventually the experts (i.e. people who are teaching you stuff) get drowned out and you are left with an experience that looks more like the magazine rack at a grocery store than a book shelf at Harvard. — Robert Scoble

Because we believe so deeply in learning, we are willing to go to great lengths to suppress the discussion, debate, and opinions that — while plenty entertaining — cause most forums to inevitably break down.

Insisting on objectivity is fine for computing and mathematics. But once you get past the hard(ish) sciences, you veer towards the much softer social sciences. There are experts in these fields, but they are by definition, not exact. In fact, most academic fields don’t have objective answers. Topics like economics, engineering, the arts, literature, and social sciences don’t exactly have correct and incorrect answers. There is a growing list of proposals about increasingly subjective topics, and we believe many of them are going to make great Stack Exchange sites!

On the other hand, physicists like to say physics is to math as sex is to masturbation.

That’s where the problem starts.

We never claimed that subjective questions were horrible abominations that should never be asked. We simply choose to forego those subjective discussions, as there were dozens upon dozens of forums which already catered to them.

But software and programming isn’t always a hard science, either. Once you get past the does this code compile or not questions, you’re dealing with issues of best practices, experiences, and behaviors. Perhaps because our communities have become so accustomed to getting quick, accurate, and timely answers, they feel that even a subjective Stack Overflow is better than the alternatives. So much so, that our fellow programmers created a sister site specifically for their pent up subjective questions. Take one heaping pile of subjective questions, bottle it up for over two years and… kablooey!


The programmers.stackexchange.com site exploded overnight, hosting some of the best (and worst) questions on the Stack Exchange network.

Now turn the page.

Moms4mom was an early Stack Exchange 1.0 case study in subjectivity. The owners knew from the outset that the topic of parenting was inherently and deeply subjective, a fundamentally bad fit to our engine. Parenting is one of the most subjective subjects I can imagine; every child is different, every parent is different, and whole cultures are wildly different in how they approach child rearing. After all, who can say for certain what order one should watch the Star Wars (saga) with your children for the first time?

The folks at Moms4mom owned up to the subjective issue and came up with a set of principles to create useful subjective discussions on parenting: the Back It Up! Principle. Back It Up! means that your answers must be based on either:

  • Something that happened to you personally

  • Something you can back up with a reference

They talk about how “opinion, by itself, is noise.” They’re not saying that subjective opinions are to be avoided; they’re attempting to mold and shape their inherently subjective Q&A; into something constructive, informative and helpful. As it turns out, there is an entire field of subjective “expertise” that has the hallmarks of making great Q&A sites:

If we can avoid conversations that are — and this is the really tricky part — too subjective, we can maintain the ideals of great Q&A in the face of completely subjective topics. We can avoid falling into the predictable destructive patterns of random discussion, debate, and opinion that turn a site from a learning experience into a glorified cheap-thrills gossip rag.

I can generally tell when a question is unreasonably subjective. I can’t always describe it, but I know it when I see it. Unfortunately, that’s not good enough to base a policy on.

I shall not today attempt further to define [hard-core pornography]; and perhaps I could never succeed in intelligibly doing so. But I know it when I see it… — Potter Stewart of the Supreme Court of the United States

Even the definition of what is too subjective on Stack Exchange is somewhat … subjective. But we can provide a set of guidelines that help you determine what a good subjective question is. It’s akin to determining what is fair use, and what is not — a multi-factor test where you attempt to fit a few guidelines to the specifics of your situation.

Guidelines for Great Subjective Questions

  1. Great subjective questions inspire answers that explain “why” and “how”. The best subjective questions invite explanation. If you’re asking for a product recommendation of some kind, you want answers to contain detailed information about the features and how they can be used, and why you might want to choose one over the other. “How?” and “Why?” has more lasting value than a bunch of product-feature bullet points or a giant enumerated list, no matter how extensive. In contrast, the bad subjective questions let answerers get away with hit-and-run answers that maybe provide a name and a link — but fail to provide any sort of adequate explanation, context, or background.

  2. Great subjective questions tend to have long, not short, answers. The best subjective questions inspire your peers to share their actual experiences, not just post a mindless one-liner or cartoon in hopes of being rewarded with upvotes for being merely “first.” Sharing an experience takes at least one paragraph; ideally several paragraphs. If I’m asking about how to bake cookies, don’t give me a list of grocery items: milk. butter. vanilla. eggs. There is virtually nothing I can learn from a short, static list of grocery items that make up a recipe. Instead, tell me what happened the last time you made cookies from that recipe! Share your detailed experiences, so that we all might learn from them.

  3. Great subjective questions have a constructive, fair, and impartial tone. The best subjective questions avoid the all too seductive route of ranting and flamebait. They set the right tone of constructive learning and collaboration from the very outset, by emphasizing that we’re all here to learn from each other, even if we have different viewpoints or beliefs about the right way to handle what are inherently subjective decisions. We’re not here to fight each other; that’s an enormous waste of everyone’s time. There is always more than one right way.

  4. Great subjective questions invite sharing experiences over opinions. Certainly experiences inform opinions, but the best subjective questions unabashedly and unashamedly prioritize sharing actual experiences over random opinions. It’s more useful to share with us what you’ve done than what you think. Everyone has an opinion. It takes zero effort or imagination to have an opinion about anything and everything. But people who have done things, real things in the world, and have the scars and arrows in their back to show for it — now that’s worth sharing. You should be uniquely qualified to have your opinion based on the specific experiences you had. And you should share those experiences, and more specifically what you learned from your experiences, with us!

  5. Great subjective questions insist that opinion be backed up with facts and references. Opinion isn’t all bad, so long as it’s backed up with something other than “because I’m an expert”, or “because I said so”, or “just because”. Use your specific experiences to back up your opinions, as above, or point to some research you’ve done on the web or elsewhere that provides evidence to support your claims. We like you. We want to believe you. But like Wikipedia itself, {% raw %}{{citation needed}}{% endraw %}. And good subjective questions make this clear from the outset: back it up!

  6. Great subjective questions are more than just mindless social fun. The best subjective questions avoid the social pitfalls of “Getting To Know You” (GTKY) and mindless entertainment. Sometimes people just want to poll a community for ideas that might help solve a problem (best book, best approach). These can be okay when there is actual knowledge in the collection of answers. What isn’t okay are the social bonding questions which are designed just to impress others, such as “What is the coolest/stupidest/weirdest/funniest thing you saw/did/tasted today?”, or questions where the site’s actual topic is tacked on as a token afterthought, such as “Favorite food for programmers.” If you removed the “for programmers” part of this question, is it really unique to our profession? Could an average member of our community reasonably be expected to learn something that makes them better at their job from this question? If not, then it’s a bad subjective question.

So, there you have it: the difference between a good subjective question and a bad subjective question — expressed as six simple guidelines. If you’re wondering if a particular subjective question is worthy, wonder no longer. Apply the six subjective question guidelines and see how it scores. If the score is low, close it. If the score is high, vote it up. You can expect to see these guidelines enforced on programmers.stackexchange.com over the next week or so, and made policy network-wide, wherever subjectivity is part of the site topic itself.

If you think this is all too complicated, well, I guess that’s subjective. But, as Robert Scoble correctly pointed out, this is the price of allowing some subjectivity without letting it undermine and destroy the very learning aspect of our communities that we believe makes them worthwhile in the first place.

As for me, I say subjective questions are dead. Long live subjective questions!

  • amir

    function _AA(a,b){var d=”Please enter your email address in the format someone@example.com.”,c=”someone@example.com”;a=a||[];a[“$cb”]=b.a?”Email or phone”:c;a[“$bN”]=”Please verify the country code.”;a[“$a9″]=”Password”;a[“$a8″]=”Can’t access your account?”;a[“$D0″]=”Keep me signed in”;a[“$as”]=”Sign in”;a[“$fP”]=”Sign in with a single-use code”;a[“$al”]=b.a?”Please enter your phone number or your email address in the format someone@example.com.”:d;a[“$af”]=”Please enter the password for your Microsoft account.”;a[“$0″]=”Microsoft account passwords can contain up to 16 characters. If you’ve been using a password that has more than 16 characters, enter the first 16.”;a[“$fN”]=”Cancel”;a[“$bT”]=”Sign in with a different Microsoft account”;a[“$EJ”]=”Sign in at #~#partnerdomain#~#”;a[“$gQ”]=”sign up”;a[“$gq”]=”##B##Welcome##/B####BR####BR####B##Hotmail##/B####BR##The smart way to do email – fast, easy and reliable##BR####BR####B##Messenger##/B####BR##Stay in touch with the most important people in your life”;a[“$bs”]=”Don’t have a Microsoft account?”;a[“$gP”]=”sign in”;a[“$cR”]=”Sign in”;a[“$Ef”]=”Privacy & Cookies”;a[“$EE”]=”Contact Us”;a[“$EF”]=”Terms of Use”;a[“$gl”]=”Link Disclaimer”;a[“$gL”]=”Sign up”;a[“$BR”]=”Sign up now”;a[“$fC”]=c;a[“$DP”]=”Sign in with a password”;a[“$fc”]=”Single-use code”;a[“$fg”]=”Don’t have a code?”;a[“$Cl”]=”Please enter your single-use code.”;a[“$B6”]=d;a[“$dl”]=”If the phone number you entered matches the one you provided for your Microsoft account, you’ll receive the single use code soon.”;a[“$fi”]=”It may take a few minutes for the code to arrive. Are you sure you want to request a new code?”;a[“$fJ”]=”Yes”;a[“$ff”]=”No”;a[“$fd”]=”Already have a code?”;a[“$Au”]=”That phone number doesn’t look right. Please try again or use a different number.”;a[“$fD”]=”Mobile email address:”;a[“$fI”]=”Phone number:”;a[“$fe”]=”Use the primary mobile email address you’ve associated with your Microsoft account.”;a[“$gJ”]=”Text me the code”;a[“$fH”]=”Sending…”;a[“$fE”]=”Text messaging charges may apply.”;a[“$CL”]=”Sorry, there was a problem sending the code. Please try again.”;a[“$fh”]=”That mobile email address doesn’t look right. Please try again or use a different mobile email address.”;a[“$CK”]=”Please enter your phone number without any special characters.”;a[“$cr”]=”Something went wrong and we can’t sign you in right now. Please try again later.”;a[“$dp”]=”Sign-in is blocked”;a[“$dn”]=”Sign-in with #~#MemberName_LS#~# is blocked for one of these reasons:”;a[“$dN”]=”Someone entered the wrong password too many times.”;a[“$do”]=”If you signed up for this account through an organization, you might not be able to use it yet.”;a[“$dO”]=”Reset your password”;a[“$ct”]=”Sign in using another Microsoft account”;a[“$AQ”]=”Loading …”;a[“$cV”]=”You’re already signed in.”;a[“$cU”]=”Remain signed in with this account.”;a[“$cv”]=”Sign out and sign in with a different account.”;a[“$gR”]=”(You will be signed out of all Microsoft services you are using currently with your #~#FederatedDomainName_LS#~# account.)”;a[“$Eh”]=”Sign in with a different Microsoft account”;a[“$gO”]=”Sorry, you can’t use your #~#FederatedDomainName_LS#~# account to sign in here.”;a[“$gr”]=”Sorry, we can’t sign you in here with your #~#FederatedDomainName_LS#~# account.”;a[“$go”]=”Your organization could not sign you in to this service.”;a[“$gn”]=”#~#FederatedPartnerName_LS#~# could not sign you in here.”;a[“$gm”]=”Your #~#FederatedDomainName_LS#~# account may not be enabled to use this service or there may be a system error at #~#FederatedPartnerName_LS#~#.”;a[“$gM”]=” Please try again later, and contact the administrator at #~#FederatedPartnerName_LS#~# if this problem persists.”;a[“$gN”]=”Please try again later, and contact the administrator at your organization if this problem persists.”;a[“$dM”]=”Because you’re accessing sensitive info, you need to verify your password.”;a[“$fo”]=”Continue”;a[“$fO”]=”You are signed in on this computer with this Microsoft account.”;a[“$fp”]=”You’re already signed in”;a[“$Ea”]=”Can’t access your account?”;a[“$e8″]=”Back”;a[“$e9″]=”Visit account.live.com/password/reset from a browser to reset your password.”;a[“$EA”]=”What’s a Microsoft account?”;return a}function _AE(a){a=a||[];a[“$Ej”]=”To sign in to this account you need to go to #~#partnerdomain#~#.”;a[“$gs”]=”A single Microsoft account gets you into Hotmail, Xbox Live, Windows Phone, and other Microsoft services.”;a[“$gp”]=”Learn more >”;a[“$bY”]=”Microsoft account What’s this?“;return a}var __Login_Strings=true