Welcome Wagon: Classifying Comments on Stack Overflow
Last month, Joe wrote about the Welcome Wagon work that we are doing to make Stack Overflow more welcoming and inclusive. Our current work involves projects across domains from asking questions to framing community standards and more; one project we have been working on is understanding how comments are used and misused on Stack Overflow.
We are a data engineer (Jason) and data scientist (Julia). As folks who code for a living and use Stack Overflow, as well as work here, we have certainly experienced and witnessed unwelcoming behavior in Stack Overflow comments first hand, whether through condescension, snark, or sarcasm. Our goal with this specific project is to understanding these issues so that we can start to address them. This blog post outlines our initial findings, what we could learn with more data, and next steps.
Classifying comments
I (Jason) wrote The Stack Overflow Comment Evaluator 5000™, a simple application that presents you with a comment thread from a post on Stack Overflow and asks you to rate each comment in the thread as Fine, Unwelcoming, or Abusive.
Comments on Stack Overflow can already be flagged for being rude or abusive, but this flag is typically used only for the most egregious and toxic comments, which are thankfully rare. We’re looking here to characterize comments that are unwelcoming in a way that isn’t flagrant hate or abuse but would still make you think twice about participating in our community. Things we thought might fall into this category would include condescension, snark, sarcasm, and the like.
We estimated how many comments we could rate, given the number of folks we have internally and the time we would be asking of them. Then we loaded up the right number of comment threads into the application and invited all of our community managers, designers, developers, executives, site reliability engineers, and product managers to spend an hour rating comments. We had 57 participants who made 13,742 ratings on 3,992 comments.
Prevalence of comment categories
If we take a majority vote on the rating of each comment (with ties going to the worse rating) comments on Stack Overflow break down like so…
| Rating | % of comments |
|---|---|
| Fine | 92.3% |
| Unwelcoming | 7.4% |
| Abusive | 0.3% |
According to those of us deeply involved here and familiar with Stack Overflow, about 7% of comments on Stack Overflow are unwelcoming. What did some unwelcoming comments look like? These combine elements of real comments to show typical examples.
- “This is becoming a waste of my time and you won’t listen to my advice. What are the supposed benefits of making it so much more complex?”
- “Step 1. Do not clutter the namespace. Then get back to us.”
- “The code you posted cannot yield this result. Please post the real code if you hope to get any help.”
- “This error is self explanatory. You need to check…”
- “I have already told how you can… If you can’t make it work, you are doing something wrong.”
This stuff isn’t profane, hate, or outright abuse, but it’s certainly unwelcoming. Looking at majority voting is one approach, but the experience of being not welcomed is not a majority vote kind of thing; it’s deeply personal. What if we looked at the distributions of the ratings by individual?
Among the 57 individuals who participated, the median result for comments that are neutral or fine was 93.2% and the median result for unwelcoming comments was 6.5%. We can see from this graph that there is considerable variety in people’s experience of the site with respect to the comments they saw; the histograms have a broad shape. Take a look at just the distribution for the unwelcoming ratings. Four of us didn’t find anything unwelcoming, and three of us thought that in excess of 1 in 5 comments were unwelcoming. This speaks to the variability of experience; for example, what an experienced, professional developer from a privileged background finds unwelcoming may not be the same as for a more junior, or less privileged developer.
What can we learn from our initial rating task?
In this first attempt at rating comments, we have been able to measure the prevalence of unwelcoming comments, as perceived by experienced Stack Overflow community members/employees, as well as how much that varies. This first group of raters includes people from underrepresented groups in tech such as women, people of color, gay folks, and trans folks. We do not see evidence that having an identity in one or more of these groups led to an individual labeling comments as unwelcoming at a higher rate, at least not among the small group of ~60 participants in our initial task. It is possible that we would see a statistical difference with a larger sample size of raters.
There are enough comments in this first rating task for us to assess whether unwelcoming comments are more prevalent on questions or answers on Stack Overflow. Let’s compare all the comments from the month the rated comments were drawn from, along with the rated comments.
The proportions of comments that were classified as fine by the raters are consistent on questions and answers with the population of comments overall. We see a shift for the comments classified as unwelcoming or abusive, however; those comments are more prevalent on questions than answers, with a relative percentage shift of almost 10%. When we think through what is contributing to unwelcoming comments in our community and what to do next, this is important to keep in mind. Indeed, the dynamics at play here are part of why we are revamping our workflow for new askers.
We also would like to look at how much this sample of raters agree with each other with some measure of inter-rater reliability. You might guess from the histogram of rating distributions that the inter-rater reliability is not going to be high; we labeled comments as unwelcoming at different rates. Given the kind of rating data we have (not every person saw every comment) we can use Krippendorff’s alpha to measure reliability; this is a measure that ranges from zero (nobody agrees) to one (perfect agreement). For comments that were rated by at least three people, Krippendorff’s alpha for this initial dataset is 0.39. This would be too low for qualitative research in an academic study like those done in the social sciences. If you have been around Stack Overflow a long time, you might be interested to know that this is much better than the reliability for the comment classification project done using Amazon Mechanical Turk about 5 years ago.
What does a reliability measure like this mean? It reflects the real diversity in how we all experience the Stack Overflow community, based on our experiences and personalities. People who work at Stack Overflow agree more about what is unwelcoming than the Mechanical Turk workers did, but even the 60 or so of us do not agree enough that we could code comments reliably enough to be used in academic work. What this diversity of thought does not mean, to be clear, is that it is hopeless to address issues in how comments are being used for our community as a whole. We can take what we are learning here and move forward.
Impact of unwelcoming comments
So about 7% of comments on Stack Overflow are unwelcoming, depending on who you ask. What does that mean? First of all, this is not good enough for us. Stack Overflow is a place for developers to help each other; our goal is to be a professional space that makes our industry, our profession, and yes, the internet a better place. Everyone who codes should feel welcome to participate here.
Second of all, a prevalence between 5% and 10% can have a big impact on a community. Let’s sketch out a back-of-the-napkin estimate. If a typical developer visits Stack Overflow once or twice a week to solve a problem, the question they visit has an answer, and each post (question and answer) has two comments (keep in mind that comments are more visible to visitors than answers), we would conservatively estimate that a developer visiting Stack Overflow would see 1 to 3 condescending, unwelcoming comments every single month of their coding lives. Will one unwelcoming comment a month drive everyone away? Clearly not, as Stack Overflow still works for many. But it will convince some that it’s not worth it to contribute here, and the next month’s comment will convince a few more, and so on. And this only considers the readers of these comments; those who the comments are directed at will naturally feel more dramatic effects.
So, where does that bring us now?
- This is the first step for us in addressing how comments are used at Stack Overflow, and we are exploring options for moving forward. We believe strongly both that human moderators are key and that human-in-the-loop machine learning can offer us powerful tooling.
- It takes care to be understood well online, and people have different reactions to the same words. Remember that many more people than the post owner may read your comments, so write for posterity and make a conscious effort. When you see unwelcoming behavior, please flag it.
- We at Stack Overflow want to more clearly frame our expectations around our community standards. Watch for updates about the evolution of our “Be Nice” policy into a fully articulated code of conduct.
- We will be fielding this comment classification task more broadly soon, in order to learn more about how our community understands interaction via comments. Look for further work from us on this in the near future.
87 Comments
“a developer visiting Stack Overflow would see 1 to 3 condescending, unwelcoming comments every single month of their coding lives.”
So what? That’s what the average Twitter/Facbook/whatever user sees every *hour*. Do they quit?
Anecdotally, I know lots of people have higher expectations out of a site intended for professional use, as opposed to a site like Facebook which they fully expect is a non-professional environment.
If StackExchange intends to be a platform for learning, it needs to pay attention to well-established principles of pedagogical learning. Different people learn in different ways, and no one approach can work for everyone. If you want your contributions to be taken seriously as material for learning, you need to up your game and talk like a teacher, not like an annoyed roommate complaining about whose turn it is to do the dishes.
But Stack Exchange is actively and massively recruiting regular people like you and me to produce this learning material. We are mostly enthusiast or professional developers, not pedagogues, and Stack Exchange knows this. I didn’t originally sign up to be a teacher, I’d be surprised if you did. I think there’s a significant difference between a learning platform and a Q&A community. I would only expect professional teachers on the former and I thought Stack Exchange is the latter.
There is also a network effect for social media sites that isn’t present here. There are lots of spaces to ask for tech advice besides here; there’s only one Facebook.
What I hear you asking is “Why do we have to change and put more effort into being a welcoming community for these groups of people? Can’t they just put up with it like I can?”
I think nobody should have to put up with it and to tell a prospective user “StackOverflow is the best place to get coding help. You should join us even though you’ll have to deal with feeling unwelcomed, which is phenomenon we know exists from data, user stories, and people’s perceptions of us. We just need you to let it slide too because we have more important things to worry about than your feelings.” is highly unrealistic and problematic.
You and me can obviously handle unwanted and abusive comments, but others can’t. We’re lucky to be able to handle this (or lucky that we’ve had the time and place to practice handling abusive and unwanted comments). I think we shouldn’t miss out on huge swaths of potential users (all people!) by setting the bar for entry to a height that feels comfortable to us rather than what is comfortable for others.
I can’t believe this poll at all. Most comments are asinine. I rarely use it unless I have to a point of last resort.
Yes they do quit! People quit Twitter all the time because of this exact reason. I rarely use Twitter because I don’t want to waste my time getting into an argument, or waste my energy ignoring inflamatory comments. I use StackOverflow because it helps me be productive and focussed.
Personally I have never seen a negative comment on SO, presumably because the moderators are doing such a great job. It gives me a good feeling and that’s why I keep coming back.
I LOVE the culture of StackOverflow.
Firstly the “unwelcoming” comments aren’t unwelcoming. They’re all valid criticisms, I imagine. You haven’t given us the context in which they’re said, which’d be extremely helpful here.
The abusive comments make up 1 in 250. Which is tiny. That’s such a small amount that I find it hard to believe you’re even worrying about it. It’s -genuinely- impressive that it’s so much lower than a great deal of other websites that seek to achieve the same thing as this one. You should be extremely proud of this. It’s never going to be perfect.
I love this site and I love what it does. It’s genuinely a great platform for doing what it does. But you should work on improving the parts of it that are lacking (like chat!) before you try to handle such a minutely small problem.
The unwelcoming comments are valid criticisms, that can be expressed in a much more welcoming manner. The problem is large enough that StackOverflow is stereotyped and memed as being unwelcoming. That warrants attention, in my book.
“My anecdotal evidence outweighs the obvious benefits of these changes, and therefore you should run your company to accommodate my needs.”
I think you’re coming at this from the wrong angle. You’re thinking “let’s calculate a metric, and if that metric is below X, we don’t have a problem. Everyone who thinks there is a significant problem is wrong.”
Whereas I think the reality is: a huge number of people think there is a problem. Women in particular, who don’t contribute because they think it’s an unwelcoming place. So I’d reframe it as “let’s calculate a metric X, and now we know that at level X, we have a problem. We don’t yet know at what level we don’t have a problem anymore, but it must be less than X’.
The problem with your “let’s calculate a metric X, and now we know that at level X, we have a problem. We don’t yet know at what level we don’t have a problem anymore, but it must be less than X’ is that the calculated metric X probably is not the reason for the problems. So even if we get less than X, the problem will remain.
This blog post states a metric of 7.4% “unwelcoming” comments and suspects this as source of the problem. But the given examples are not “unwelcoming” as they are because they all have a history behind. Not a single example is a first comment after a user posted a question. So the metric of 7.4% “unwelcoming” comments probably are not the source of the problem that users feeling unwelcoming here.
Much more is one of the sources of the problem the fact that questions about programming are mainly connected to the knowledge level of the asker. SO claims to be a Q/A platform for professional and enthusiast programmers and for questions about programming that are tightly focused on a specific problem. So the questions must exactly tell that specific problem. This is often nothing what a beginner can do. So in my opinion SO is not a Q/A platform for beginners in programming. So this user group will always feel not welcoming here simply because it is not. Maybe SO should providing a special beginners Q/A portal additionally?
Another source of the problem is the down voting of questions without any reason telling. New users ask a question which does not fit here or is not well describing the problem. Thus the question gets anonymous down voted multiple times. One user then tells the asker what is the reason for the down votes. What do you think are the feelings of the asker about that user? Well now the comments war begins… Maybe SO should first rethink that down voting of questions system?
… This could be continued….
But I think you got my point. Even if we get less than 7.4% of such “unwelcoming” comments as in the given examples, the problem that users feeling unwelcoming here will remain.
I am a huge fan of automatically removing unwanted comments. I did so for several years. That said, I’m disappointed in how this is playing out here. I’m disappointed on both a community level and a technical level.
On the community level, I am very disappointed that 57 Stack Exchange employees were able to evaluate bad comments, determine they were bad enough to put in the hall of shame post here, and then do nothing about them. It took users less than 15 minutes to find those comments on Stack Overflow and identify the “rude” users. Users who are rude because they asked why a certain tag was on a question. Did none of your 57 users have a diamond where you could remove the comment from the site? Even if that’s the case, all of you have the option to flag a comment. Even that wasn’t done.
On a technical level, you evaluated less than 4000 comments. That is a few hours worth of comments on a single week day. (source: http://data.stackexchange.com/stackoverflow/query/872382) Is that really representative? How did you determine which comments to use in your evaluation?
Hi Andy,
How do I upvote your post?
JonH
With such a low percentage this shows exactly why there is NO problem. I still believe the efforts of stackoverflow/exchange are such a hostile place is a bunch of bs. The site IS welcoming and it IS helpful, if you only put in the effort to learn how to use it. Thanks to the community for making it such a great place by eliminating the garbage users.
When I read Joe’s post, I was expecting to eventually see raw data published, not just summary stats. Will it be?
There are lots of questions that the limited summary stats you’ve given us leave us unable to explore – like the extent to which different staff agreed on their classifications of WHICH comments were unwelcoming, or whether, in the case of staff members who flagged some critical comments as unwelcoming, there was any other comment that conveyed the same criticism with a different tone that they DIDN’T think was unwelcoming. Without being able to tap into data like that, we can’t tell whether the supposed problem is tractable at all (or whether it’s an unavoidable consequence of people having opposite standards of politeness from each other), nor can we figure out what tweaks to comment tone would solve it (at least from the perspective of the people who participated in this survey).
Also – you mention all sorts of characteristics of the people you got to participate in this survey, from the color of their skin to the sort of folks to like to have sex with, but you haven’t mentioned anything about what *culture* they’re from. Was this a survey in which the majority of participants were Americans, and which thus primarily reflected American notions of politeness, or did you have an international cohort?
“Was this a survey in which the majority of participants were Americans, and which thus primarily reflected American notions of politeness, or did you have an international cohort?”
Without more data I am led to assume that this is the main reason for all the effort put into this topic. SO is an US company for mainly US people having US values (probably not representative for the whole country, though). As an European I don’t think the values derived from this culture are shared by everybody, everywhere.
Maybe my personal bias is showing, but these 5 example “unwelcoming” comments don’t look like that to me at all. I’ve definitely gotten way harsher ones on my posts in the past and at no point did I feel them to be unwelcoming whatsoever.
And leaving these up so readers can identify the users leaving them, or even completly citing them so they can be easily found in SEDE is a privacy violation in my opinion and nobody deserves to be put in a hall of shame like that, especially not on stackexchange where the standing policy has been to allow people to correct their misgivings in private. Disappointing.
As it was stated above, those examples of unwelcoming comments above weren’t actual comments pulled from SO; they were pieced together from little bits and pieces of unwelcome comments. You can’t identify the writers of the original comments, as they weren’t written by a user. There is no privacy violation. No one has been put in a hall of shame.
The example comments above are just that – examples – meant to give a general idea of what is meant by ‘unwelcome comments’.
Think of the comments that inspired these as (bad) boilerplate human code, the unwelcoming bits as relevant snippets pulled from the overall boilerplate code, and the above examples as multiple snippets compiled into ‘user language’ that you should NOT execute for the good of the overall end program.
>Maybe my personal bias is showing
Yes, I think so. I feel similarly, but I’ve had enough conversations with other people who don’t contribute to SO to know that other people hear these comments very differently. All of them contain some kind of unnecessary snarky tone, although in number 4 (“The error is self-explanatory. You need to check…”) it’s very subtle and probably unintentional.
“So about 7% of comments on Stack Overflow are unwelcoming, depending on who you ask. What does that mean? First of all, this is not good enough for us.”
What would be an acceptable rate? 5%? 1%? 0.1%? 0.01%? Why?
“This stuff isn’t profane, hate, or outright abuse, but it’s certainly unwelcoming.”
The problem is it’s *not* “certainly” unwelcoming. I am on the welcoming side of the recent debate, and think people should be nicer and more welcoming to people in comments, etc. However, I would not consider:
“Also, any time you have enumerated columns, you can be sure that something’s gone very, very wrong with your design. That said, you’re probably after LEAST(). But don’t do that. Fix your design.”
To be unwelcoming. *Maybe* it is a little abrupt, but it really depends on context. Some people are very matter-of-fact and you can tell that from how they write their questions… such a person would welcome and appreciate a comment written like that above.
What’s considered “welcoming” is very different culture to culture, and I think the Stack Overflow staff needs to be VERY cognizant of that while addressing welcoming problems on a global scale. In some countries smiling at a stranger and saying hello and asking them how they are is considered rude/invasive and makes people uncomfortable, but in the US it is polite.
I would like to see the next step be for you to separate out the batch of comments rated by everyone you asked, and filter them by country of origin. If you don’t have a big enough sample size, talk to some of the users who said they were willing to provide feedback, and then have them rate these comments using the same application that you built, and, very importantly, note their country of origin. Then post a report on your findings, so you can see how people in the US compare on what they find welcoming vs people in Russia, or people in the UK, or people in Australia, or people in Brazil, or people in Egypt, etc.
Only then can you really consider this a holistic approach and not deal with accusations of framing problems.
Otherwise, I think you probably ought to be very candid about this being an effort to push a *US-centric* policy with regard to conduct and culture.
Those “unwelcoming” comments all have one thing in common: exasperation. The commenter is frustrated that the questioner isn’t making progress the way they’d like. The other thing they have in common is that they all contain actual advice on how to improve either the question, or how to get to an answer (although in the case of the “and this is tagged JavaScript why” case, very obliquely so). In other words, these are comments from people who *care*: they don’t want you to just go take a hike and never see you again, they actually want to see your question get an answer.
In other words, one person’s “unwelcoming” is another person’s passionate attempt at providing help. That should also help shed some light on why the people who are providing that help get frustrated when they’re told they’re Doing it Wrong, directly or indirectly: it’s like someone telling you that what you’re doing now is great, but if you could please do it without the passion you’re putting into it now that would be even better. Doesn’t sound like an attractive prospect, does it?
That’s not to say these comments couldn’t or shouldn’t be reformulated, by the way. Whether it’s possible to bridge the gap between input and desired output (and if so, if this can be done economically) is a thing I’ll gladly leave to more dedicated minds.
Amendment: the examples have been swapped out since I made my post (as apparently they were a little too specific and traceable to individuals). Unfortunately, this undermines my point since the current examples no longer contains specific advice on how to improve the question (at least not obviously so). You’ll just have to take it from me that my post made more sense when I made it.
Context is almost everything here. I can’t help but notice that in the sample of 5 “unwelcoming” comments, certainly 3 and likely 4 out of the 5 comments show frustration from the commentor in that they’re not getting through to the questioner. Look at all the key phrases:
“…You won’t listen to my advice…”
“Also…” (at the start of a comment, implying there was at least one other comment before it)
“For the last time…”
“…I have already told you…”
None of those strike me as the first comment you will see on a question. What was the tone of the comments that came before?
While it’s laudable to try to make the site more welcoming, is that the right conclusion to draw from such comments? Are these comments just the culmination of a conversation that went downhill? Should the focus be on whether the question is unwelcoming, or should it be on training both the learners and the teachers to know when to walk away from a deteriorating comment thread?
That was my thought as well; I wonder what the sequence number of the comments were that were judged unwelcoming? I’d expect there to be some correlation between the commenters’ attitudes and how many comments had been exchanged.
This is a good observation. It’s even more important if we take it in context.
Stack Overflow is attracting a lot of students, with homework questions. That’s not in itself a problem, of course. However, we know that quite a few of those people will never become actual practicing programmers, and a major factor in that drop-out is that some people do not have the talent to learn programming.
Now these “unwelcoming” comments seem to match exactly that demographic, of students that will never become professional developers. Should Stack Overflow care that these people do not return? No – it’s not that they find Stack Overflow hostile, it’s that they no longer need SO in their career.
Reading your proposed code of conduct, somebody posting all five examples, say one a week, would be facing suspension as a repeated offender.
Based on those examples, how many accounts do you expect to have to suspend after say… a month of the new code of conduct taking effect?
What percentage of those accounts do you expect to come to terms with your guidelines and improve and what percentage to you expect to leave or lurk instead?
I don’t understand how some of those example comments can be considered “unwelcoming”. I’m all for improving the way people talk to each other here, but these examples are surprising. The third one for example is (to me anyway) a useful comment and even includes the word “please”!
The impact assessment makes an assumption that the unwelcoming comments are evenly distributed between various questions and answers. That assumption is unwarranted or at least unproven. From the examples given, the comments could be reactions to low-quality questions which in turn are not helpful to anyone but the actual asker. This would limit the impact of those comments on the searchers.
To further quantify the impact of the unwelcoming comments it would be useful to link the incidence of those comments with number of views and score for the questions and answers the comments occur on. If most unwelcoming were attached to low view count questions, their impact will be further mitigated.
I’d also like to join to the other people in the request to publish the raw data. As SO’s very existence proves, more eyes are better than fewer.
Beauty is in the eye of the beholder.
I have been accused of making unwelcoming and/or condescending remarks in the past on other forums, though most other forum members there recognize them as being more typical “Doug” comments which are a combination of inability to discern context when none is provided (i.e., unwillingness to make assumptions that may be invalid when details are not stated) and the clinical nature of my responses (no, I am not warm and fuzzy).
For example, on a tax forum, a tax professional asked if a particular individual could be claimed as a dependent. I responded something like “Yes, if the five dependency tests are met.” (I actually enumerated these tests and provided a brief explanation of each). This was viewed by some as being somewhat rude and flippant.
How else can you answer a question that has no details? Should I ignore it? Should I ask for the details? Should I condemn the person for asking a rather vacuous question? This was a professional forum. Yes, I could try to guess which of the dependency tests he was asking about, but then, I could be answering the wrong question or insulting his knowledge. What is the purpose of answering the wrong question when there are at most five categories to discuss?
Rather than ask for details and then answer it, I took the time to answer regardless of which test was under scrutiny. By doing so, I expected the post to help or reinforce any others who might be missing a different piece of the puzzle. I responded assuming that the tax professional could evaluate the scenario after he confirmed which tests apply and also consider those that he might not have been considering. I did not think such a professional needed me to interpret the situation for him. This is how I would want my questions answered, to assume that I have a brain and can think for myself, given the proper rules. If a follow-up were needed, the individual could certainly ask for the specific clarification needed (such as, “would it matter if the relative is in jail?”).
Would it be more unwelcoming to say, “Which of the rules are you asking about?” or “Please clarify what you are asking since the question is so vague,” or “We need more details if you are going to get any help here.” Is it not more welcoming to attempt to answer a question and by doing so either answer it adequately or demonstrate to the author (who I am assuming does not realize how unclear the question might be) that the question is not clear?
I generally don’t need or want the details, so I don’t necessarily ask for them. I am not watching a television show or reading a novel. I am helping someone to supposedly help themselves. I also have a tendency to be a bit literal, so if someone is asking a question like “Is there a way to….” I have to remind myself that it is probably not a “yes/no” question.
What causes the frustration and what SO doesn’t seem to realize, is that many new users are not here to help themselves. They want a quick, complete answer; totally packaged for their situation.
Until that’s fixed the problems will continue
I couldn’t agree with you more.
The real problem – newbies are lazy and don’t want to grow. Plus they have bad attitude for self-development.
Now they picked up new trick – call any comment with “RTFM” on it “unwelcoming”.
Also, those, who grown up on Q&A fastfood encourage this behavior.
While I see your point about (some) newbies being “lazy” and focused only on finding an answer and then moving on, I would like to point out that 1) a lot of people searching for solutions are non-native English speakers, and thus may feel quite intimidated by the prospect of responding with feedback on issues they are themselves not so familiar with, and 2) on many questions it is *impossible* for a newbie to give any response – even if they actually are the only person in the world having the answer – because a certain “reputation level” must first be met. Especially the latter limitation is very frustrating – and please tell me why I should make tens of inane postings on other “open for comment” questions just to be able to provide a single, expert response where it would really matter? The language problem is less easy to solve – but I think we can all relate to finding ourselves in a situation where one very quickly has to chase up that old high school French from deep inside memory, and finding it really doesn’t help you to discuss some detailed technical issue (with your broken laptop power cord, or whatever).
Telling someone to RTFM is absolutely unwelcoming, and painting everyone new to the site this way is ignorant and elitist. If someone posts a question that isn’t appropriate, flag it or just answer the best people way.
I would also be interested in seeing more raw data here. I care a lot about Stack Overflow and have volunteered a lot of my time because it has been valuable and helpful in my own programming education, and I felt it was important to give back to the community.
This seems like a very small sample size to draw conclusions from, especially because the participants were all either employees of Stack Exchange, or were already deeply involved in the community. As Mark pointed out, there are big cultural differences in what people perceive as polite or rude, as well.
I totally agree that there are cases where commenters are rude. As Stack Overflow works towards raising the community standard (a good thing), I hope that legitimate criticism is still encouraged. Some (not all) of the shamed examples here read to me as fair, constructive criticism. I have not been “in the data” as much as Julia and Jason, so maybe I would feel differently after seeing the raw data. Do you have plans to publish it after the study is expanded?
I admit to making some of the unwelcome comments, particularly the last example.
But as someone else said, context is important. They generally come after spending a significant amount of time trying to help more pleasantly. After a while I become exasperated at the cluelessness of the questioner. I could just give up silently, but I don’t think that would be considered much more welcoming.
“Stack Overflow is a place for developers to help each other” — that’s what I thought when I first joined. While there’s some of that, it’s mostly become a place for novice programmers who don’t know what they’re doing to try to get experts to debug their programs for them.
That’s not supposed to be how the site works, but we let them get away with it. And now we’re being criticized because once in a while we get frustrated and post slightly negative comments?
I agree that comments on SO can sometimes have an unwelcoming tone. The toughest issue I see is dealing with borderline comments that provide good advice while coming across as patronising or sarcastic.
I’m disappointed that some users don’t seem to see this as an issue. No, Stack Overflow isn’t a wasteland of toxicity. But that doesn’t mean I haven’t seen users respond to genuine questions with “*Why couldn’t you answer this yourself?*” or “*Did you even try googling?*”
As a strong supporter of being more welcoming, I was surprised at a couple of the examples of unwelcoming comments shown here. Apparently I am more conservative on what I consider to be unwelcoming than I thought I was. Compared to those examples, comments that I have been thinking of as unwelcoming must be abusive.
Great post! Just considering the percent of unwelcoming comments, someone may think that this problem is not relevant.
Besides, ordinary users seem not to be treated harshly. I guess, a great part of the posts receive more upvotes than down and the majority gets zero scores.
It might suggest that the ordinaries are not making undesirable participation.
However, you mentioned an important question: what is the impact of unwelcoming comments, even being around 7% of all comments?
Maybe a future work for StackOverflow data science team will be to measure this impact.
So, it would be nice to answer questions such as:
How many users stop participating after unwelcoming comments?
How many users reply unwelcoming comments?
Does the mobilization of ‘good users’ can help to improve the quality of the discussions, where each learner can better trust that their interactions with others are reliable?
‘Maybe a future work for StackOverflow [sic] data science team will be to measure this impact. ”
I believe Jon Ericson posted about that already on Meta Stack Exchange; it’s one of the recent network-wide featured posts, so I’m sure it won’t be hard to find.
Many of the users commenting here are stating the ‘unwelcoming comments’ are not unwelcoming, shouldn’t be taken without context, they’re exasperated from helping, etc. I could not disagree more. I don’t know what context one could possibly put those comments into where they would be deemed appropriate, respectful, and helpful to a user. Giving someone help should not come with a tone of ‘hurry it up and be a genius at this already’. This is a site where people are coming for help with a problem already and the tone is really unnecessary and quite frankly, UNhelpful.
“I don’t know what context one could possibly put those comments into where they would be deemed appropriate, respectful, and helpful to a user.”
I do.
What should be becoming readily apparent to people who didn’t already know it is that people on Stack Overflow have different perspectives on what is polite or welcoming, what is neutral, and what is rude or unwelcoming.
Context is important here precisely because of this fact, and the fact that we don’t have the whole picture. Maybe the comment is in response to someone insulting their mother? Such an exasperated comment looks pretty civil and welcoming then! The bottom line is we can’t pass fair judgment on these comments until we know the whole picture. Were these initial comments? Were the intended targets being rude/unwelcoming back/before hand? Is this their first interaction? Have other people already tried to help the intended target and gotten nowhere? These are all things answered by additional context that would greatly change the tone of the comments shared above and especially the comments shared above before the blog post was edited to swap out the comment examples.
Here’s one of the “unwelcoming” comments:
“The code you posted cannot yield this result. Please post the real code if you hope to get any help.”
What is the “welcoming” version of that? (Assume that the first sentence is factually correct.)
“The code you posted cannot yield this result. Please post the real code.”
That’s all that is required IMO. Adding the “if you hope to get any help” is what turned the comment snarky.
Since I have spent years trying to learn how to communicate with overly sensitive individuals when delivering feedback, if I had written that, I would have (by the 20th edit) come up with something like this:
I am feeling discouraged in helping you since I cannot fully understand how your code produces the result you describe. I am sure others may have a similar problem in trying to help. Please provide more information about the actual data along and some explanation of how the code you posted works so that I or someone else can better assist you.
If I were being paid (e.g., as a manager) to help a subordinate, I might actually say something like that. On a free forum, after the 20th tweak, I would likely delete the comment altogether….not just welcoming, but wimpy. More likely, if I wanted to write the first comment and realized it would be thought of as “snarky” I would not waste my time tweaking it to be “welcoming” for someone obviously rude enough to post poorly constructed garbage and then beg for help. That is more offputting than the response, in my book.
“The code you posted shouldn’t yield this result. Is the code posted here exactly the same as the code in your project?”
This suggestion makes it even worse. “shouldn’t” and “cannot” are two entirely different things. The first one means “I think it’s wrong to yield this result” which is my opinion about it whereas the other one says “it technically cannot yield this result” which indicates a bug.
More detailed analysis here would be great. How many of these ‘unwelcoming’ comments are directed at questions (or answers) from low rep users vs high rep users? How do the comments correlate with the score of the question (or answer)?
I personally notice that low rep users receive far less grace and tolerance than high rep users.
I’d also like to take a second to address another thing in this thread. A lot of people are saying a variation of “Those unwelcoming remarks aren’t unwelcoming. They’re valid criticisms and important things to say.”
This is an overly bold and definitive statement when talking about personal experiences. Try saying “Those unwelcoming remarks aren’t unwelcoming **to me**.” instead. Otherwise at best it sounds like you’re not listening to real criticisms and data and feelings of others, or at worst, attempting to cover over people’s problems. I’m giving people here the benefit of the doubt in guessing that they are implying the “to me” part, but that takes conscious mental effort to do. By clarifying that you acknowledge that this is your feeling about the subject and others may feel differently will bring a much more inviting atmosphere for discussion. Explicitly saying it out loud may even change how you think about this subject. Give it a try!
On the topic of “valid criticisms” I completely agree (but not with how they’re worded). As an experienced programmer and experienced Q&A user and bug responder I get it completely. However, as another user pointed out, these “unwelcomed comment” examples are all dripping with frustration and/or giving them an additional expectation for getting help. I think we can all agree that “This is becoming a waste of my time” is wrong in any context but where does “I have already told how you can…” fall?
These seem representative of comments I’ve seen around the site. They can read as and be anything from gatekeeping, rule lawyering, frustration, terseness, attempting to stop someone from just saying “solve my bug”, curation, socializing a new member to our cultural values, badge and reputation hunting, or honest help and more.
In my experience showing the site to others (and my learning to use the site) they feel scared. They understand it is a site with a high expectation of quality and see lots of ways to get it wrong and few ways to get it right. Even when you do write a good question you can be accused of making a duplicate (how dare you muddle up our site with duplicates!) Or putting in the wrong community (how dare you make us disorganized!) You can get a negative score or lose reputation (feel the burn of the down arrow!) They feel nervous that they won’t meet the expectations of the (real or perceived) gatekeepers or that nothing will appease the great SO mods.
Comments are a deeply personal connection, in the midst of a very well automated internet. As such valid criticisms can act as a pin popping someone’s energy bubble. We forget that a lot of emotions go into asking for help: you couldn’t figure it out, but you know the answer will be so humiliatingly easy; you missed dinner trying to get it done but still couldn’t; what if someone else finds out you weren’t an expert after all; what if you spend an hour coming up with a well formatted bulletproof question and they tell you you wasted your time.
A valid criticism at the wrong time or in the wrong mood is a fantastic way to make someone say “forget this.” Adding on sarcasm, impatience, elitism makes 1 to 3 of these experiences even more likely to cause someone to leave early, especially when a person has gotten used to a life hearing “You’re not good enough to join our club” or “Quit now before you embarrass yourself more.”
Figuring out tools, new design decisions, and new mod strategies to help avoid comments that drive people away is extremely important, even if it means questioning how we use the valid criticisms that keep this place hygienic and useful.
Thanks for this, I appreciate your point of view.
Very thoughtful comments on an interesting article.
I’m a low-ish reputation user, I read the site often but hesitate to contribute and the nagging feeling of “you’re not one of us” resonates. I also find the refusal by some users (see comments here and on Stack Overflow Meta) to acknowledge that other users may find some behaviours unwelcoming just as off-putting as snarky comments on the main site.
Well said.
“What is the “welcoming” version of that?”
That’s easy: “The code you posted cannot yield this result. Please post the actual code that exhibits the problem.”
Inadequate. But the one that really got on my nerves was “if you can’t make it work you’re doing something wrong.” This is the strongest example, but it can’t be judged fairly without real context.
I now feel that this be welcoming thing is intended to be misused.
I agree that that such a comment is “unwelcoming” at best, possibly also unanalytical, unimaginative, unhelpful, and uninformed.
One thing that many of us either forget or never know is that many times our environment has evolved. It could be a library we added or an add-in that we use or some “trick” we invoked in our normal template 100 years ago, or an option we set up in an application (or within the operating system itself). It is also possible that we did not add these things but that they were “given” to us by our work standard install, so we might assume it is part of what might be normal for everyone else. The number of potential differences is incredible. I have been in environments where we had two identical machines, set up identically with the same install of software on each and the same levels of drivers on each. It took us weeks to find out why something worked on one but did not work on the other (it was the revision level of an HP networking card that was different in each machine even though the cards were supposedly identical).
Not everything that works when I try it but doesn’t work for you is your fault because you are doing something wrong.
I’m glad steps are being taken to make Stack Overflow a more welcoming environment. I stopped contributing besides voting (the only way I can interact without being punished) years ago. I’ve been telling everyone I know to just use Stack Overflow for a reference rather than to ask questions or give answers because I don’t want them to have the same negative experiences I did. Even if this is implemented, it won’t prevent the majority of questions from new coders from being heavily downvoted.
I had this idea before, but it was (ironically) heavily downvoted: make a staging ground for Stack Overflow noobs to ask and answer questions and learn how to moderate and such before they graduate to the real Stack Overflow. Maybe people will learn skills and gain confidence instead of becoming fed up with the elitist environment.
SO has a lot of poor questions at the moment. I wonder how much these two overlap.
If you divide the 7% into comments on questions from people who’ve asked at least one good question, and people who never have, what happens? Are the two rates very different? (Use any test for “good”.)
If you divide the 7% into comments on questions that are deleted from the site quickly, and questions that stay, what happens? Are the two rates very different?
I sincerely hope that those who complain about “unwelcomingness” will soon be haunted by the spirits that they called, and notice that whatever they say, there will always be somebody who finds something to offended about.
As someone who finds it (almost pathologically) difficult to “socialize” (i.e. to be polite and welcoming), and who is not familiar with the subtleties of the English language, I consider the recent strategy and proposals of Stack Overflow as hostilely discriminatory and excluding.
You’ll have to find some balance here. And I’m pretty sure that you’re not on the right track.
But time will tell whether the most valuable contributions that are made to this site come from those who are not good at socializing (and thus spend 14 hours per day with coding and learning), or those whose job is to promote rules for behavior that they find fluffily-comfortably desirable, but have nothing to do with what this site once was about.
(It was coding – just in case you wondered…)
Currently, when e.g. starting a question with “Hi!”, that greeting gets removed, also removing warmth, humanity, human emotion, respectful etiquette etc. I understand it also helps keep the message brief and to the point. However, have you considered of stopping such enforcement to foster more community spirit?
(In similar vein, e.g. a “Thanks!” in comments is disallowed, as it doesn’t fulfill the minimum comment length requirement.)
There is a way to say thanks: upvoting. That’s the way to do it and provide points for the person answering, or just noting others of the useful comment. If everyone started posting thanks-notes then they would overwhelm the site. And if just the original poster is allowed to thank, it feels completely weird.
Since this is a Q&A site and not a discussion forum, I am completely for removing these superfluous “warmth, humanity, human emotion” etc. It’s not a discussion. It’s a problem statement and a fix for it.
Let me ask you this: if you read a manual/FAQ for a washing machine, which of these would you like to see?
“Hi, I’d like to wash some clothes on my new washing machine and I would like to kindly inquire should I use the setting 1 or 2 when I have my Glarbon branded clothes in mauve? Maybe someone could explain me how to work this, since I’m a first time user of this machine and I’d really like to was these trousers. Thank you very much in advance, John
Hi John! Thanks for asking! We value your questions and would like to say that you should use the setting 2 for mauve colored Glarbon clothes. Again, thanks for asking!”
Or would you prefer this:
“Should I use setting 1 or 2 when washing mauve colored Glarbon branded clothing on this machine?
You should use setting 2 because it’s designed for mauve colored clothing”
I think the latter is the one everyone would want. And that’s what this site in my eyes and in everything it says about itself is. It’s for more than your problem. It’s for everyone having the same problem. It doesn’t matter if it’s asked by John or Mary, it doesn’t matter if there’s thanks or you’re welcome. It’s succinct, helpful, factual.
Of course others may disagree and someday the “ban” for pleasantries may be lifted. But as I expained I see the reason for it. It keeps the questions and answers clean, faster to parse, and helpful to everyone. Not just one person.
In my personal experience, it’s the behavior encountered that is more unwelcoming than any specific statement or comment. One Stack Exchange site I was new to closed my first question with little or no explanation and then punished the one person who was trying to help me. I then went on meta to better understand what I did wrong, and no one replied to me. That’s far more unwelcoming than receiving a somewhat snarky comment after hours of help (I totally agree with the folks above who mentioned that context is very important).
It’s funny that there’s been so much hand-wringing about negative and unwelcoming comments when the solution is practically staring us in the face. StackExchange has lots of helpful answers and relatively few unhelpful ones because we’ve incentivized helpful answers and disincentivized unhelpful ones.
Unlike answers, comments can be voted up but not voted down. That means that a) there’s no penalty for making unwelcoming comments, and b) there’s no feedback mechanism that tells people that their comments are unwelcoming. I think the lack of a negative feedback mechanism for comments is preventing the community from self-policing unwelcoming comments as effectively as it could. Adding a button to down vote comments to go with the existing up vote button would cause commenters to be a little more aware of how the community sees their comments. And keeping track of downvotes on comments would allow the system and/or moderators to easily identify those who routinely make negative/unwelcoming comments, and perhaps restrict their ability to comment for a while.
Feedback, both positive and negative, is the thing that makes communities like StackExchange and Reddit as successful and as valuable as they are. There’s no reason not to apply that lesson to improve the quality and helpfulness of comments.
This.
Great point!
Good point.
I’ve seen the concept of “unwelcoming” come up a lot recently in the context of the discussion and efforts around improving the Stack Overflow community.
As I was reading this, something occurred to me that I haven’t seen highlighted enough… when looking at the concept of “welcoming” or “unwelcoming”, and which way a comment or post will be perceived by a user, implicit in that evaluation is an assumption the user has **just arrived**. That’s part of what “welcoming” someone is all about. You don’t welcome someone who has always been there.
That might matter for evaluating these comments, where we need to know the context of who the comment is directed towards to really evaluate things correctly. A comment that might be fine to an established user might not be so good for someone new. It also might **not** matter, in that we’re trying to create good artifacts for future programmers, and so a new person could still at some point read comments originally directed at established users.
Either way, though, I think this concept needs more attention and thought. Is there a difference in whether a comment is merely unwelcoming depending on the target user, and if so, where and how can we draw that line?
One thing that I feel strongly about both ways is the phrase “No subtle put-downs or unwelcoming language.” I agree with the intent as I understand it, but have real problems with the statement itself.
For the first part of this phrase, it makes me wonder who the thought police will be to determine whether I intended a subtle put-down or not. For example, if I say something like, “If you test this and it doesn’t work for you, you may have to use a different technique.” Is my point that the user isn’t even trying to test anything? Is my point that the user doesn’t have sufficient analytic skills to determine if something works or not? Is my point that the technique won’t always work in every scenario? Well, the jury may look into my head and accuse me of bad behavior if my thoughts are determined to be bad. Will I have to not only spend time making sure my comments are not “unwelcoming” but also not able to be misinterpreted by even the most malicious judge?
The other part of this is the use of “unwelcoming” as if it means something that it may not. The phrase “He is not unintelligent” does not mean the same as “He is intelligent.” There is a middleground. Someone not unintelligent could be average, and the above says we should not use comments in that middleground. In other words, an “unwelcoming” statement could be a neutral one like, “I have not considered that situation before.” If we mean “vicious” let’s say “vicious.” If we mean “nasty” or “snarky” perhaps we should state that. If you mean sarcastic, say so.
Really if someone asks “How do I reference particular sheets in VBA?” without any further details, do we really want something that is not “unwelcoming” in response? Must we really respond with something like, “Great question! I had that question myself but I found there were different answers depending on what kind of module it was being called from and which Office application it was called from. I’d love to help you if you could give me a smidgen of additional information. Could you please add that to your post for me. I am anxiously waiting to know what you tried and what happened when it didn’t work. Thank you!” or would something unwelcoming and clinical like, “Can you please update your post to describe what you’ve tried, what you expected, how it didn’t meet your expectations, and what you mean by selecting a sheet?”
If you are going to assume that everyone knows what you mean, you will be wrong. I suspect that this is going to make people think twice about asking for clarification and without clarification, some of us will not be able to offer any assistance.
I just wonder if you’re giving any thought to the following. Many SO users are not native English speakers. They may not always understand subtleties of the English language and therefore unintentionally post a comment that you would flag as ‘unwelcoming’.
So one question is: how to welcome the unintentionally unwelcoming?
Two comments.
First, it would be good to somehow take into consideration that many SO users are not native English speakers. They may come across as what SO-developers classify as ‘unwelcoming’, while in actual fact they just make a language mistake. (Example, I recently enjoyed a conference where a non-native speaker was quite profane while presenting. Judging by the level of English and the rest of the talk, this was clearly not intentional).
Second, there are cultural differences across countries. To an extent, this is already highlighted by the inter annotator disagreement you already wrote about. But that is just between SO employees. Taking myself as an example: I’m Dutch, and we are known for being very direct when it comes to comments. Not necessarily unpolite, but we do tend to go straight at it. We pride ourselves in such efficiency, but it can come across as unpolite or even rude when talking to people from other countries. So I think that the inter annotator disagreement on what can be called ‘unwelcoming’ is probably a lower bound on the disagreement across the SO user base.
Same for us Finns. I often say we’re like the Ents: we don’t speak unless it’s worth saying. We’re succinct. We don’t do overt pleasantries since in our language and culture politeness and friendliness is automatic and subtle and you’ll know when someone is being rude. We don’t have words like “please” etc which may make many Finns seem rude in other languages which do have them, just because they’re not used to them.
Also since this is a technical site I would claim most people want succinct to-the-point questions, answers, and comments. If an error message says the error clearly (they don’t always do this, I’m looking at you Android Studio / Gradle / Android SDK with your JSON error messages that aren’t shown as clear errors) that should be pointed out since the asker might not have understood it. This makes them look at it again. Of course it shouldn’t be said rudely and explanation of it is warranted, but still it’s good to point out the information is there for the next time they have this issue.
I’d welcome so much more information about the way the “study” was done, but seems it will not be presented.
“This error is self explanatory. You need to check…” I disagree with this as being unwelcomed. Maybe specifying “This error message is self explanatory” instead. Sometimes people don’t check carefully the error message & there is little to say more about that (it happens to everyone). Again this depends on context. And not everyone is fluent in english so in those subtle cases I think there is no need to nitpick. Maybe that’s because I’m non-native english speaker so I always guess there is nothing confrontational (or personal feeling) when someone writes something like that.
I often see comments like the cited examples and agree that they’re unwelcoming. It’s surprising to me that so many here think this is an acceptable way to communicate.
While I agree the frosty tone on SO is a problem, I’m expecting a huge chilling effect in removing such eye-of-the-beholder comments intransparently. Sometimes, a “this is in the manual” comment or pointing the asker to a good Google query *is* the solution they need. *Especially* when you can enter the question title into Google and find a good answer to the problem among the first three hits. That happens frequently and is not what we are donating our valuable time for. If comments start being moderated even more strictly and opaquely so you can’t be sure any more which of them will survive the niceness algorithm (or flagging by overzealous users) that’s likely to hugely affect my interest in doing anything at all here any more.
You are so right about potentially becoming gun-shy about helping others, especially, if using a language with which someone is unfamiliar. Will this come off as professional and polite and efficient, or curt and bothered and unwelcoming? If I am in a rush and want to point something out, should I try to type a comment quickly to potentially help someone and possibly be accused of being unwelcoming because of the way it came out or just not type it at all rather than take that risk?
Typing errors and “help” from apps that “fix” your words for you could come off as unwelcoming. I can’t find the exact example, but in one forum, there was a ruckus when someone responded “Have you even tried this….” but folks calmed down when they realized that it was simply asking for information as a follow-on and accidentally typed “even” when he was typing “ever,”
This all takes me back to the thought control in The Prisoner where the worst thing anyone could be accused of was being “unmutual.” Of course, there were treatments for that. Maybe we will come up with some here as well.
It is rare that any forum is viewed by all members as being “welcoming.” On one particularly pleasant and well-mannered forum, the members were appalled when they were accused of being cold and aloof. Why? Well, many of the members had made true friends there. They knew their schools, their families, their cars, their hobbies, etc. and would make obscure references to a pet or a television show, or a recent illness that made newer or less-active members feel left out. Thus, the camaraderie that they showed (to those that they knew) was viewed as unwelcoming by newcomers.
Hopefully, we are not attempting to please everyone.
The main problem with this whole discussion about trying replace “unwelcoming” behavior with always “welcoming” behavior is that partially it totally contradicts human behavior. No human communication ever is emotionless. And thats not wrong or an error, thats how we humans are.
There must be rules to prevent this from escalating to abusive behavior. To be clear, those rules must be given and must be adhered and sanctioned also.
But if the examples provided here are “unwelcoming” behavior, then this is not meant. Instead is meant that all users here shall work as if they were support employees of SO. A support company of course can demand from it’s employees always using “welcoming” behavior to customers and never using kinds of sarcasm nor rebuke even if the customer behaves unfriendly or even malevolent.
And this is the dilemma here.
Either SO wants participation of users who are individuals from heterogeneous groups of humans and who are not employees of SO. Then those participation always will be combined with the human behavior of those participants, with their opinions, emotions, heterogeneous kinds of humor, heterogeneous tolerance levels against malevolent behaviors of others and so on.
Or SO wants always behavior in sense of a company-customer relationship towards it’s users, which then better shall named customers. Then SO needs employees for this and cannot using participation of users who not are employees.
While I just posted my own comments of concern, I wish I could to upvote this and many other responses of concern. It’s like you’ve read my mind!
I seriously feel the breadth of the impact this welcoming exercise has not been thought through enough.
While all of this recent “lets be more welcoming” exercise is great in theory (and I support it), I’m worried about how much thought has been put into its impact on the actual people who answer regularly.
I’ve been with SO for a few years now (5+ off the top of my head i think) and I can’t recall a time when I saw a comment I found “unwelcoming”. I, of course, could be one of those people who are much more lax about this than the people who found the 7% comments unwelcoming but that makes me wonder: what if you end up censoring (be it via implicit self-censor or other means) your regular answerers who never actually intended anything unwelcoming but merely pointed out the fact that not enough effort was put into the question (which is often the case for new questions).
Yes they could potentially be more welcoming but if you’re a regular, and you have to make that choice of self-censoring while the incoming questions continue to be of bad quality; would you want to continue answering? I personally have already stopped as most of my time is spent answering questions in chat room rather than answering question that never get any response from OP (and that of course doesn’t even involve any self-censoring impact).
What if this whole thing backfires and we actually lose more people who would have answered good questions otherwise? How much thought has been put into this aspect? As somebody else said, 93% of good content in a site this size is a huge achievement. Yes things can be always better, but what if the cost is higher than you anticipated? It’ll be pretty hard to take a U-turn from that.
Kind regards from a concerned resident,
Maverik
One thing I was wondering: Did the analysis include asking the people whose questions & answers the comments were posted on whether _they_ found those comments to be unwelcoming? There’s a BIG difference between asking some random third party whether something is or isn’t welcoming and asking the person the comment is aimed at whether it is or isn’t welcoming, after all.
Have you ever considered that many, many engineers are in the spectrum, have some problems recognizing what is ‘nice’ and what is not, and may feel intimidated from posting anything, due all this emphasis on ‘niceness’? (No, I’m not saying that people in the spectrum are jerks)
Wow, I was just about to post this. When Tetsuo says “spectrum” he refers to the Autism Spectrum Disorder. Many folks are not “diagnosed” or even suffer from what would be called a “disorder” (it has to negatively impact your life to technically be a disorder – if you can cope with the condition you don’t technically have a disorder). Nonetheless, there are many people who are engineers, programmers, scientists, etc. who exhibit the characteristic lack of emotional intelligence (or at least empathy, as they may well know how you feel, but be unwilling or unable to respond accordingly).
I am among those on the spectrum (obviously in the more high functioning end) and I have had to work incredibly hard to be mindful of others feelings. Personally, I can hear the objectivity in many of the comments that were used as examples – which is really what those on the spectrum tend toward: Blunt, objective statements. They come across as hostile because, in the USA, we live in a hyper-sensitive culture that requires us to be mindful of the emotions of others at all times and accommodate those emotions.
I believe that the focus on being more welcoming is important. I think all the work that SO is doing to be inclusive is valuable and should be continued. I think what we’re missing is the other half of the conversation – the part that asks newbies to not be easily discouraged. Both sides can meet in the middle if we don’t give either preferential treatment – the “experts” can learn to be more personable and friendly, while the “newbies” can learn to take blunt criticism and can learn to overlook an occasional terse comment.
We live in a complex world and straying too far in any direction will always earn you the ire of one group of people or another. SO will lose many of its highly knowledgeable experts if it continues to wage its war against minor social faux pas. We must endeavor, as a community, to demonstrate what the middle ground looks like. We must not alienate any one group – and this includes folks on the spectrum as well as people who simply have low emotional intelligence or empathy. They can be (and often are) excellent sources of knowledge in their chosen fields. If we are going to mandate that they must become more sensitive, then it is only fair to ask the most sensitive to “grow a thicker skin” as well.
I like the general idea but I don’t know about the comment evaluator.
First of all: to rate some comments one would have to know the context. If someone writes a non-answer, then it is ok, to point that out in a comment, I think. But if someone writes a valid answer but someone points it out as a non-answer because it does not fit in their closed mind mindset – that’s a totally different story.
The rating of the comments in the comment evaluator was very restrictive : it wasn’t possible to rate anything as better than fine (some of the comments weren’t just being not unfriendly, they were very nice and helpful, it would be nice to be able to evaluate that too). There was no option for “can’t say” or “it depends on the context”, so you ended up rating comments as fine where you just didn’t know.
I think it would be better to be able to rate comments directly in SO, not just have a +1 option and flag as abusive, but be able to rate them, perhaps with an extra option of why (e.g. not helpful, unclear, not real english or very abusive of English language and thus not intelligible.
This is a really interesting Topic.
I love Stackoverflow, it helped me a lot of times during my studies for IT. Like most i too had some troubles with really unhelpful Comments or plain misleading ones. On side enabling Comments is a freedom given to everyone simply because of free speech. But I can understand that some of them are just there for the sole purpose of being mean or misleading because why not? People on the receiving end cant do anything about it anyway, right? Good article, was a very pleasant read. Looking forward to seeing more like this.
Put the effort into evaluating questions before they’re posted instead. Especially for new users.
I find stackoverflow is a VERY useful website. I have found the most accurate answers to many of my questions on this site.
I am a software developer with over 40 years of experience in at least 20 different languages. I have my own startup now. Many of the answers on this site have helped me get past issues that I have faced in developing software for my website.
Unfortunately, the “reputation” mechanism on this site is so restrictive that I cannot share any of my own experience on issues with others. It seems I need to post questions in order to gain “reputation”. All of the questions I have had had have been answered (correctly or incorrectly) by other people with “reputation”. So I am stuck as a “newbie” even though I have more experience that many of the people how have somehow gained “reputation”. Catch 22.
It’s not really a loss for me as I get all the information I need from this site.
Hope you fix your broken “reputation” policies at some point in the future.
Hey Bruce – there are lots of new questions posted every day, many from beginners. Perhaps monitor the “new” questions area and sort tags that relate to programming languages you specialize in. Hope that helps.
Is the dataset used in this project publicly available?