area51 December 23, 2010

No Artificial Intelligence in Area 51

On Monday afternoon, we unceremoniously closed down a Stack Exchange site: Artificial Intelligence. Not that many would have noticed. It had an laggardly 83 questions in its 12 days of existence. It wasn’t so much the lack of questions that was of concern — a site can stay in beta as long as it takes — but the…
Avatar for Robert Cartaino
Director of Community Development (former)

On Monday afternoon, we unceremoniously closed down a Stack Exchange site: Artificial Intelligence.

Not that many would have noticed. It had an laggardly 83 questions in its 12 days of existence. It wasn’t so much the lack of questions that was of concern — a site can stay in beta as long as it takes — but the conspicuous lack of _expert-level _questions. This was also the emerging opinion amongst the users:

I committed to this proposal some time ago, hoping that this might become a site for researchers or knowledgeable academics asking serious technical questions about artificial intelligence here. It seems I was dearly mistaken … Most of the questions are those asked by the merely curious.

70-80% of the questions didn’t run much deeper than “When will we have intelligent computers?” and “What is your favorite AI blog?”

I can understand the curiosity. As a computer enthusiast, I am somewhat intrigued by artificial intelligence. But I couldn’t even begin to ask a question suitable for a knowledgeable researcher. I’d be one of the merely curious… as were most users on the site.

AI’s problems began almost immediately when users started asking the first questions:

It has long been established that no question is too entry-level nor too basic. Everyone is welcome. But, in these earliest days, we are DESIGNING a site for experts. To attract experts, you need a site where people are asking very interesting and challenging questions, not the basic questions found on every other Q&A site. Remember, the pro sites WILL attract the enthusiasts, but not the other way around!

The earliest questions on a site will set the tone and topic of the site for a long time.

The AI site conspicuously lacked that “tone and topic” from day one, so it had nowhere to go and was closed down.

Lessons for Area 51

The purpose of Area 51 is to prove that a site has critical mass before it launches. That didn’t work here. The followers and committers accumulated very slowly over six months, so there was a low response rate from the committers to join the private beta. We may need to consider something like aging commit votes so older proposals don’t simply accrete votes over a long period of time. We need to establish that a proposal has sufficient momentum (an escape velocity) before it creates a site.

It is difficult to say if many experts were part of the AI proposal; I don’t think we ever had them. But Area 51 has no way to measure if the private beta will include a sufficient number of qualified practitioners, so we need to make adjustments. Here are some changes we adopted from this experience:

A site can stay in private beta. Typically, a site is ready to launch after their prerequisite seven days in private beta. But if the early site hasn’t worked out the kinks of its definition or audience, we can work with the community before launching a half-cocked site to the public. If the problems are impossible to reconcile, the site can always be closed to try again. As we improve the process of Area 51, this should be exceedingly rare, but AI has become the new baseline for “failed to make it out of private beta.”

Establish a site’s expertise early. New users, anxious to jump start their communities, inevitably start asking uninspired questions that have all been asked 100 times before on every phpBB forum. You’ve seen them: “What is your favorite…”, “What is the best…”, “What is the definition of…” Unfortunately, these idle questions can fill the front page in the opening days, and left unchecked will permanently color the tone of the site. Your front page is your billboard, it defines your target audience.

Watch for and discourage pedestrian questions early in the beta. Certainly, questions of all levels are welcome… but not in the earliest, most formative days of the site. These questions may someday become wildly popular, so politely — very politely — invite those users to ask their questions again after the expertise of the site has been established. Top blogs, best books, buying recommendations: those are not the hallmarks of expertise. They’re the seeds of the merely curious. A site filled with these sorts of idle, pedestrian questions will never attract the core of experts it needs to survive.

Podcast logo The Stack Overflow Podcast is a weekly conversation about working in software development, learning to code, and the art and culture of computer programming.

Related

the-loop January 22, 2020

The Loop #2: Understanding Site Satisfaction, Summer 2019

We’re excited to share research highlights about the work we’ve been doing to understand how satisfied people are with Stack Overflow. We’ve been working hard to explore what users like best about Stack Overflow and what their top pain points are, with the goal of improving the overall experience of using the site. To this end, we’ve launched a site satisfaction survey, in which we continually survey users about their experiences using Stack Overflow.
code-for-a-living February 27, 2020

The eight factors of happiness for developers

I recently came across this sketchnote by Tanmay Vora, and it really resonated with me. As a developer it got me thinking about how this might translate into the life of a developer and our happiness. Based on this sketchnote here are the eight factors of happiness applied to developer life. 1. Resentment Harboring resentment…