Anytime you find yourself answering the same question over and over and over and over … blog post time.
This is that blog post.
This cycle has repeated itself on more sites than I can remember — When a new community approaches the end of their beta period, users start looking forward to graduation. So when that 90th day looms, anticipation starts to turn into speculation about whether the site is going to survive.
- Does this site have a chance of succeeding?
- Is this site viable?
- Visits per day — should we be worried?
- At what point will this site grow out of the beta stage?
- What are the criteria for getting site out of “perpetual beta?”
- Site Not On Track to Survive Public Beta
- What happens now?
In reality, 90 days is a minimum length a site is expected to to remain in beta. The blog post, When will my site graduate?, explains that a site can stay in beta as long as necessary to reach critical mass. As long as the questions represent real problems and consistently receive great answers, the site isn't going to get closed down. "It takes as long as it takes."
So why all the angst?
Communities should generally know when the site is failing. Questions don't get answered, quality declines, community up-keep wanes; In short, the site stops providing a good experience. But that doesn't satisfy the inquisitive analyst in all of us.
When users seek out a report on their performance, they turn to the analytics of Area 51.
Wow, pretty scary. Right? With big, red letters and "Worrying" stamped all over the place, the angst is understandable.
Let me dispel a widely-held misconception…
The Area 51 summary does not represent some sort of "report card" filled with pass/fail grades. If you're expecting someone to show up on the 90th days and say "Sorry, times up. It's time to go home," it doesn't really work that way.
So what do these statistics mean?
The Area 51 statistics provide an opportunity to see where your site can improve. "Worrying" and "Okay" ratings tell you where to focus efforts to push a site closer to graduation.
Questions per day
A steady influx of questions is a natural side effect of a growing, healthy site. But when the number of new questions becomes "worrying," some folks might exhort to "seed" the site to push those numbers higher.
Joel suggested a healthier alternative by rallying users around specific events as a catalyst for asking interesting questions you come across in your day to day work. Any event that gets your community going — a hot new release, an upcoming convention, any news-worthy event — Here’s how he did it on Ask Different:
Now that OS X Lion is shipping, there will be zillions of Mac users upgrading, and they'll have lots of questions. And since all those questions will be new, Ask Different will have as good a shot at having the best answer than any of those, you know, competitive sites. Essentially, this is a great time to recruit new members! As you install and learn Lion, whenever you have questions, no matter how silly, ask them here. You're not the only one having that question. Millions of other people will, too. Ask them even if you think you're going to be able to find the answer yourself... and if you do find the answer, go ahead and answer it yourself …
Read the post and issue a call for questions around interesting events that will be super-popular in your community. Those questions will bring in lots of traffic from search engines and will attract some great new users who will add value for years to come.
% Answered
The saving grace of the statistics above is having 98% of their questions answered. That's fantastic. "Excellent" means visitors have a high confidence their questions will get great answers quickly.
If your site is teetering near 90% or lower, you can probably do better. A concerted effort to get those hardest-to-answer questions answered should help. If you have a lot of questions not worth answering (i.e. "unanswerable" as asked, or low quality), it might be time for a site-wide cleanup effort. That's best initiated and organized through a meta post. Go for it.
The % answered provides a great "pulse" of the site. The most important criteria of a site should be whether experts enjoy answering the questions. If experts think the questions are stupid, then they'll lose interest in the site and questions won't get good answers anymore. This whole thing is about about providing a good experience for the people looking for expert answers to their questions, and the % answered is a good metric to watch.
Avid Users
Another area you can work on is participation. Having a strong base of 'avid users' comes from voting up good content. If you're not voting regularly, you're not building up a class of leaders that can help run and maintain the quality of the site.
All those other statistics will come in time. Don't worry about the actual numbers. I get nervous when users start quoting numbers and propose ways to artificially drive them higher. Calls to lower the bar on quality or close less questions are focusing on the wrong thing. There’s more to a healthy Stack Exchange site than having a lot of questions and traffic. It's about providing a good experience for the people looking for expert answers to their questions.
So why is my site so "worrying"?
As far as the "worrying" statistics above, it's not really all that unexpected. Most Stack Exchange sites are not expected to be an overnight success. Most go through a steady period of building up content before reaching critical mass.
What we generally see with Stack Exchange sites is nice, steady traffic going kind of horizontally for a while; then, at some unpredictable point, we hit critical mass and POW all the indicators start climbing inexorably. This is the right point for a site to come out of public beta.
Can you tell us when we'll graduate?
Unfortunately, we are not yet able to predict when a site will reach critical mass. A large part of this summer will be spent looking at the traffic data we've accumulated over the last three years to make sense of it all.
If your traffic indicators aren't dropping precipitously, that's a good sign. If your traffic is falling, we'll let you know through meta initiatives.
When your site finally reaches that tipping point, Jin will start posting some concepts for the final design. Watch your meta site to provide feedback.
As for when that will happen — as soon as we know, you'll know.
In the meantime, focus on keeping your quality high, and use the share links to promote your most intriguing content.