The Stack Exchange network is a huge dynamic system with tens of thousands of interactions taking place each day. As the number of users grows and we improve our site engine, the social aspect of the network becomes more complicated: the number of user groups united by motivation on the site grows, and the groups themselves begin to interact with each other in new, complicated ways, forming feedback loops. As a result, it becomes difficult to determine the current state of our social system and what we should work on to make users succeed on the sites.
Reducing complexity
As in any system, the social part of the Stack Exchange network can be viewed from the standpoint of complexity and connectedness. Complexity indicates how difficult it is to understand and interpret the ongoing processes in the system. Connectedness indicates how the failure of one component affects the overall system. It is believed that the higher the complexity and connectedness of a system, the more difficult it is to keep the system working.
It seems to me that while we have not yet reached the saturation point of connectedness, the social complexity on Stack Exchange is difficult to overestimate. Each day, we see very different people interacting with each other:
- New users at all levels of knowledge, background, and expertise from all over the world.
- Curators varying in activity level, domain knowledge, and preferred contribution.
- Long-time users with different motivations and their own visions of the future of the site.
These and other user segments create unique contributions that combine to provide an incredibly diverse set of driving forces that might even look like chaos. With all this social complexity, the CM team must daily answer one simple question: Is the community healthy? And if not, what problem areas should we work on today to help our users be successful on our sites?
Our first attempt: The meta health indicator
To answer the question above in a systematic way, we created a prototype of a health indicator for meta communities. The indicator summarizes several metrics and has three states:
- It is green when the community is doing well.
- It is yellow when there are minor problems.
- It is red when the community needs special attention.
All metrics used to build the indicator are relative, which allows us to use the indicator on all network sites regardless of their size. We think of the indicator as a set of heuristics and try to keep the metrics as simple as possible to make the interpretation easy.
What does a healthy meta community mean?
A meta community is “healthy” if:
- (1) It is actively used (2) for its intended purpose
- The growth of the number of active users leads to more social interactions on the main site that should result in more discussions on meta. (Metrics: compare the number of meta active users with the number of meta logged in users.)
- We need a meta site to enable users to set up the main site in their unique way, in other words, to discuss site settings, moderation policies and execute organizational work. (Metrics: compare the number of meta active users with the number of meta activities.)
- It facilitates diversity of opinion
- Discussion questions and answers should be asked and answered by a variety of users. (Metrics: the number of unique users who answer questions with the discussion tag to the number of those who ask the questions.)
- There should be multiple answers to a discussion question. (Metrics: compare the number of answers with questions with the discussion tag to the number of the questions.)
- A lot of users are active on the meta site
- If users want the main site to thrive, they will contribute to meta when they have time and the opportunity. As a community grows, the number of questions and answers on the meta site should increase as well. (Metrics: compare the number of meta logged users with the number of logged users on the main site.)
The case studies
- Meta Stack Exchange. The meta community is healthy. The number of active users grows as well as the number of activities per active user. In discussion questions, there is a diverse set of opinions.
- Meta Mathematics. The meta community is healthy. Number of active users remains stable over time with a slight increase, the number of activities per user increases. Even though the number of discussions and people participating in them has declined slightly in the past two years, this year the trends have turned up.
- Meta Stack Overflow in Russian. The community needs some work. The current meta community seems to be stable. At the same time, most activities on meta are conducted by a group of users that slightly shrinks in size over time, i.e. people visit the meta site but do not act. We are observing that the number of folks who start discussions has reduced over time along with the number of things that are being discussed.
Stay tuned!
Even though this is a research project in progress, it shows us what in an amazing time we live in. The time when the combined effort of domain knowledge and data analysis makes it easy to understand how one, from the newest user answering their first question to the CEO of a company, can contribute to the community even more proactively. After all, everyone of us is here to make the internet a better place to find detailed answers to any question on our beloved topics.