Stack Overflow isn’t just a platform for asking and answering coding questions, it’s also a rich ecosystem for understanding the challenges, trends, and behaviors within the developer community. Behind the scenes, a dedicated team of researchers employ a variety of methodologies to gain valuable insights into how developers learn, collaborate, and problem solve. This research plays a crucial role in shaping the future of the platform and the tools that empower developers worldwide. Join us as we explore what we learned last quarter, hope to learn this quarter, and how these efforts contribute to making the site an even more valuable resource for millions of users.
Last quarter’s projects
New content types
We surveyed over 600 developers—including many passive users and non-account holders—to understand which needs felt most important and where current solutions were falling short. We presented developers with user-centered needs statements and used a max-diff survey to prioritize these based on importance and satisfaction.
We segmented our findings into the following categories:
- Low-value opportunities: Some needs simply aren’t worth pursuing. Needs such as finding events consistently ranked as being low-importance for our target audience.
- Underserved opportunities: These represent needs that are medium-high importance, but neutral satisfaction with current solutions in the market suggesting there is an opportunity to make an impact.
- Bold opportunities: These represent needs that are very high importance and medium-high satisfaction. Going after these needs will require us to deliver a differentiated solution to what is already in the market.
- Niche opportunities: These needs stand out for a specific user segment but aren’t widely seen as important across the broader userbase. These may be valuable to pursue in the future for targeted audiences, but less urgent for now.
- Saturated opportunities: Some needs are already well-addressed in the market indicating limited opportunity for meaningful differentiation or improvement.
Three core themes emerged across needs in the underserved & bold opportunities that will guide our focus moving forward. These include expert consultation & feedback, collaborative knowledge building, and personal skill development. We will continue to explore this area through the next quarter.
Expert consultation and feedback
One of our teams has been exploring the opportunity around expert consultation and feedback in hopes to affect our outcome of creating an environment where users would want to engage. This quarter we spoke with both people who have a problem they need to solve and people who may be interested in helping others. Our study focused on what type of help users wanted for their unique situations and when human help matters most.
This is what we learned from the 15 users we spoke to who were seeking help:
- Help seekers struggle to get their problem solved on Stack Overflow. Writing a good question takes too much time in today’s world especially when most don’t know how to write in the Stack Overflow quality standard. Then there is a chance they’ll either get negative feedback or won’t get an answer at all.
- Help seekers see value in AI, but prefer human help. Users felt that AI could help with easy questions, organizing thoughts, and exploring directions. While they prefer human help for feedback that feels more specific and grounded in experience, some are willing to rely on AI if it means getting help faster than what our site offers today.
- Help seekers want help understanding their problem. While everyone ultimately wants an answer, most participants expressed interest in help figuring out what part of the issue needs more attention and guidance.
We were pleasantly surprised to find out that there were people who would be willing to participate in an expert consultation concept. We spoke with 17 people (helpers) who previously answered a question on the site:
- Helpers are frustrated with the lack of details in questions. It’s hard to help people when those looking to help others are missing information. And if they try, they risk losing reputation for an unwelcome answer and negative comments.
- Helpers see the value of both AI and human help. Similar to help seekers, helpers believe that AI is great at formatting, helping with common problems, and with older API and technologies that are already well-documented. But questions that require a deep understanding or specific use cases need human help.
- Helpers had various concerns related to language barriers and simply a lack of time, with respect to helping with expert consultation, but most were at least moderately interested, especially if they were compensated.
This team will be continuing to explore what this might look like, including some upcoming on-site tests.
This quarter’s projects
We have been working hard to increase the amount of research we are doing at Stack Overflow, supporting our product managers, designers, and tech leads to each do their own research. Here is some of what you can expect throughout the next quarter
- Exploring new content types around collaborative knowledge building and expert consultation and feedback.
- Exploring users’ mental model and how that affects the way someone posts to Stack Overflow. How do discussions and questions differ? Should we simplify our entry point? Is there a place for every question?
- Evaluating search usability and how we might help users find answers faster. But also how might users find relevant content to consume and engage with more easily.
- Usability tests for features like chat, coding challenges, etc.
Other research this quarter
- Evaluating how users discover questions they ultimately answer
- Evaluating the Stacks editor’s usability and user experience
- Evaluating potential question and answer metadata for our API
How you can help
We need highly-engaged research participants, especially new users and users who don’t engage often. If you are this type of user or know someone who fits this description, please sign up for research invitations in your account settings! We hope to speak with you soon.