Developers’ distaste for documentation, especially among independent contributors, is a well weathered meme in the world of software. Even Stack Overflow is not immune to this tension. “For management, an age old problem is that engineers hate to write documentation. It’s a check box on the process, something that takes time to write but that you feel no one will read it. There is this black hole syndrome where you spend hours working on something and get no feedback. I’ve heard people joke that documentation is where ideas go to die,” explains Tom Limoncelli, head of SRE. “Sometimes it feels performative. Why are we writing this? Because our policy is that we will document every service? That hurts morale, no one wants to feel like their time is wasted.”
In an effort to rethink how documentation work, we recently introduced Articles, a new type of content inside Stack Overflow for Teams that is meant to pair with our traditional questions and answers. Articles allows for longer-form documentation and prose that can sit side by side with shorter Q&A. The goal is to build a system for documentation that can be both proactive and reactive, flexible and on-demand. We decided to sit down with team leads from across Stack to learn what it’s been like to drink our own champagne, be among the first to use our own product, and how it has changed their approach to documentation.
Migration and meaning on SRE
Chris Hunt, a site reliability engineer at Stack, had actually begun a process of reworking his team’s documentation several months before we introduced Articles. His team found itself with a fractured system that had stopped working well in several areas, a condition that often befalls documentation, even with the most well-intentioned coders. “We have three different wikis, and two of them were broken, so we couldn’t update them at all.”
The SRE team prides itself on a culture of continuous improvement. After all, it’s responsible for ensuring that Stack Overflow and our network of exchanges stay up 24 hours a day, 365 days a year. When there is even a brief outage, we hear about it, from millions of developers across social media. “In SRE, we have a culture of ‘leave things better than you found it.’ If you have an alert that’s noisy, fix the things causing the fires. If you found documentation that wasn’t helpful, update it for the next person.”
“The author of 90% of our documentation doesn’t work here anymore!”
So how did Hunt and his team wind up with three wikis, two of which were broken? Often, with documentation, the issue is about the transition between different systems or different employees. “I see two common issues,” says Hunt. “One, maybe they wrote it to the best of their ability, but it hasn’t been updated in a while, and it’s no longer accurate given changes in other systems. Two, the author of 90% of our documentation doesn’t work here anymore! Now, I could take on the task of updating and migrating that information, but as every developer knows, that means I am now the owner, and I may be hesitant to take on that responsibility, especially if I’m not familiar with how and why it was originally written.”
The SRE team wasn’t alone. Erwin Alberto, who heads up our IT team, was also in the process of trying to design a new system for documentation when he learned we were creating Articles. “We had stuff all over the place. Some were in wikis, some in Google Docs, some in email. Sometimes it was on a work drive, sometimes on a personal one. At that point, even though we had hundreds, maybe thousands of pages, it’s really like having no documentation at all, because the whole point is having something that anyone on your team can find and access when they need it.”
Migration
Alberto had already put in the hard work of auditing existing content with his team. His advice? Start with your worst pain points first. “We were in a position where there was a lot of great expertise on the team, but a lot of it was hidden in peoples' heads.”
So, how do you decide what’s worth migrating and what’s no longer relevant? Erwin and his team looked at what would be the most common support tickets and the Q&A on Teams that had the most engagement and traffic. This allowed them to identify the most pressing topics, and then divide them up, with team members committing to write pieces as part of their next sprints.
The IT team assigned one author for each piece on the sprint, with at least one other team member assigned to check the Article immediately after for quality control. “In a way, part of the audit of content happened in Articles.” Alberto remembers, “It was the easiest way to get visibility into the different fragments. We put them up and then edited them.”
Another way this problem is being addressed is as part of the onboarding process for new hires. Josh Duffney, another site reliability engineer, joined the company in June of this year. He began asking lots of questions on our internal instance of Stack Overflow for Teams. He was transforming something intangible into digital artifacts that everyone on the team could use and improve. “A lot of what I have been learning and committing to Teams is tribal knowledge. A good example is how to test our terraform modules. I can ask that question, record the answers, and then bring a bunch of Q&A together to create an Article that documents the difference between my assumptions and the reality.”
“It really reduces the friction around contribution.”
Duffney sees a big difference between the existing wikis and the system of documentation the SRE team is now working towards with Articles. “As someone who was recently onboarded, the pain points with the existing systems was that it was fractured across three wikis, and it’s in a hierarchy that is tough to navigate. As someone new, I don’t know the context of the directory structure.” With Teams, a query will surface any Q&A, Articles, or collections tagged with a topic like SRE, Talent, or legal. “Relying on search and tags is a natural fit for how we operate on the internet anyway. It really reduces the friction around contribution.”
Level Zero Support
Having done the heavy lifting of content auditing, Alberto’s team is now migrating everything to Articles. He says that he sees the combination of long form pieces and Q&A as a new level of support. “We used to have level one, which was sending a ticket to the help desk, and it was something we could easily resolve for you. Level two was a more complex problem that maybe required an engineer or specialist from a certain team to figure out. I look at this new system as a level zero.” Before sending us a ticket, folks can search Teams. If they find a question that solves the problem, great. If they need more details, they can follow links to in-depth articles or collections that bring together Q&A and article with the same tags.“
This activity is both generous and selfish. “One motivation is great support for our colleagues, call it great customer service. But another is to avoid having to answer the same question—How do I reset my password? How do I install and set up OpenVPN?—a dozen times a month,” says Alberto. “If we can teach people to look to our knowledge base first, and if the results they get are helpful, they will learn that habit, and that will free my team up for the bigger challenges that require our personal attention.”
Integration as Innovation
Dean Ward, a senior developer on our architecture team who heads up work on billing systems, says he looks at Articles as a chance for a fresh start. “Normally I would be getting a ping in group chat apps or I would get an email thread. Very little of that information exchange makes its way back as productive additions to the original documentation.”
“I think the fact that Articles are linked to Q&A, it can drive you to keep everything fresh. It’s the little nudge. There are a lot of two way links between things,” explains Ward. For a developer, a wiki isn’t a place they want to spend time very often. “Articles, and traditional documentation, is not something you use everyday. For developers, Q&A can be something they touch every day, or a few times a week, and so that means your documentation has a much better chance of staying up to date.”
Ben Matthews, the engineering manager on Talent and Ads, says that even after a year on the job, he would ask a question and be directed to a piece of documentation that he never knew existed. Because knowledge was scattered across so many different platforms and so frequently exchanged in ephemeral conversations on chat or in person, he never felt confident that an answer existed or needed to be created.
Giving users the chance to enter text into a box and search with tags isn’t exactly breaking new ground when it comes to functionality. But that’s not the point. “In some ways it’s obvious. It feels like table stakes,” says Matthews. “But when you layer on the connection to Q&A, the integrations so you can search, ask or answer from within a work chat or email thread, and the gamification that adds just a little more incentive to the writer, overall it adds a lot of momentum and removes a lot of friction around content creation.”