[Ed. note: While we take some time to rest up over the holidays and prepare for next year, we are re-publishing our top ten posts for the year. Please enjoy our favorite work this year and we’ll see you in 2025.]
In the Developer Coefficient, a study commissioned by the fintech giant Stripe, developers reported that they spent over 17 hours a week on maintenance tasks like debugging and refactoring—work classified as “toil.”
Our 2024 Developer Survey revealed a lot of the same issues and grievances for professional developers. The biggest frustration, by a large margin, was technical debt. Conversely, the thing that made developers happiest was improving the quality of their code and developer environment. And looking to the future, the two areas where developers felt they would get the most value out of GenAI tools were code testing and documentation.
We explored the emerging trend of AI-generated tests in a recent piece, and want to continue the discussion by looking more deeply at modern practices for documentation.
To what degree does great documentation help cut down on the “toil work” and technical debt that leads to frustration and burnout among developers? And to what degree can it support the things that make them happiest, like code quality?
Does Documentation Really Impact Toil Work?
Before we dive into what role AI-generated documentation might have, there’s empirical evidence that good developer docs have a positive impact on work like refactoring or debugging.
A meta-study of over 60 academic papers on software quality and documentation found that the benefits are reflected in many aspects: shortened task duration, improved code quality, higher productivity, or any other improvements related to software development, and research finds that documentation often takes up 11% of developers’ work hours.
This study from PLOS ONE, published in 2023, developed a model to test what techniques would have a positive or negative impact on the process of refactoring. The authors write that “documentation assists in onboarding new team members and ensures consistency in refactoring practices across the team.”
Documentation was especially valuable when it came time to refactor code by providing a blueprint that saved time and improved focus. The researchers found that good documentation “ensures that refactoring efforts are directed towards tangible and specific quality improvements, maximizing the value of each refactoring action and ensuring the long-term maintainability and evolution of the software.”
As our co-founder Joel Spolsky put it, documentation encodes generational wisdom that goes beyond the simple specs of what was built. “Think of the code in your organization like plumbing in a building. If you hire a new superintendent to manage your property, they will know how plumbing works, but they won’t know exactly how YOUR plumbing works,” said Spolsky. “Maybe they used a different kind of pump at their old site. They might understand how the pipes connect, but they won’t know you have to kick the boiler twice on Thursday to prevent a leak from springing over the weekend.”
Challenges in Maintaining Effective Documentation
If we know from decades of research that documentation is a key component of creating and maintaining quality code, why is it so often considered low-priority work developers would rather avoid if they can be writing code instead?
In fast-paced development environments, particularly those adopting Agile methodologies, maintaining up-to-date documentation can be challenging. Developers often deprioritize documentation due to tight deadlines and a focus on delivering working code. This leads to informal, hard-to-understand documentation that quickly becomes outdated as the software evolves.
Another significant issue is that documentation is frequently viewed as unnecessary overhead. Developers may believe that code should be self-explanatory or that documentation slows down the development process. However, this mindset contributes to a steep learning curve for new team members and increases the time spent on maintenance tasks, as developers must spend additional time understanding poorly documented code.
Outdated or insufficient documentation exacerbates technical debt. As the codebase changes, documentation that isn’t updated can mislead developers, leading to errors and inefficient workflows. This not only disrupts developers’ flow state - the mental zone where programmers are most productive and creative - but also contributes to frustration and burnout.
There is a terrific study on this topic, aptly titled: Why do developers struggle with documentation while excelling at programming? It includes interviews with 17 developers with a range of experience and is worth a deep dive to understand the forces that often get in the way of good documentation.
There is an emerging discipline known as documentation engineering that tries to bring the acts of writing and coding closer together, bringing the work of documenting code more in line with the style and goals of an engineering department.
“Writing good documentation is key infrastructure work,” argues Fabrizio Ferri-Beneditti, a documentation engineer who spent years as a technical writer at Splunk. “Remove documentation and your products cease to exist, their inner workings left to the imagination of users who have better things to do than divining the behavior of API endpoints or CLI options.”
To prevent documentation from becoming a second-class citizen in the software development lifecycle, Ferri-Beneditti argues that documentation needs to be observable, something that can be measured against the KPIs and goals developers and their managers often use when delivering projects.
He tells the developers he works with, “Something that will remove problems after launch, for example, or reduce the support burden or help sell the latest feature because that feature will perform better and have better customer support. You really have to understand what picks the interest of developer in each case, or at an organization or team level.”
Is AI-Generated Documentation a Good Answer?
And to what degree should we keep humans in the loop?
There are several different ways that a software developer or development team could approach this problem. We’ve chatted before with Itimar Friedman of Codium, which has a feature called Read the Docs built for this specific use case. It integrates with version control systems like GitHub, GitLab, and Bitbucket, then automatically fetches the latest code changes and triggers documentation updates. This feature tries to ensure that the documentation remains up-to-date with the latest codebase, reducing manual intervention and saving developer time.
GitHub CoPilot also has this feature. In the video below, you can see how the AI system works sort of like a predictive engine, taking a prompt from the developer and then extending the thought and adding relevant details. This is a more hands-on approach than what Codium is doing, but can still save a lot of time if the developer typically finds writing docs to be a tedious chore.
One key element of toil work is its disruption of the developer’s flow state: the mental zone where programmers are most productive and creative. Interruptions like hunting down missing documentation or clarifying confusing code pull developers out of this flow, contributing to burnout and lower job satisfaction. We confirmed this in our recent developer survey, where respondents spent more than 30 minutes a day searching for solutions to technical problems. Given how crucial documentation is in reducing technical debt and maintaining code quality, automating its creation is a natural solution.
GenAI systems have emerged as a powerful tool in addressing this challenge. These AI systems can automatically generate documentation either in real time as the developer writes code or after the fact by reviewing the codebase. While some offer generic suggestions based on their training data, others are finetuned on your particular codebase or given your code and docs as data that can be queried by a system using retrieval augmented generation (RAG).
By offloading the burden of documentation creation onto AI, developers are free to stay in their flow state, focusing on the tasks they enjoy—building and problem-solving—while still ensuring that the documentation remains comprehensive and up-to-date.
Perhaps most importantly, this synergy between GenAI and human developers does not remove human oversight. Developers remain in control, guiding the AI and reviewing the output for accuracy, which ensures that the final product aligns with their intentions. This partnership preserves a sense of agency while eliminating the toil of tasks that many developers would prefer to avoid. In this way, AI enables human coders to focus on higher-value work, enhancing both productivity and job satisfaction.
Best Practices for Documentation in an Agile Environment
Leveraging modern tools and techniques to keep documentation relevant
It’s important to look at these tools as enablers of new best practices that humans still manage, not as robots that handle all the work by themselves.
To tackle the work of creating and maintaining good documentation, several best practices have emerged for maintaining effective documentation in continuous software development (CSD). One approach is to use modern tools that automate documentation retrieval and generation. Tools that integrate with the development environment can automatically update documentation as the code changes, ensuring that it remains current without requiring significant manual effort.
Another practice is the implementation of executable documentation that evolves alongside the code. This includes techniques like literate programming or using tools that generate documentation from code comments and annotations. By embedding documentation within the code itself, developers can ensure that it stays synchronized with the codebase.
Adopting a strategy of minimal upfront documentation with detailed design documentation afterward can also be beneficial. This allows teams to remain agile and responsive during development while still capturing essential information for future reference. Such practices support better knowledge transfer and help reduce the toil associated with onboarding and maintenance.
Leveling Up With Automated Labor
Learn to delegate and oversee tasks that don't energize you
Even AI-first companies know the pain of outdated docs. “It’s a problem we know well and every company does. Engineering docs get out-of-date pretty easily and it's hard to stay in sync with the code,” says Matt Zeiler, co-founder and CEO of Clarifai. Zeiler, who studied with luminaries like Geoff Hinton and Yann Lecun, sees a natural evolution on the horizon. “I do think given all these co-pilots for code development, a future where that is in sync with your documentation is pretty likely. If it can generate the code, it can generate the docs and keep them up-to-date."
Using AI assistance to reduce toil work and improve overall code quality is quickly emerging as a key element of a great developer experience. GenAI systems offer a powerful means to address this challenge. By automating the creation and maintenance of documentation, they free developers to stay in the flow state, focusing on tasks they enjoy while ensuring that essential documentation is not neglected. This not only enhances productivity but also improves job satisfaction and code quality.
Stack Overflow for Teams is our natural choice for knowledge management here at Stack Overflow. Stack Overflow for Teams allows for the centralization of code review documentation and relevant information for engineering and technical teams.
By embracing AI-powered documentation tools, development teams can significantly reduce toil work, mitigate technical debt, and foster an environment where developers can thrive. Wise organizations will also keep humans in the loop, ensuring that documentation engineers or technical writers act as editors and stewards of any AI-generated documentation, preventing errors or hallucinations from creeping into otherwise accurate docs.
As the software industry continues to evolve, integrating these tools and practices will be essential for maintaining competitiveness and achieving long-term success.