\u003C/figure>\n\u003C!-- /wp:image -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>But back in 2008, building a community-driven Q&A site at this scale was far from being obvious. Instead, it fell somewhere in the \"complex\" quadrant (with some aspects in the \"complicated\" quadrant, like tackling the scaling issues we had). There were no good answers on how to build this yet, no experts who could show us the way. Only a handful of people out there faced the same issues.\u003C/p>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>For over a decade, we addressed our scaling issues by prioritizing performance everywhere. As one of our founders, Jeff Atwood, has famously said, “Performance is a feature.” For much of our existence, it has been the most important feature. As a consequence, we glossed over other things like decoupling, high cohesion, and test automation—all things that have become accepted best practices. You can only do so much with the time and resources at hand. If one thing becomes super important, others have to be cut back. \u003C/p>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>In this article, we walk through the choices we made and the tradeoffs they entailed. Sometimes we opted for speed and sacrificed testing. With more than a decade of history to reflect on, we can examine why best practices aren’t always the best choice for particular projects.\u003C/p>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:heading -->\n\u003Ch2 id=\"h-in-the-beginning\">In the beginning...\u003C/h2>\n\u003C!-- /wp:heading -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>When Stack Overflow launched in 2009, it ran on a few dedicated servers. Because we went with the reliability of a full Microsoft stack—.NET, C#, and MSSQL—our costs grew with the number of instances. Each server required a new license. Our scaling strategy was to scale up, not scale out. \u003Ca href=\"https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-2016-edition/\">Here’s what our architecture looks like now\u003C/a>.\u003C/p>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>To keep costs down, the site was engineered to run very fast, particularly in accessing the database. So we were very slim then, and we still are—you can run Stack Overflow in a single web server. The first site was a small operation put together by less than half a dozen people. It initially ran on two rented servers in a colocation facility: one for the site and one for the database. That number soon doubled: In early 2009, Atwood hand-built servers (two web, one utility, one database) and shipped them to Corvallis, OR. We rented space in the PEAK datacenter there, which is where we ran Stack Overflow from for a long time.\u003C/p>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>The initial system design was very slim, and they stayed that way for most of the site’s history. Eventually, maintaining a fast and light site design became a natural obsession for the team. \u003C/p>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:heading -->\n\u003Ch2 id=\"h-safety-s-off\">Safety’s off\u003C/h2>\n\u003C!-- /wp:heading -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>If you look at the programming languages that are used today, they fall on a spectrum of high and low-level based on how much that language abstracts the bare metal functionality. On the upper end of high-level languages, you have JavaScript: memory allocation, call stacks, and anything related to native machine code is handled transparently. On the other end, you have C: allocate and free memory for variables manually, no garbage collection, and doesn’t really handle \u003Ca href=\"https://stackoverflow.blog/2020/07/08/improving-performance-with-simd-intrinsics-in-three-use-cases/\">vectorized operations\u003C/a>. High-level languages provide safety but have a lot more runtime overhead, so can be slower. \u003C/p>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>Our codebase works the same way. We’ve optimized for speed, so some parts of our codebase used to look like C, because we used a lot of the patterns that C uses, like direct access to memory, to make it fast. We use a lot of static methods and fields as to minimize allocations whenever we have to. By minimizing allocations and making the memory footprint as slim as possible, we decrease the application stalls due to garbage collection. A good example of this is our open source \u003Ca href=\"https://github.com/StackExchange/StackExchange.Redis\">StackExchange.Redis library\u003C/a>. \u003C/p>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>To make sure regularly accessed data is faster, we use both memoization and caching. Memoization means we store the results of expensive operations; if we get the same inputs, we return the stored values instead of running the function again. We use \u003Ca href=\"https://stackoverflow.blog/2019/08/06/how-stack-overflow-caches-apps-for-a-multi-tenant-architecture/\">a lot of caching\u003C/a> (in different levels, both in-process and external, with Redis) as some of the SQL operations can be slow, while Redis is fast. Translating from relational data in SQL to object oriented data in any application can be a performance bottleneck, so we built \u003Ca href=\"https://github.com/StackExchange/Dapper\">Dapper\u003C/a>, a high performance micro-ORM that suits our performance needs.\u003C/p>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>We use a lot of tricks and patterns—memoization, static methods, and other tricks to minimize allocations—to make our code run fast. As a trade-off, it often makes it harder to test and harder to maintain.\u003C/p>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>One of the most noncontroversial good practices in the industry is automated tests. We don’t write a lot of these because our code doesn’t follow standard decoupling practices; while those principles make for easy to maintain code for a team, they add extra steps during runtime, and allocate more memory. It’s not much on any given transaction, but over thousands per second, it adds up. Things like polymorphism and dependency injection have been replaced with static fields and service locators. Those are harder to replace for automated testing, but save us some precious allocations in our hot paths\u003C/p>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>Similarly, we don’t write unit tests for every new feature. The thing that hinders our ability to unit test is precisely the focus on static structures. Static methods and properties are global, harder to replace at runtime, and therefore, harder to \"stub\" or \"mock.\" Those capabilities are very important for proper isolated unit testing. If we cannot mock a database connection, for instance, we cannot write tests that don't have access to the database. With our code base, you won't be able to easily do test driven development or similar practices that the industry seems to love.\u003C/p>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>That does not mean we believe a strong testing culture is a bad practice. Many of us have actually enjoyed working under test-first approaches before. But it's no silver bullet: your software is not going to crash and burn if you don't write your tests first, and the presence of tests alone does not mean you won't have maintainability issues.\u003C/p>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>Currently, we’re trying to change this. We're actively trying to write more tests and make our code more testable. It's an engineering goal we aim to achieve, but the changes needed are significant. It was not our priority early on. Now that we have had a product up and running successfully for many years, it's time to pay more attention to it.\u003C/p>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:heading -->\n\u003Ch2 id=\"h-best-practices-not-required-practices\">Best practices, not required practices\u003Cbr>\u003C/h2>\n\u003C!-- /wp:heading -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>So, what’s the takeaway from our experience building, scaling, and ensuring Stack Overflow is reliable for the tens of millions who visit every day?\u003Cbr>\u003Cbr>The patterns and behaviors that have made it into best practices in the software engineering industry did so for a reason. They make building software easier, especially on larger teams. But they are \u003Cem>best \u003C/em>practices, not required practices. \u003C/p>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>There’s a school of thought that believes best practices only apply to obvious problems. Complex or chaotic problems require novel solutions. Sometimes you may need to intentionally break one of these rules to get the specific results that your software needs. \u003C/p>\n\u003C!-- /wp:paragraph -->\n\n\u003C!-- wp:paragraph -->\n\u003Cp>\u003Cem>Special thanks to Ham Vocke and Jarrod Dixon for all their input on this post. \u003C/em>\u003C/p>\n\u003C!-- /wp:paragraph -->","html","2021-12-22T14:55:07.000Z",{"current":386},"best-practices-can-slow-your-application-down",[388,396,401,406,408],{"_createdAt":389,"_id":390,"_rev":391,"_type":392,"_updatedAt":389,"slug":393,"title":395},"2023-05-23T16:43:21Z","wp-tagcat-best-practices","9HpbCsT2tq0xwozQfkc4ih","blogTag",{"current":394},"best-practices","best practices",{"_createdAt":389,"_id":397,"_rev":391,"_type":392,"_updatedAt":389,"slug":398,"title":400},"wp-tagcat-code-for-a-living",{"current":399},"code-for-a-living","Code for a Living",{"_createdAt":389,"_id":402,"_rev":391,"_type":392,"_updatedAt":389,"slug":403,"title":405},"wp-tagcat-engineering",{"current":404},"engineering","Engineering",{"_createdAt":389,"_id":402,"_rev":391,"_type":392,"_updatedAt":389,"slug":407,"title":405},{"current":404},{"_createdAt":389,"_id":409,"_rev":391,"_type":392,"_updatedAt":389,"slug":410,"title":411},"wp-tagcat-performance",{"current":411},"performance","Best practices can slow your application down",[414,420,425,430],{"_id":415,"publishedAt":416,"slug":417,"sponsored":12,"title":419},"1d082483-6dc6-424b-8b09-9c84b54779da","2025-09-02T17:00:00.000Z",{"_type":10,"current":418},"back-to-school-developers-at-stack-overflow-have-some-advice-for-you","Back to school? Developers at Stack Overflow have some advice for you",{"_id":421,"publishedAt":416,"slug":422,"sponsored":12,"title":424},"5cd91820-9515-4be5-87ae-e919fd443c18",{"_type":10,"current":423},"getting-started-on-stack-overflow-a-step-by-step-guide-for-students","Getting started on Stack Overflow: a step-by-step guide for students",{"_id":426,"publishedAt":416,"slug":427,"sponsored":12,"title":429},"614538a9-c352-4024-adf1-fa44a9f911b6",{"_type":10,"current":428},"stack-overflow-is-helping-you-learn-to-code-with-new-resources","Stack Overflow is helping you learn to code with new resources",{"_id":431,"publishedAt":416,"slug":432,"sponsored":12,"title":434},"763b1d36-83d8-4178-9c2d-32d705ea1d7b",{"_type":10,"current":433},"introducing-your-newest-study-buddy-stackoverflow-ai","Introducing your newest study buddy: stackoverflow.ai",{"count":436,"lastTimestamp":437},54,"2023-08-29T09:03:26Z",["Reactive",439],{"$sarticleModal":440},false,["Set"],["ShallowReactive",443],{"sanity-A5zojsdY1hsQ5q3eu1Jnw-tIRFAq2nhtqMZ2diag-1Q":-1,"sanity-comment-wp-post-17556-1757323503664":-1},"/2021/12/22/best-practices-can-slow-your-application-down"]