Server

community May 17, 2011

The Speed of Light Sucks

Our current datacenter is in New York City. Yep, where they make all that great salsa. So whenever you make a request to any Stack Exchange site, the internet tubes must connect from your location to our datacenter in NYC. We are not (yet) immune to the laws of physics, so depending on the distance…
Avatar for Jeff Atwood
Co-Founder (Former)
company March 8, 2011

PEER 1 Hosting – Making your data center more awesome!

In 2003, Fog Creek Software (aka Joel’s other baby) moved offices, and decided to ditch its internal T1 and look for a colocation provider. Joel was impressed with PEER 1 Hosting’s customer service, the shiny new data center in NYC, and PEER 1 Hosting even volunteered to host Joel on Software – for free! When we…
Avatar for Alison Sperling
Director of Marketing (former)
company October 30, 2010

Database Upgrade

As part of our datacenter migration, the database server received a substantial upgrade: Oregon 48 GB 2 Xeon X5470 CPUs 8 total cores @ 3.33 Ghz NYC 64 GB 2 Xeon X5680 CPUs 12 total cores @ 3.33 GHz However, a few things didn’t go quite to plan in the migration. Much to our chagrin,…
Avatar for Jeff Atwood
Co-Founder (Former)
company October 22, 2010

Datacenter Migration Oct. 23

This Saturday, October 23rd, starting at about 2 PM Pacific, we will be migrating all of our primary sites from the Corvallis, OR datacenter to the New York, NY datacenter. Please be advised that this is a major move, and while we will do everything we can to prevent major service interruptions (largely with a…
Avatar for Jeff Atwood
Co-Founder (Former)
company October 9, 2010

Stack Overflow Outage

As you may have noticed many of Stack Overflow’s websites suffered some down time today from 6am EDT for about an hour and Stackoverflow.com is still offline for maintenance. Our collocation provider in Oregon experienced an unexpected UPS failure that caused us to lose power. Once they were able to restore power Geoff, who was already on…
Avatar for Kyle Brandt
Director of Site Reliability (former)
company August 29, 2010

Stack Overflow Sponsors HAProxy

As you can see from our network configuration, HAProxy is a much beloved part of our infrastructure. Willy Tarreau, the author, has been extremely responsive and helpful to us in the past. So when we reached a surprising dead-end in our quest to find a reverse proxy that could block HTTP clients using too much…
Avatar for Jeff Atwood
Co-Founder (Former)
company February 7, 2010

Thermal Event at Datacenter

Our hosting provider, PEAK, let us know that they had a cooling compressor fail in the facility. The primary database server was apparently taken offline at 2:53 AM Pacific Time by this thermal event. The backup database server is still online and has the most recent (12 AM) backups restored to it; we’re currently just…
Avatar for Jeff Atwood
Co-Founder (Former)
background January 25, 2010

Six Whys – Or, Never Trust Your Network Switch

Remember Joel Spolsky’s fine article “Five Whys”? Sure you do! It contained this paragraph: Michael spent some time doing a post-mortem, and discovered that the problem was a simple configuration problem on the switch. There are several possible speeds that a switch can use to communicate (10, 100, or 1000 megabits/second). You can either set…
Avatar for Jeff Atwood
Co-Founder (Former)
background January 14, 2010

Stack Overflow Network Configuration

I think we’ve finally arrived at a semi-stable network layout for running Stack Overflow and the rest of the trilogy. Here’s a diagram of our current network layout: The most recent changes were all in the name of redundancy: move to dual HAProxy routing instances add a third Stack Overflow server to the HAProxy rotation…
Avatar for Jeff Atwood
Co-Founder (Former)
company December 18, 2009

Scheduled Site Maintenance – Saturday

This Saturday, December 19th, from 4 PM to 8 PM PST, all trilogy sites and the blog will be down for a spot of maintenance. What are we doing? A few things, several of which were inspired by comments on our Server Rack Glamour Shots: Upgrading one server which does a lot of VM utility…
Avatar for Jeff Atwood
Co-Founder (Former)
background December 14, 2009

Stack Overflow Rack Glamour Shots

We installed our secondary (backup) database server tonight. Geoff took the opportunity to snap a few glamour shots of the Stack Overflow server rack at our host PEAK Internet. I present them here for your unbridled enjoyment and pleasure: update: Based on feedback from this post, we went back and improved our rack hygiene: These…
Avatar for Jeff Atwood
Co-Founder (Former)