How Do Software Developers in New York, San Francisco, London and Bangalore Differ?
When I tell someone Stack Overflow is based in New York City, they’re often surprised: many people assume it’s in San Francisco. (I’ve even seen job applications with “I’m in New York, but willing to relocate to San Francisco” in the cover letter.) San Francisco is a safe guess of where an American tech company might be located: it’s in the heart of Silicon Valley, near the headquarters of tech giants such as Apple, Google, and Facebook. But New York has a rich startup ecosystem as well- and it’s a very different world from San Francisco, with developers who use different languages and technologies.
Here on the Stack Overflow data team we don’t have to hypothesize about where developers are and what they use: we can measure it! By analyzing our traffic, we have a bird’s eye view of who visits Stack Overflow, and what technologies they’re working on. Here we’ll show some examples of what we can detect about each city based on one year of Stack Overflow traffic.
In this post we’re going to focus on the four cities that visit Stack Overflow the most: San Francisco, Bangalore, London, and New York. (In this analysis, we counted all traffic within 50 miles of a city: this means San Francisco includes a larger part of the “Bay Area”, such as Mountain View and Cupertino).
San Francisco vs New York
First we’ll compare the two most popular American cities for software development: San Francisco and New York.
When developers are using a programming language or technology, they typically visit questions related to it. So based on how much traffic goes to questions tagged with Python, or Javascript, we can estimate what fraction of a city’s software development takes place in that language.
For example, there were 187 million question views from San Francisco in the last year, and we can see that 12.8% of these visits were to questions with the Python tag, compared to 10.3% of New York’s traffic.
Most of these common technologies look like they make up a fairly similar fraction of NY and SF traffic, but we’re interested in stark differences. What tags (among the 200 most high-traffic tags) showed the largest difference between San Francisco and New York?
One clear difference: New York has a larger share of Microsoft developers. Many tags important in the Microsoft technology stack, such as C#, .NET, SQL Server, and VB.NET, had about twice as much traffic in New York as in San Francisco.
There are also patterns in the technologies that are more common in the San Francisco area, especially languages developed by Apple (Cocoa, Objective-C, OSX) and Google (Go, Android). We can also see several influential open source projects, especially ones associated with Apache (Hive, Hadoop, Spark).
Rather than looking only at the most dramatic changes, we could visualize the SF/NY ratio compared to the total visits:
This confirms that C# (in NY) and Android (in SF) stand out as the highest traffic tags that show different behavior, with tags such as Excel, VBA, Cocoa, and Go showing more even dramatic differences. Meanwhile, the Java tag has about the same level of traffic in each city, as do several “language agnostic” tags such as “string”, “regex”, and “performance”.
New York, San Francisco, Bangalore, and London
Let’s expand the story to include Bangalore, India, and London, England. Together these four cities make up 11.1% of all Stack Overflow traffic.
Each of these cities is the “capital” of particular tags, visiting them more than the other three cities do. Which tags does each city lead in?
This fills out more of our story:
- London has the highest percentage of developers using the Microsoft stack: while New York had more Microsoft-related traffic than San Francisco, here we see London with a still greater proportion. Since both London and New York are financial hubs, this suggests we were right that Microsoft technologies tend to be associated with financial professionals.
- New York leads in several data analysis tools, including pandas (a Python data science library) and R. This is probably due to a combination of finance, academic research, and data science at tech companies. It’s not a huge lead, but as an R user in New York I’m still personally happy to see it!
- Bangalore has the most Android development, with two to three times as much traffic to Android-related tags as the other three cities. Bangalore is sometimes called the “Silicon Valley of India” for its thriving software export industry, with Android development playing the largest role.
- San Francisco leads in the same technologies as it did in the comparison with New York (except for Android). In particular (thanks to Mountain View), it’s indisputably the “Go capital of the world.” (This is true even if we look at the 50 highest-traffic cities rather than just the top 4).
This portrait of four major developer hubs is just one of many ways Stack Overflow traffic can tell us about the global software engineers ecosystem. If you’re a developer looking for work in these cities, we have listings in New York, San Francisco, and London.
63 Comments
That’s a nice blog, David. Let’s hope to see more of those soon! Thanks
Nice post! Can we get a tool to get this analysis for more cities? I’m curious about LA.
Not sure if this tool will be publicly available, but there’s a live updating map over at https://talent.stackoverflow.com/about-targeting (I too am curious about LA, FWIW.)
I’d like to see at a minimum a top 50; being Tokyo-based, I’m curious to see where it featured. Seoul managed 17th, so I wonder if that says anything about the relative English-language skills of Korean versus Japanese developers.
US has 7 cities out of top 20 programmer’s cities & India has 6 cities out of top 20 programmer’s cities. 🙂
Waiting for Ahmedabad, India to be in Top 20 list.
Top traffic on SO, not top programmers cities. A theoretical metropolitan city could be composed almost entirely of individuals who know next-to-nothing about programming but are suddenly struck with a simultaneous compulsion to learn, and thus that city would rank rather high in SO’s traffic. Another city could be composed of best-in-the-world programming experts who get paid more than most CEOs in other cities, alongside brilliant young minds who want to get their hands dirty, along with a plethora of local meetups & user groups where the old-timers are more than happy to share their knowledge with young’ins first-hand. That city would likely rank very low in SO’s traffic rankings.
Looking at you, New York tags.
San Francisco? Why not do Silicon Valley but do San Francisco?
If you read the post (and not just the charts 😉 you’ll see SF is shorthand for most of the Bay Area.
Out of curiosity, you wouldn’t happen to be able to say how far down Metro Atlanta was on the SO traffic list, would you?
Sure! Looks like it is 35th worldwide (little below Dallas, little above Madrid), and 9th in the United States (beating Philadelphia at 10th).
Is this data publicly available? 🙂
Please let it be!
Yep shock horror! AU is not on that SO list.
Where is the list then? That it’s not on the chart (https://i.stack.imgur.com/cRPva.png) is no news flash so I assume you’re not talking about that.
I’m talking about the chart that you pasted.
I should be more specific, because I was trying to say Brisb, Perth, Melb and Syn is not on it… Which I thought it should.
Also, it is unfair to include Singapore because it is a country! Just small..
Those Austalian cities (Brisb, Perth, Melb and Syn) must not account for that much SO traffic. Not hard to understand considering population.
Also, Singapore is a city as well..
How far down on the list are the Australian cities Sydney, Melbourne, Brisbane, Perth and Adelaide?
Interesting. powershell is top in london.
great post, interesting that swift traffic doesn’t make a blip (probably still dwarfed by objective-c).
I was asked this on Meta http://meta.stackoverflow.com/q/336388/447156
Very interesting. These are ‘big cities’ – you’ve chosen metro areas with the most visits to SO. Could you analyse other areas that might be of interest, such as startup cities? In terms of startups, you’ve got London but not Berlin or Tallinn, for example. (Cologne, Helsinki and Stockholm might be of interest too for fast-moving IT cities. I admit I’m showing a European bias here, but I find the IT landscape can be surprisingly different outside America, something that’s shown up partially here with your inclusion of India.)
But there may be other groups of cities – for example, is there a different among cities in India? Asia as a whole? Australia vs US/Canada/England? Germany, Russia and the US? South America?
Are you claiming that Cambridge is in London? What about Oxford….
Depends of course on which point is used for measuring both London, and how aggregated the user data is (So Cambridge might be 49 miles, but Histon might be 51 miles, but is Histon considered part of Cambridge for that?). Arguably, for the UK, a better approach would be to use the Travel to Work areas that get generated from Census results [ http://ons.maps.arcgis.com/apps/MapSeries/index.html?appid=397ccae5d5c7472e87cf0ca766386cc2 ]
Have you looked at tags on job adverts and CVs based on locations?
Wonder! to not see the tags of JS frameworks
There’s probably a highly distributed use of JS frameworks, since JS is still largely a front-end language, work with it can be done remotely/from anywhere; a web firm in Tennessee can service companies in NY, in Africa, in Russia, etc. So JS developers likely won’t be centralized in large groups (discounting a bit of clustering in large cities anyway due to higher population concentrations).
That’s true of all web technologies, and arguably all programming technologies, no?
No, I believe it’s not as applicable (but not 100% non-applicable) to non-web languages. Companies or environments where desktop-based development tends to happen on-site way more often. Either you have a group of in-house developers, or you contract out to a development firm in your town, or both. A lot of companies want to move or have a presence in the towns where the big software companies are located for this reason (in addition to being where the talent is–another big reason). With web-based programming, there’s no need for on-premise work, because the product is 100% virtualized the entire time.
You can see angularjs in the 4th diagram of this article. 🙂 On the right hand side for the chart, close to JQuery and php
A nice blog indeed, i would also like to take a closer look at the stats if your tool and the datas are openSource.
Do you have an api endpoint we can send requests to?
really nice take on what the developers around the globe are focusing on!
Exactly the reason I dissociated completely from VBA. Recruiters around London won’t let you live if you’re within the first 10 on SO within the #VBA.
Awesome! Would you make an public access to a live version of this data so that people can play with it and look for their fav techs? (for instance, I’d be interested in how Haskell takes place in this picture).
Are there really no developers in Alaska? The map that Jon Ericson doesn’t show a single one!
And one in montana?
That map is a live view, showing a dot whenever we get a visit from a developer in the town at that dot. So if you keep it open long enough, you’ll see dots appear in (for example) Bozeman, Missoula, Billings, etc.
Great, great analysis. I would love to see the same for the 4 sons of StackOverflow in different languages. Well, I’m really interested in Spanish, but I think people on all of them would enjoy seeing something like this.
I’d be curious about how market share demographics might affect these seeming city-specializations. For example, maybe SO gets a disproportionate volume of Microsoft-related traffic because it’s actually a less popular technology stack overall.
It could be that UK-based developers are more likely to use UK-based Q&A sites in general, but find the volume of local support and mindshare around the U.S.-based Microsoft technology stack lacking. Thus, a greater volume of Microsoft-focused UK developers might turn to SO’s more U.S.-centric community than would UK developers overall.
All just speculation, of course, and driven by suspicion of the idea that U.S.-owned Microsoft’s largest developer market share would be in a foreign developed nation. After all, it’s not like iOS, for example, which is a prerequisite to reaching a certain platform/user base, and a market mostly populated by casual software not dealing with sensitive information.
Interesting article and stats. Thank you. Just a note. Bangalore also has several MNCs, some with >20K employees working on other technologies: ex: maintenance projects, mainframes, etc. Most or almost half of this number will NOT have internet access. So the numbers you are seeing are typically from hobbyists and startups – that probably is why there is a skew towards android and front-end development.
That’s interesting! Would you expect a proportion of these developers to visit on mobile, meaning we might get a sense of their breakdown by filtering for mobile traffic?
Less likely to have mobile access if Internet is blocked.
Great stuff, I’d love to see more of this kind of stuff. As a Londoner and loather of M$ I’m so depressed by this! Powershell and VBA, Jeez: how utterly, utterly embarrassing. Fortunately I don’t actually work in IT here.
Fascinating to see how big Indian coders are: Delhi ahead of Paris but other Indian cities too.
All this is showing is that a higher percentage of stack overflow users in London use Powershell and VBA then in other “IT cites”. This is to be expected as London has a lot of cooperation head offices with IT departments who will be supporting Excel etc and doing a lot of large scale system admin with Powershell.
I expect that the USA “IT cites” don’t have nearly the same number of none IT corporations based in them.
We don’t have a break down between professional developers working within a development team and none developers who have to do a little programming on the side to get their job done.
Really mature annotating a company by $ symbols.
Interesting. It’s important to remember that SO users are probably not a random selection of developers: many corporates would instantly fire a developer who discussed anything to do with their work on the internet. Also, the questions people ask on SO are not a random selection of problems that developers face; there will be many questions where they will turn first to colleagues or suppliers.
Disagree with the first point — many times you have a generic language question and that’s when you come to SO, nothing to do with your actual code base. You also probably read other’s questions a lot more than you ask your own. Agree with the second point though; this probably represents junior devs than it does more senior ones.
This data isn’t about questions asked, it’s questions visited.
I’m surprised Atlanta isn’t included
I am surprised there’s no Germany
Very interesting!
Wow, this is a cool analysis.
Very interesting article…though seeing pandas in list was overwhelming to me!
San Francisc0 really is going hard on New York , NY is losing in s0 many ways
Bangalore is leading in Android tags. In actual, it leads in Java as well. Hence, Android makes sense. Great analysis by the way!
it does not depict the technology used, it says people are struggling with issues, I am into .net and rarely visit stack overflow for .net
Our experience has been that most Stack Overflow traffic is coming from professional developers who have gotten “stuck” somehow and are trying to clear a roadblock in their code. What resources do you use when you hit a roadblock in .NET?
take my example, I am into .net rarely struggle in .net, I am learning mobile dev, for this i frequently visit stack, this will count me in mobile not in .net. I know Bangalore ground reality, if there is a walk-in for .net for any level, there will be 5K candidates at interview.but here it shows Blr leads in android, not quite agreeing 🙂 . but report is good one, so many new trends come to know.
I would be interested to know how the ‘within 50 miles’ was defined. Is that 50 miles from the centre of the city or 50 miles from the outskirts. It would make the different between ‘London’ covering the whole of the Thames Valley (and half of Oxford and Cambridge *8′) or all of it.
It’s 50 miles from the center to the center of other cities (our data is aggregated by city). Having checked now, Cambridge is just slightly within the 50 mile radius, but Oxford is not.
Glad to know that Bangalore is in second position.
insightful. Nice blog