Backstage is an open-source IDP by Spotify that reduces everyday friction, cognitive overhead, and operational toil for developers.
We previously talked to the Backstage team in 2022.
Poor Ryan. If only he had Backstage at his last job.
You know what makes Backstage even better? Our Stack Overflow for Teams integration.
Connect with Pia on LinkedIn.
Markus Pscheidt gets today’s shoutout for winning the Populist badge with their answer to Dynamic tag values for the Counter metric in Micrometer.
TRANSCRIPT
[Intro music]
Ryan Donovan: Hello everyone, and welcome to the Stack Overflow Podcast, a place to talk all things software and technology. I'm Ryan Donovan, your host today, and we are touching base again with the Backstage team at Spotify. My guest today is Pia Nilsson, the GM for Backstage and the head of Developer Experience. So welcome to the show, Pia.
Pia Nilsson: Thanks so much, Ryan. Glad to be here.
Ryan Donovan: So, before we get started, we like to get to know our guests, find out how you got into software and technology.
Pia Nilsson: Oh my God. So, how I got into it– I think I fell backwards into technology. I thought I would become a social psychologist—a researcher—but then I happened to take a programming class way back in the time when I was studying, and I completely fell in love with it, and it changed my life; and then I became a software engineer for 14 years; and then moved into leadership a few years from now. So, that's what I've been doing at Spotify: leading developers – as, sort of, I gravitated towards platform engineering over the years, as I love building things that last.
Ryan Donovan: Mm-hmm.
Pia Nilsson: And I was a feature developer of course, for many, many years, and I kept feeling every new place I went, like, where have they figured it out – how to build beautiful software? And it was difficult to find, I must say. And I also didn't find it at Spotify, but, I think we have come a long way since then and so that's sort of a 'what I'm doing here', basically.
Ryan Donovan: Okay. Well, we talked to some folks from the Backstage team way back in September, 2022. I'm sure plenty has changed with Backstage and with developer experience, in general. What are the sort of things that you all are thinking about, and have been thinking about for the last three years?
Pia Nilsson: That's a lot.
Ryan Donovan: So, that's a big open question, yeah. It's your field to run in.
Pia Nilsson: So, Backstage has been growing organically, just like– like crazy. It's such a wonderful journey to be part of. We now have 3,400 adopters, and those are only the ones we know of – of course, there are so many more than that, but we know that there are 3,400 companies using Backstage now, and among them, of course, very many of them are Fortune 500. So, we have: Airbnb, Booking.com, H&M, Toyota, Lego.
Ryan Donovan: Mm-hmm.
Pia Nilsson: Many, many others. So, we keep talking to them all the time and we are learning so much from them as they are from us, I believe. So that's, sort of, just a lovely journey of learning. I think learning platform engineering together, with Backstage as the vehicle, and then over these years, so many of them are asking for more support from us, as we have a few more years in the business of developer portals. So, we have been shipping our products to them as well, as they have been asking for some of our battle-proven products internally. And lately, we're also shipping Backstage as a SaaS, and the way we would run it—so you get the templates and so much guidance and opinionated Backstage—basically, only to get yourself bootstrapped into faster developer experience so that the developers at all of these companies can focus on the problems in the company, not sort of, running an IDP. But the core of the product and the heart of the idea is the same, as we believe in this open platform for folks.
Ryan Donovan: Yeah. I mean, I remember when I was exposed to it. My previous job: I had some amount of—I was an internal tech writer—some amount of developer experience management, but just discovering all the services and who owns what was such a chore, such a challenge. Is that changing? I know we had the Pandora's box moment of ChatGPT, and AI is everywhere. Has that changed developer experience?
Pia Nilsson: Oh my god, yes. AI is changing developer experience– it feels like every week? But maybe, maybe not that often, but it's a massive shift going on in the developer experience business and domain. It's super exciting, and AI is, of course, the center of it. What it means is that an IDP becomes even more important.
Ryan Donovan: Mm-hmm.
Pia Nilsson: Remember we built Backstage because we had an exponential component growth way back in 2016, which feels like when the dinosaurs were around. Like, we saw an exponential growth: components outnumbering the number of humans we could hire, back in 2016. That's just gonna be, you know, 10 x-ed, 1000 x-ed, with AI, and the code generators that now exist—and we are using them at scale at Spotify—like Windsurf, Cursor, and Claude Dev, and all of these, and they will just change the world for sure. So, what that means, of course, is many things, but for one: more components, more code to keep track of, to keep compliant, to keep secure, to keep automated at, and to own.
Ryan Donovan: Mm-hmm.
Pia Nilsson: Right. So, having an IDP keeping track of all of this is more important than ever.
Ryan Donovan: Right. I mean, with all that, I imagine you have agents to keep track of best practices for prompts, to keep track of templates. I mean, you mentioned three different coding tools. Are there sort of ways to track usage in each? Are there ways to, sort of, set up template prompts for each?
Pia Nilsson: Great question. So, we have an approach with what we do at Spotify, as in: meet the developer where they are.
Ryan Donovan: Mm-hmm.
Pia Nilsson: Which means: we are using the IDP as the vehicle to drive change, and AI is the change we need to be driving in all platform engineering domains, but that doesn't mean everyone needs to go to the IDP for every single task. There are many surfaces.
Ryan Donovan: Right.
Pia Nilsson: Not all that many, but at least five or six ‘developer surfaces,' we call them. And we need to meet—we, that care about developer experience and developer speed— need to meet the developer where they are so that it becomes super useful for them. Now, how we think we should do that at Spotify, and also in the commercial offer, of course, is really ‘decorate,’ if you will, the knowledge that I have as an engineer in my IDE with the backstage ecosystem component knowledge. And the entire, sort of, metadata on all the data components, and the software component, and the people, of course, and the ownership, and all of that. So, then the IDE is sort of empowered to write code that is possible to standardize, for example.
Ryan Donovan: Mm-hmm. And when you say ‘decorate,’ do you mean that in a sort of metaphorical sense, or are you actually using decorators in code that are expanded out?
Pia Nilsson: Yeah, I was meaning the code.
Ryan Donovan: Yeah. Okay. So the Backstage automatically picks that up, integrates it...
Pia Nilsson: Yeah. So we have—we are shipping an AI portal, or AI platform, inside of portal—and, of course, internally as well, which both is an agent—it's a chat UI of course, in the web UI, but it's an agent as well—able to ingest MCPs, and it's an MCP itself, able to be ingested someplace else, right? So, one has to be, in our view, have this flexibility when you're addressing developer experience. It's not about an IDP. IDP is the vehicle and the hub that ties everything together, but then you need to provide the knowledge where the developers are. And, that's also – the tools sprawl that we're now gonna see, and already are seeing in AI, is just gonna get worse. And sort of, Backstage, I think, has a great role to play there, to keep track of everything, but also to enable these tools so that we do stand for an open platform.
Ryan Donovan: Yeah. You mentioned the ‘tool sprawl.’ I feel like the developer experience and IDPs are sort of getting ingested in this larger platform engineering space, right? Where it is a, you know, ‘DevOps plus tools,’ sort of situation. How do you feel like Backstage is fitting into that platform engineering space?
Pia Nilsson: Well, when we're thinking about developer experience – at Spotify, we primarily talk about developer experience, and then one might wonder, 'well, why aren't you talking about platform engineering?' And of course they are almost interchangeable, but there is a little distinction that I'd love to point out…
Ryan Donovan: Right.
Pia Nilsson: So, developer experience is, sort of, the approach, because what we are all here for—we, sort of, ‘platform people’—is to make the company more effective and increase the satisfaction level of the employees – that usually goes hand-in-hand. So, it's the developer experience that's the goal, right? Platform engineering, which can be used interchangeably, of course, but platform engineering—if you are a little clean in the definition—is the toolbox. Like, it's the CI, CD, monitoring all of the software lifecycle domain… That's the toolbox. And, you know, all of the vendors exist there; but if you have a great platform engineering toolset in a company, it does not necessarily mean that you have a great developer experience, because they might not be linked or connected in a great way.
Ryan Donovan: Mm-hmm.
Pia Nilsson: So, you're actually not gaining that much speed, because it– you need to have a sort of interconnected journey. So, we usually talk about developer experience really intentionally, because we want to put people's minds on, 'what is the path? What is the– what is the journey? What is the job to be done by the developer?'
Ryan Donovan: Yeah.
Pia Nilsson: We don't necessarily need to, sort of, focus on CI. That's just a means to an end, you know?
Ryan Donovan: Right. That interconnection is– it's a surprisingly hard thing – just getting everybody to understand what the sort of procedures are, what… just, even things like 'service discovery' is always a challenge. How are you thinking about maintaining that and, you know, integrating those tools, and making those tools more connected within an IDP?
Pia Nilsson: That's a great question. So, our approach is: we really believe that the domain of engineering and platform engineering will always move faster than any closed-source company can. So, that's why we really believe in an open platform. Now, even if one goes to our SaaS offer, it's entirely based on the Backstage open source. So, it's absolutely adhering to that principle, still. And why? It's because of—your question here is great—because how else are you gonna have the next big AI vendor integrated?
Ryan Donovan: Right.
Pia Nilsson: They need to integrate themselves and, usually, hopefully own that integration themselves, because then it's gonna be stellar.
Ryan Donovan: Mm-hmm.
Pia Nilsson: And we're, see, that's what we're seeing now with the Backstage plugin ecosystem – it really works. We have 115 plugins—last time we counted, a few weeks ago—which I think says a lot. And most of them are, most of the big DevX vendors own a Backstage plugin that they maintain. So, I think that's how you, sort of, integrate. And then on top of that, one wants, sort of, these plugins to interrelate as well. That's when stuff becomes really pretty, I think.
Ryan Donovan: Yeah. I know that plugin ecosystem is powerful. We have, with our 'Stack Overflow for Teams' product, we have an increasingly intimate integration. I know we released something recently about it, but that—you know, like you said—it's that interrelation. Like, there's so many pieces out there moving in a modern, you know, distributed software company. Do you think it—you know, it may be a spicy question—do you think there's a point where there's too many tools, where somebody needs to pair down tools?
Pia Nilsson: Yeah. I don't think so. Sort of, I think that question has been asked for decades and I still never see that the answer is 'yes.'
Ryan Donovan: Right.
Pia Nilsson: Because, you know, it's– the tools around us are constantly evolving to meet new needs, and new needs are definitely coming now with AI. So, we're gonna see so many useful tools come our way, hopefully. So, I think, as over time you would deprecate some tools that aren't serving a big enough need anymore, because the needs are moving on. But I think one should really focus on, 'what are the needs?' And then find the solutions that are solving for those.
Ryan Donovan: Mm-hmm.
Pia Nilsson: And, in my view, I kind of like the openness-principle here. Instead of, as we can see in some places, sort of, moving to one player—sort of, one vendor—who will offer it all for you, I really think, both: that's a vendor lock-in that I don't know why you would ever want that; and secondly, I just doubt you would be able to keep up.
Ryan Donovan: Right.
Pia Nilsson: It's just, it hasn't been possible so far, and I don't think we're slowing down as an industry – rather, the opposite.
Ryan Donovan: I mean, that's– that was gonna be my other question: is that, you know, 'you're solving these tools to meet needs,' the opposite in trying to solve too many needs with one tool, you know?
Pia Nilsson: Yeah.
Ryan Donovan: And I think you gave a yes for that. Yeah. So, we talk about open source and anti lock-in. What's the, sort of, ‘openness’ that you all are promoting with Backstage? Like, how mad are you gonna be if somebody switches to another IDP?
Pia Nilsson: Well, we probably didn't do a good enough job in that case. So, you know, we are not gonna be mad, but we are gonna learn. So, our approach is that we need to build a really useful open platform that works well for companies like ourselves, at scale; if we do so, we are pretty confident this is gonna be a great fit for other companies as well. But you know, there are different needs, and if different companies have particular needs that are met by someone else, then that's just great. It's a very big bond.
Ryan Donovan: Yeah. So, one of the things we think about, you know, even when there's an unmet need and you have the right tool for the unmet need – getting people to adopt that tool can be difficult. Do you have strategies, thoughts, on how to get people to pick up a tool that is useful?
Pia Nilsson: I think, at the heart of that, sort of, our narrative on that question—our thinking around that question—is, well, if you build something really useful, you'll have the users, right? That's also why one can't single-handedly focus on just one tool. Like an IDP – it's not the end-all, be-all. We are– we're not here about a single tool, we're here for developer experience – improving effectiveness for a company, as well as satisfaction, and joy. And how we think you drive internal adoption is: meet the developers where they are.
Ryan Donovan: Mm-hmm.
Pia Nilsson: That's what we do; and also, figure out the biggest pain points. One may have some different pain points as a developer, and then some other pain points as a platform engineer, and then some other pain points as a platform engineering leader, right?
Ryan Donovan: Mm-hmm.
Pia Nilsson: Like, a person like me – I care a lot about fleet standardization, and fleet automation, and AI migrations. That's what I like. I really care about those things. Not every single person in a company necessarily needs to do or think about that every single day.
Ryan Donovan: Right.
Pia Nilsson: Someone else might care about, 'well, what should I be doing today? What are the incidents? What are the components that I own? What happened last night on On Call?' Like, these are sort of different needs. So, one should figure out like, ‘what are the most pressing needs that we need to solve for in our company in the platform engineering domain?’ Sort of. And then, I think the portal and Backstage can be this great vehicle to drive such change; and if you go about it like that, you will have internal adoption, you know? Because you're solving for a need that needed solving for.
Ryan Donovan: That's an interesting thing to point out that, you know, there's different needs for different parts of the company. Do you think about segmenting Backstage? Do you have plugins that you yourselves create to, sort of, meet those needs and get out of the way of the people who don't need them?
Pia Nilsson: Oh, absolutely. So, for example, we have a standardization product that is called ‘Soundcheck.’ Of course, we'd love it if all engineers would go into Soundcheck and check, 'how golden am I on all the services?' – golden is: you're adhering to all the standards we set in the tech standards at Spotify. So like, if an orbiter engineer goes in and say like, 'how golden am I?' That's great, but they don't necessarily need to do that every day; so, it's primarily used for tech leadership, and product leadership, and such. So, that's one product. And another product is aimed for product evolution – sort of, product development. So, our experimentation platform: that's not an engineering platform engineering tool, per se; it's more like a product development tool. So, we definitely have different personas and different products that are very specific about what kind of needs they fill.
Ryan Donovan: Mm-hmm. And I imagine, you know, with a tool like this, part of the thing you want us to break down silos – something we think about too. Do you have advice for folks, for, you know, one: identifying silos; and then getting your teams to break out of them and actually collaborate?
Pia Nilsson: Great question.
Ryan Donovan: I mean, a million-dollar question, right?
Pia Nilsson: I think all companies are in need of, sort of, constantly thinking about, 'how do we inspire folks and help folks collaborate across org boundaries?' We figured out one product that helps us, really, a lot. It's– we call it "skill exchange," and nowadays we're also offering that commercially. What that does is, you sort of have your internal CV, so you have your courses there and you can extend it to, like, ‘opportunities’ we call them—so, job opportunities, like, embed opportunities—so you go someplace, help a team for a few weeks, and then you go back to your home team – different sort of ways to interact. And then of course we have Hack Week. So, we run Hack Week, and then we make that more collaborative through totally transparently sharing all hacks in this hack portal, which also is inside of scale exchange, of course. So, if one can sort of use a product like that to tie into supporting the engineering culture (that really wants to collaborate usually)—I think we're in a very collaborative community at large as engineers—I think just finding some small ways to incentivize and help that culture, that goes a long way. And then, then there's this technical answer, I think, to the question as well: how do we help people collaborate? Well, we simplify our tech stack—it's the hard way, sort of, but very important—so, standardized, so that people actually aren't totally confused when moving from one organization to the other in the tech stack, and the choices of libraries, and frameworks, and all of that. If it looks a little bit the same and you can find everything in portal and Backstage, then you sort of really support growth among people and also, you know, collaboration.
Ryan Donovan: Yeah. So, not everybody gets to use their ‘pet language,’ or their ‘pet-tech.’
Pia Nilsson: Yes, exactly.
Ryan Donovan: For a lot of organizations, we talk to the way that they sort of encourage and build that sort of collaboration: is how somebody who is kind of a shepherd for that, who it's their job to break down barriers to get people to use the tools to build support – do you think that's necessary?
Pia Nilsson: So I definitely think one needs to find a 'champion' for a problem – a problem that, whatever product you want to have adoption on, solves for. So, one has to find a person that agrees with, 'yes, this is a problem for me personally, for my team, for the teams around me. I agree this would be great to solve with this product of yours.' So, for us, we do that through the plugin architecture. And for example, way back in, when we were rolling out Backstage internally and we wanted people to start using it, we didn't ask them to start using Backstage; we instead went to the data platform—as just one example—and we asked them, 'what are your products here?' And then they showed us all of their products, and then we sort of said, 'well, we can maintain them inside of the Backstage, but you're gonna own the code, but you– you don't need to run the Backstage instance. We do that for you.' And that's like grant work, so you gotta find, like, a way to make sure that the 'champions' still solve the problem they care about within your product. They keep ownership of, obviously, their expertise domain, and you do something for them. So, I think we have to reverse it, in a sense – we need to figure out, 'how do we help the domain expert to make their jobs easier?'
Ryan Donovan: Well, everyone, it's that time of the show where we shoutout somebody who came on the Stack Overflow, dropped a little knowledge, shared some curiosity, and earned themselves a badge. Today, we're shouting out the winner of a populous badge. Congrats to 'Marcus Shite' for dropping an answer that was so good, it outshined and outscored the accepted answer, and dropped it on the question: 'Dynamic tag values for the Counter metric in Micrometer.' So, if you're curious about that, we'll have an answer in the show notes. I am Ryan Donovan, I edit the blog, host the podcast, here at Stack Overflow. If you have questions, concerns, topics, whatever, you can email me at podcast@stackoverflow.com. And if you wanna reach out to me directly, you can find me on LinkedIn.
Pia Nilsson: And I'm Pia Nilsson. You can find me at LinkedIn, or visit Backstagespotify.com and click talk to us.
Ryan Donovan: All right, thank you for listening everyone, and we'll talk to you next time.