SPONSORED BY INTUIT
Over the past five years, Intuit went through a total cloud transformation—they closed the data centers, built out a modern SaaS development environment, and went cloud native with foundational building blocks like containers and Kubernetes. Now they are looking to continue transforming into an AI-driven organization that leverages the data they have to make their customers’ lives easier. Along the way, they realized that their internal systems have the same requirements to leverage the data they have for AI-driven insights.
On this sponsored episode of the podcast—the first in a series of four—we talk with Pratik Wadher, senior VP of Product Development at Intuit, about how it is building a AI-enabled development platform that increases the development velocity for their 7,000 plus developers.
Episode notes
In terms of sheer volume, the AI/ML program at Intuit is massive. They make 58 billion ML predictions daily, enable 730 million AI-driven customer interactions every year, and maintain over two million personalized AI models.
Wadher notes that Intuit uses development velocity, not developer velocity. The thinking is that an engineering org should focus on shipping products and features faster, not making individual devs more productive.
No, the robots aren’t coming for your jobs. Wadher says their AI strategy relies on helping experts make better insights. The goal is to arm those experts, not replace them.
Intuit’s not here to hoard secrets. They’ve open sourced their DevOps pipeline tool, Argo, which a lot of companies used for AI and data pipelines. Intuit has recently launched Numaproj, which open sources a number of internal tools and capabilities.
Congrats to Lifeboat badge winner Bill Karwin for their answer to Understanding MySQL licensing.
TRANSCRIPT
[intro music plays]
Ben Popper Hello, everybody. Welcome back to the Stack Overflow Podcast, a place to talk all things software and technology. I'm your host, Ben Popper, joined as I often am by my colleague, Ryan Donovan. Hey, Ryan.
Ryan Donovan Hey, Ben. How are you doing?
BP I'm doing well. So we have another sponsored podcast series for you and we have a great guest today, Pratik Wadher, who is the SVP of Development Platform and Experiences at Intuit, and we're going to be talking about accelerating development velocity through AI Ops and investment in open source. Ryan, I know this topic is near and dear to our heart. We published a great post on our own blog from one of our engineers about how to balance developer velocity with burnout, and then recently you wrote a great piece about what it means to be performant and whether that refers to the software itself or in some cases building the right tool for the criteria you have set out. So without further ado, we'd like to invite Pratik on the show to discuss his take on a bunch of these topics. Pratik, welcome.
Pratik Wadher Thank you, Benjamin and Ryan. I’m looking forward to this conversation.
BP So we always start out, just situate our listeners a little bit. Tell them a bit about how you got into the technology industry, a little bit of early education and career, and then how you found yourself in the role you're at today.
PW Well I'll start with my early background. I am actually born and raised in Kenya. So if you're aware, that's a country in East Africa. I am of Indian descent, but I was born in Kenya, so was my dad. My grandfather immigrated to Kenya at the early age of 12, so effectively Kenyan. And I guess I've been very technologically inclined from the get go. And I like to tell the story that growing up in Kenya, I didn't have access to computers or software or know anything about it, but I did know how to fix toasters and little gadgets and devices that broke down around the house, so that's kind of my early introduction to engineering.
BP Electrical engineering first. That sounds fun.
PW Electrical engineering and electronics, and that was sort of my passion. So I actually did my undergraduate degree at King's College London and again focused on computer systems and electronics. And my early introduction to computers was a Commodore 64. That's very early basic programming that I started learning before I went into college.
RD Yep, me too.
PW There you go. So then as I did my undergraduate, I very quickly, or slow I guess, started migrating to the software aspects of the field. I started doing early digital signal processing based products, and then it eventually just graduated into sort of more mainstream software development. And the short story is, eventually I graduated and I had an offer to work at a startup in the US. I was pursuing a PhD, but I got an offer to work at a startup and I took that opportunity and said I’d always come back to a PhD. Unfortunately that never happened.
BP All right, still time. And you were at a company, Applatix, that was acquired by Intuit, where you were the founder and CEO?
PW That's correct. My career journey has been startups all the way, starting with my first job. And Applatix was a company that I co-founded and led until the acquisition by Intuit. It was sort of at the beginning of the cloud native ecosystem journey, a lot of focus around companies moving to the cloud. And as we were looking at that transition, one of the areas that has always bugged me when I've been doing my startups is every time I sort of start from ground zero, build up a development environment and a deployment environment. And with cloud, it always felt like maybe this was sort of the nirvana. It would essentially allow us to move faster and have these sort of modular components that you could quickly leverage and build the starting block business.
BP If somebody hasn't created a cloud company named Nirvana we're missing out. Somebody in marketing needs to get on that. So the company was acquired and you moved inside of Intuit, and did you immediately move to the role you're in now or was there an evolution that brought you there?
PW I think it's an evolution. Initially the company was acquired because Intuit was at sort of the early phase of transitioning from sort of a data center-centric company to essentially moving to more of a modern cloud native environment, moving into the cloud, and so the expertise that was needed was really a systems background that would essentially allow us to move the company into the cloud. So that's kind of where we started and focused on. And the last five years that I've been here we've essentially transitioned the entire company onto the cloud, we've closed our data centers, we've built a modern development environment and a SaaS environment that is essentially cloud native. And when I say cloud native, it's using the foundational blocks like Kubernetes and containers and a high level of AI and data based approaches. And about three years ago I think we declared a vision to actually build a platform, to essentially transition into an AI driven expert platform. And so with that vision we had the foundational blocks that we had been working towards as becoming cloud native, becoming more capability oriented, more microservices oriented, and it sort of gave us a good foundation to essentially start building the platform.
RD So the platform, this is for external customers, or is this for your internal developers?
PW It's both, and it's very interesting to think about it this way because when we talk about an AI-driven expert platform, the thing that immediately jumps to mind is our customer-facing products. Whether it's the tax side or it's the small business side of the house, how do we enable these customers? Our founding tenet has always been that we want our customers essentially to save more or to have more money in their pockets, to work less, and to really have the confidence in their financial identity. So with those three things the focus has been around all the data that we have in these products. How do we leverage that data to actually make the lives of our customers easier? And that's always been the focus, right as we started it. But very quickly we realized that our internal systems and capabilities that we're building to power these products also had the same requirements. So for example, when we talk about our observability platform, which is geared to very quickly detect any issues that happen in our products, we quickly found that we needed a way to capture all of the operational data that was going into the system and then essentially build and train models to drive anomaly detection, for example. So the same techniques that we were using externally are also being used internally.
BP Interesting. Yeah, Ryan and I have had a number of guests on the show recently to discuss things in this area. Those are mostly observability vendors. Was there a reason that you decided to build that stuff internally, or when you say build it, do you mean relying a bit on a mix of internal tooling and external vendors?
PW It's always a mix, but as part of our cloud journey one of the tenets that we set out for ourselves is that a lot of innovation happens in the open source world. And our idea was, how do we capture that innovation? How do we actually leverage that, and contribute that back, but then bring it in? So we have very much focused our efforts on enabling the community at large to build some of these solutions and then of course combine that with best of class vendor solutions that are available so that we have a hybrid approach.
RD So part of your purview is developer experience at Intuit, correct?
PW That's correct.
RD Yeah. So you have I think 7,000 developers. What is your day-to-day there with that number of developers and how are you working with them to improve their experience?
PW Yeah, yes, there are over 7,000 developers. And as I said, the platform that we have built is utilized by all of the developers at Intuit. Now, it covers a gamut from if you start with the core compute and storage and networking arena, going all the way to things like observability, anomaly detection, our UX fabric which powers all of the user experiences that our customers face. So all of those things actually culminate into what we call our development portal. So we have one development portal that every developer at Intuit is familiar with, and it provides them with all the basic tools that they would need to create these experiences, whether they want to create a web app or a library or a mobile app or a microservice, for example. They are what we call paved roads that are available, and within minutes any developer at Intuit will be able to essentially have an environment that is very ready with all of the core building blocks and that's how they start. So our day-to-day job is really making sure that that core platform is operating and available for all our developers.
BP It's really interesting to hear you say that. We spoke with some of your colleagues on the design system side and they had similar ideas about empowering developers by making sure that they could easily access what they needed to in terms of, “These are the totems of the brand; These are how we think about design.” But you can then go off with these guideposts and build something of your own and then it will be easy to bring it back sort of into the centerfold and use it even if you're in a very different division. And I'm sure there's lots of different parts across Intuit. Ryan and I also had an interesting conversation recently with the folks at Spotify who had built this developer portal and they would go out and they'd be demoing something and all everybody wanted to know about was the developer portal tool in the background so they eventually open sourced that. That seems like it's a really integral part at a large organization, especially one that's fast growing, to empower developers to have a developer portal where they can go and find the tools they need, and like you said, the paved roads that show them how they can travel from the tools to the building to the finished product.
PW Yeah, and by the way, we've worked extensively with the Spotify team because the portals are very similar. There's a lot that goes into building these portals. But the common theme though is, how do you enable your developers to quickly do what they need to do without getting encumbered with all of the bits and pieces that are not directly in their line of sight? So things like security, for example. It's one of those areas where Intuit is heavily focused on. Our products need to be very secure. But training 7,000 developers to be well versed in every facet of security is almost impossible, so how do you essentially build these capabilities so that they don't have to think about the basic issues surrounding security but they're just available to them as they build their product?
RD So the developer portal thing– I'd worked at GrubHub and was trying to solve a lot of those issues. What other things do you have going on to kind of help with the developer velocity and happiness and make sure that these huge teams are getting things done in the way that they need to?
PW I think developer velocity– actually, we don't call it developer velocity. Let me emphasize that. We actually call it development velocity. And there's a big, big discussion because every time I've talked to companies and teams when we start focusing on developer velocity, then you quickly get into how do you make your developers more performant or productive, versus really how do you make your overall development cycle more productive? So when you start thinking about that then you really have to look at all the different aspects of product development. And it starts with, do you have the right design product management and product development functions? Do you have the right tooling? But also, are you measuring the right outcomes? At Intuit, when we talk about development velocity we fundamentally start with that all we care about is ultimately what gets shipped to our customers. We don't care about lines of code or story cycles. We care about what eventually gets shipped to our customer. And when you start with that basis you then have a very, very common and aligned goal across the companies and the groups. So starting with all of our software assets, whether it's a library or a mobile app, we essentially measure the same thing. Whether it's an AI model, same thing. What does it take for you to actually get this into production, and how fast are you getting it into production? What's your frequency of getting it released into production? So when you start there then it becomes a lot easier. And then we really start tracking back from there and saying, “Okay, what are the things that are preventing you from moving faster? And having a centralized portal like the development portal where every software asset is available and cataloged and we're tracking it, all of that data is moving into an operational data lake. Then we have the right units of measurement and really start to look at things at the right angle.
BP Very cool. So we've chatted a little bit now about how you prioritize productivity but balance that with developer happiness and how you think about productivity in a more holistic way. You mentioned the developer portal being a key to that. What role does AI play in helping to achieve some of the same goals we're discussing here?
PW Yeah, so our AI strategy is sort of unique in two ways. So everyone talks about AI and everyone has most likely a data lake where they're capturing the data, they're running models, they're training models against the data and generating insights. What we've done differently is that we have essentially combined our AI approach with also our people approach. So we not only invest in the data, the AI, and the models, but we are trying to blend the digital and the human aspects of this. VR, like for example, our financial experts. So when you are looking at TurboTax and you're trying to file your taxes, yes, the system is actually giving you a lot of insights based on the data that we have available. But as you start looking at this and you have an issue, then we can actually combine it with human expertise that can actually help you looking at those insights. So that hybrid approach is a little different and that's sort of the foundation of our AI-driven expert platform.
BP But it's interesting to hear you say, “Hey, we can prompt you. Maybe you want to try this deduction, maybe you could look into this option,” but then have a human available. It reminds me a lot of a startup we interviewed recently, DoNotPay, translating between legalese and things regular people can understand. That's a great application sometimes of AI to have kind of a decision tree, a language tree, to walk you through making what would otherwise be decisions that to you are very complicated or a bit obtuse.
PW Yeah, and it goes back to, like we talked about, the platform. How do you enable your developers to actually make use of these very easily? So when you build the right platform, then the developers actually do not have to worry about being experts in the AI domain but they can actually utilize the platform for what they need very quickly to generate the insights that they need. So I think that really helps– building the right platform to actually enable. You can't have seven thousand AI experts in a company.
BP It would be expensive. It would be very expensive.
PW But each of them does have a lot of insights in the particular domain that they're working in. And so then the question is, how do you enable them? How do you arm them to actually develop these tool sets and capabilities?
RD Yeah. It's interesting you talk about having the AI and people work together, because I think we've had a couple articles about people worrying about if the robots are coming to take your job. And so can you talk a little bit about how they're working together and what they're actually doing to make things better?
PW So first of all, that whole notion, again, blending it with the human expertise, I think we underrate the human expertise that comes in. AI models can help you generate insights, but how do you use those insights to actually make sure that you're getting the right results? There is still a human element to this. So when we, for example, developed our expert platform, the whole goal was really to arm our experts, and they're usually not direct Intuit employees, but they're at the forefront of adopting these AI tools and providing invaluable feedback back to us, because as they're working with the customers, they're helping us train the best AI models possible. So that data coming back as they're working with customers is so critical and that's what people forget, that once you create an AI model, you have to train it, and training it doesn't just mean giving it more data. Actually, you need to figure out what's the best feedback that you're getting and then essentially feeding that into the system as well.
BP Right. Yeah, I mean there's definitely reinforcement learning there in both directions, like you said. Is it looking to the data and finding the best solutions, but then when it's implemented, which is equally interesting to the user, do they have a positive outcome? So you’ve got to measure it on both sides and tweak the model that way. When you say that people don't have to be experts in AI when they're on your development team to use it, is there a drag and drop functionality? “Hey, I'm building this new feature and it's helping people who have moved between two states and have built a new house. I want to make sure they get the right X, Y, Z. When they enter their information, then they're going to be fed into the AI pipeline here, and so I'm dropping this node,” or how does it work?
PW It does. So, we're sort of at the leading edge when we talk about what we do. Things like personalized recommendations, predictions that we do for our customers, these are extremely critical and those are automated in this system. So we have platform capabilities and model designs that enable us to custom train the models for individual customers. Literally millions of unique models, they are tailored to our customers on an automated basis. And this allows us to very, very, very quickly tailor the predictions that we are seeing. To give you an idea, we have about 58 billion machine learning predictions that we do every day. We have 730 million AI driven customer interactions each year. And we have over 2 million AI-driven personalized models– personalized models, just for categorization. So when you think about this, the system is highly tuned to actually do it for each customer should the need arise. And these are done during our peak. For example, when we go through our tax peak, it’s a very critical period that most customers are filing their taxes. The average response time when we’re running these models is about 500 milliseconds So these models are delivering very, very quickly for customers.
BP It's interesting to hear you say that. I think one of the things that's interesting about the AI field is that it's a place where there's obviously a lot of corporate interest, but also a lot of open source interest and also a lot of academic interest. It seems like this would be a very attractive position in some ways to people who are at the cutting edge of the field, because as you point out, there's so many fascinating data points and so much sort of human engagement built into that that would be really interesting for them to study. What are you excited about when you look to the future when you think about the roadmap? I know you're not going to give away any secrets, but when you think about the future of what you're going to be doing for the next year or two, whether that's helping your developers be more productive or tapping into some of what's cutting edge in AI, what are you getting excited about?
PW I think for me, what's exciting is actually the innovation that I'm beginning to see in this space. So everyone does AI, everyone talks about AI. When I think back to my journey starting on the whole cloud ecosystem, the cloud native foundation, I'm actually seeing a very similar journey beginning in the AI space. So the open source infrastructure that is being built for AI, for data, is what gets me excited right now. I see almost that same kind of transition happening as containers and Kubernetes brought to sort of the cloud native ecosystem. For example, we have an open source project that we contributed to the CNCF. It's called Argo. Now Argo started with one fundamental unit which was Argo Workflows. Originally we intended Argo Workflows to be used for internal DevOps pipelines. What we are finding out now is that it's used increasingly by a lot of companies for AI and data pipelines, whether it's ingestion pipelines, whether it's training pipelines to essentially run these things. So that excites me that we are beginning to see this. In fact, about two weeks ago we launched a new open source project called Numaproj, which essentially open sources a lot of the tools and capabilities that we use internally for driving AI pipelines and AI models. And I see that that ecosystem is actually developing very fast, so that excites me a lot that we are beginning to see a lot of innovation happening in the open now and in the community where we can partner and do this.
BP Very cool.
RD That's very cool.
[music plays]
BP All right, everybody. It is that time of the show. I want to shout out a community member who came on Stack Overflow and helped save some knowledge from the dustbin of history. Bill Karwin was awarded a lifeboat badge for saving this question, “Understanding MySQL licensing.” There's a lot to unpack in here so we'll put it in the show notes. But Bill, appreciate your answer. I am Ben Popper. I am the Director of Content here at Stack Overflow. You can always find me on Twitter @BenPopper. You can email us, podcast@stackoverflow.com with questions and suggestions. And if you like what you hear, leave us a rating and a review. It really helps.
RD I'm Ryan Donovan. I'm the editor of the blog here at Stack Overflow, stackoverflow.blog. And you can find me on Twitter @RThorDonovan.
PW And I'm Pratik Wadher, SVP of Developer Experiences at Intuit, and you can find me on Twitter @PWadher. Looking forward to hearing from you.
BP All right, everybody. Thanks so much for listening and we will talk to you soon.
[outro music plays]