Who watches the watchers? LLM on LLM evaluations
While using LLMs to judge LLM outputs might seem like the fox guarding the henhouse, turns out it works pretty well (and scales better than humans).

While using LLMs to judge LLM outputs might seem like the fox guarding the henhouse, turns out it works pretty well (and scales better than humans).
While AI coding assistants are helping developers become more productive, the true value of AI lies in its ability to automate the non-coding tasks that have historically been bottlenecks, allowing leaders to create more agile teams and focus on higher-level strategic problems.
Gil Feig, co-founder and CTO of Merge, joins the show to explore Merge’s approach for reducing third-party APIs to a single call, the complexities of and need for data normalization, and the role that AI and MCP plays in the future of API functionality.
This episode provides insights and strategies to successfully navigate AI adoption in engineering teams. Learn how to build developer confidence and create environments that drive real results beyond the hype.
Ryan sits down with Marco Palladino, CTO of Kong, to talk about the rise of AI agents and their impact on API consumption, the MCP protocol as a new standard for agents, the importance of observability and security in AI systems, and the importance for businesses and entrepreneurs to leverage opportunities in the agentic AI space now.
Jeff Hollan, director of product at Snowflake, joins Ryan to discuss the role that data plays in making AI and AI agents better. Along the way, they discuss how a database leads to an AI platform, Snowflake’s new data marketplace, and the role data will play in AI agents.
Ryan chats with Karen Ng, EVP of Product at HubSpot, to chat about Model Context Protocol (MCP) and how they implemented it for their server for their CRM product.
Ryan welcomes Geraint North, AI and developer platforms fellow at Arm, to dive into the impact of GenAI on chip design.
Whether you're leading an engineering team today or preparing for an AI-integrated future, this conversation provides practical insights into where AI can have the greatest impact on your software delivery process.
For promising Gen Z students, a career as a software developer seemed like the golden ticket to career stability and success. But in the age of AI, the career promise for Gen Z software developers is gone.
Ryan is joined by our very own Ash Zade, Staff Product Manager, and Alex Warren, Staff Software Engineer, to discuss our newly released stackoverflow.ai, how it’s enhancing user experience by combining human-validated answers with AI, and our future plans for deeper personalization and community integration.
Kylan Gibbs, CEO of Inworld, joins the show to discuss the technical challenges of creating interactive AI for virtual worlds and games, the significance of user experience, and the importance of accessibility and cost-efficiency in deploying AI models.
Ryan welcomes Darko Mesaroš, Principal Developer Advocate at AWS and all-around computer history buff, to chat about the history of software development improvements and how they made developers made more productive.
Ryan welcomes Nathan Michael, CTO at Shield AI, to discuss what AI looks like in defense technologies, both technically and ethically.
Ryan is joined by Tuhin Srivastava, CEO and co-founder of Baseten, to explore the evolving landscape of AI infrastructure and inference workloads, how the shift from traditional machine learning models to large-scale neural networks has made GPU usage challenging, and the potential future of hardware-specific optimizations in AI.
Wenjing Zhang, VP of Engineering, and Caleb Johnson, Principal Engineer at LinkedIn, sit down with Ryan to discuss how semantic search and AI have transformed LinkedIn’s job search feature. They explore the engineering efforts behind transitioning from keyword-based search and the impact of AI models on LinkedIn’s job seekers and employers.
Quinn Slack, CEO and co-founder of Sourcegraph, joins the show to dive into the implications of AI coding tools on the software engineering lifecycle. They explore how AI tools are transforming the work of developers from syntax-focused tasks to higher-level design and management roles and how AI will integrate into enterprise environments.
Ryan and Eira welcome Erin Yepis, Senior Analyst at Stack Overflow, to the show to discuss the newly released 2025 Developer Survey results. They explore the decline in trust in AI tools, shifts in popular programming technologies, and the patterns Erin saw in salary growth among developers.
Ryan welcomes Mahir Yavuz, Senior Director of Engineering at Etsy, to the show to explore the unique challenges that Etsy’s marketplace faces and how Etsy’s teams leverage machine learning and AI to manage product SKUs, enrich inventory metadata, and improve both buyer and seller experiences.
Ryan is joined on the podcast by Confluent’s AI Entrepreneur in Residence, Sean Falconer, to discuss the growing need for standards for AI agents, the emerging Model Context Protocol and agent-to-agent communication, and what we can learn from early web standards while AI continues to evolve.
Ryan is joined by Kieran Furlong, CEO of Realta Fusion, to talk about the future of fusion as a safe and sustainable energy source, the computation and scientific advancements that have made fusion possible, and how fusion technology innovations will address data and AI’s rising energy demands.
Ryan sits down with CTO Aruna Srivastava and CPO Ruslan Mukhamedvaleev from Koel Labs to talk about how they’re innovating speech technology with the help of AI and classic movies. They also tell Ryan about their time in the Mozilla Builders Accelerator and their experiences as student co-founders in an ever-changing economic and technological landscape.
Ryan welcomes Illia Polosukhin, co-author of the original "Attention Is All You Need" Transformers paper and co-founder of NEAR, on the show to talk about the development and impact of the Transformers model, his perspective on modern AI and machine learning as an early innovator of the tech, and the importance of decentralized, user-owned AI utilizing the blockchain.