generative AI

How do you evaluate an LLM? Try an LLM.

On this episode: Stack Overflow senior data scientist Michael Geden tells Ryan and Ben about how data scientists evaluate large language models (LLMs) and their output. They cover the challenges involved in evaluating LLMs, how LLMs are being used to evaluate other LLMs, the importance of data validating, the need for human raters, and more needs and tradeoffs involved in selecting and fine-tuning LLMs.

Data, data everywhere and not a stop to think

Ben and Ryan are joined by Nick Heudecker, Senior Director of Market Strategy and Competitive Intelligence at Cribl, to discuss the state of data and analytics. They cover GenAI, the role of incumbents vs. startups, challenges of data storage and security, data quality and ETL pipelines, measures of data quality for GenAI, and Cribl’s role in the data and observability space.

Is AI making your code worse?

Ben and Ryan are joined by Bill Harding, CEO of GitClear, for a discussion of AI-generated code quality and its impact on productivity. GitClear’s research has highlighted the fact that while AI can suggest valid code, it can’t necessarily reuse and modify existing code—a recipe for long-term challenges in maintainability and test coverage if devs are too dependent on AI code-gen tools.

A leading ML educator on what you need to know about LLMs

Machine learning scientist, author, and LLM developer Maxime Labonne talks with Ben and Ryan about his role as lead machine learning scientist, his contributions to the open-source community, the value of retrieval-augmented generation (RAG), and the process of fine-tuning and unfreezing layers in LLMs. The team talks through various challenges and considerations in implementing GenAI, from data quality to integration.

Inside Intuit's generative AI operating system, GenOS

In today’s episode of the podcast, sponsored by Intuit, Ben and Ryan talk with Shivang Shah, Chief Architect at Intuit Mailchimp, and Merrin Kurian, Principal Engineer and AI Platform Architect at Intuit. They discuss generative AI at Intuit, GenOS (the generative AI operating system that they built), and how GenAI can scale without sacrificing privacy.