Phillip Carter, Principal Product Manager at Honeycomb and open source software developer, talks with host Giovanni Asproni about observability for large language models (LLMs). The episode explores similarities and differences for observability with LLMs versus more conventional systems. Key topics include: how observability helps in testing parts of LLMs that aren't amenable to automated unit or integration testing; using observability to develop and refine the functionality provided by the LLM (observability-driven development); using observability to debug LLMs; and the importance of incremental development and delivery for LLMs and how observability facilitates both. Phillip also offers suggestions on how to get started with implementing observability for LLMs, as well as an overview of some of the technology's current limitations. This episode is sponsored by WorkOS.
Episode 538: Roberto Di Cosmo on Archiving Public Software at Massive Scale
Episode 537: Adam Warski on Scala and Tapir
Episode 536: Ryan Magee on Software Engineering in Physics Research
Episode 535: Dan Lorenc on Supply Chain Attacks
Episode 534: Andy Dang on AI/ML Observability
Episode 533: Eddie Aftandilian on GitHub Copilot
Episode 532: Peter Wyatt and Duff Johnson on 30 Years of PDF
Episode 531: Xe Iaso on Tailscale
Episode 530: Tanmai Gopal on GraphQL
Episode 529: Jeff Perry on Career Management for Software Engineers
Episode 528: Jonathan Shariat on Designing to Avoid Worst Case Outcomes
Episode 527: Adrian Kennard and Kevin Hones on Writing a Network OS from Scratch
Episode 526: Brian Campbell on Proof of Possession Defenses
Episode 525: Randy Shoup on Evolving Architecture and Organization at eBay
Episode 524: Abi Noda on Developer Experience
Episode 523: Jessi Ashdown and Uri Gilad on Data Governance
Episode 522: Noah Gift on MLOps
Episode 521: Phillip Mayhew on Test Automation in Gaming
Episode 520: John Ousterhout on A Philosophy of Software Design
Episode 519: Kumar Ramaiyer on Building a SaaS
Create your
podcast in
minutes
It is Free
Insight Story: Tech Trends Unpacked
Zero-Shot
Fast Forward by Tomorrow Unlocked: Tech past, tech future
The Unbelivable Truth - Series 1 - 26 including specials and pilot
Lex Fridman Podcast