Today we’re joined by James Zou, an assistant professor at Stanford University. In our conversation with James, we explore the differences in ChatGPT’s behavior over the last few months. We discuss the issues that can arise from inconsistencies in generative AI models, how he tested ChatGPT’s performance in various tasks, drawing comparisons between March 2023 and June 2023 for both GPT-3.5 and GPT-4 versions, and the possible reasons behind the declining performance of these models. James also shared his thoughts on how CRISPR could potentially revolutionize LLM and AI systems, and how adding monitoring tools can help in tracking behavioral changes in these models. Finally, we discuss James' recent paper on pathology image analysis using Twitter data, in which he explores the challenges of obtaining large medical datasets and data collection, as well as detailing the model’s architecture, training, and the evaluation process.
The complete show notes for this episode can be found at twimlai.com/go/645.
Language Understanding and LLMs with Christopher Manning - #686
Chronos: Learning the Language of Time Series with Abdul Fatir Ansari - #685
Powering AI with the World's Largest Computer Chip with Joel Hestness - #684
AI for Power & Energy with Laurent Boinot - #683
Controlling Fusion Reactor Instability with Deep Reinforcement Learning with Aza Jalalvand - #682
GraphRAG: Knowledge Graphs for AI Applications with Kirk Marple - #681
Teaching Large Language Models to Reason with Reinforcement Learning with Alex Havrilla - #680
Localizing and Editing Knowledge in LLMs with Peter Hase - #679
Coercing LLMs to Do and Reveal (Almost) Anything with Jonas Geiping - #678
V-JEPA, AI Reasoning from a Non-Generative Architecture with Mido Assran - #677
Video as a Universal Interface for AI Reasoning with Sherry Yang - #676
Assessing the Risks of Open AI Models with Sayash Kapoor - #675
OLMo: Everything You Need to Train an Open Source LLM with Akshita Bhagia - #674
Training Data Locality and Chain-of-Thought Reasoning in LLMs with Ben Prystawski - #673
Reasoning Over Complex Documents with DocLLM with Armineh Nourbakhsh - #672
Are Emergent Behaviors in LLMs an Illusion? with Sanmi Koyejo - #671
AI Trends 2024: Reinforcement Learning in the Age of LLMs with Kamyar Azizzadenesheli - #670
Building and Deploying Real-World RAG Applications with Ram Sriharsha - #669
Nightshade: Data Poisoning to Fight Generative AI with Ben Zhao - #668
Learning Transformer Programs with Dan Friedman - #667
Create your
podcast in
minutes
It is Free
20/20
The Dropout
Ten Percent Happier with Dan Harris
World News Tonight with David Muir
NEJM This Week