Today we’re joined by Kyle Roche, founder and CEO of Griptape to discuss patterns and middleware for LLM applications. We dive into the emerging patterns for developing LLM applications, such as off prompt data—which allows data retrieval without compromising the chain of thought within language models—and pipelines, which are sequential tasks that are given to LLMs that can involve different models for each task or step in the pipeline. We also explore Griptape, an open-source, Python-based middleware stack that aims to securely connect LLM applications to an organization’s internal and external data systems. We discuss the abstractions it offers, including drivers, memory management, rule sets, DAG-based workflows, and a prompt stack. Additionally, we touch on common customer concerns such as privacy, retraining, and sovereignty issues, and several use cases that leverage role-based retrieval methods to optimize human augmentation tasks.
The complete show notes for this episode can be found at twimlai.com/go/659.
Are Vector DBs the Future Data Platform for AI? with Ed Anuff - #664
Quantizing Transformers by Helping Attention Heads Do Nothing with Markus Nagel - #663
Responsible AI in the Generative Era with Michael Kearns - #662
Edutainment for AI and AWS PartyRock with Mike Miller - #661
Data, Systems and ML for Visual Understanding with Cody Coleman - #660
AI Access and Inclusivity as a Technical Challenge with Prem Natarajan - #658
Building LLM-Based Applications with Azure OpenAI with Jay Emery - #657
Visual Generative AI Ecosystem Challenges with Richard Zhang - #656
Deploying Edge and Embedded AI Systems with Heather Gorr - #655
AI Sentience, Agency and Catastrophic Risk with Yoshua Bengio - #654
Delivering AI Systems in Highly Regulated Environments with Miriam Friedel - #653
Mental Models for Advanced ChatGPT Prompting with Riley Goodside - #652
Multilingual LLMs and the Values Divide in AI with Sara Hooker - #651
Scaling Multi-Modal Generative AI with Luke Zettlemoyer - #650
Pushing Back on AI Hype with Alex Hanna - #649
Personalization for Text-to-Image Generative AI with Nataniel Ruiz - #648
Ensuring LLM Safety for Production Applications with Shreya Rajpal - #647
What’s Next in LLM Reasoning? with Roland Memisevic - #646
Is ChatGPT Getting Worse? with James Zou - #645
Create your
podcast in
minutes
It is Free
20/20
The Dropout
Ten Percent Happier with Dan Harris
World News Tonight with David Muir
NEJM This Week