Today we’re joined by Jay Emery, director of technical sales & architecture at Microsoft Azure. In our conversation with Jay, we discuss the challenges faced by organizations when building LLM-based applications, and we explore some of the techniques they are using to overcome them. We dive into the concerns around security, data privacy, cost management, and performance as well as the ability and effectiveness of prompting to achieve the desired results versus fine-tuning, and when each approach should be applied. We cover methods such as prompt tuning and prompt chaining, prompt variance, fine-tuning, and RAG to enhance LLM output along with ways to speed up inference performance such as choosing the right model, parallelization, and provisioned throughput units (PTUs). In addition to that, Jay also shared several intriguing use cases describing how businesses use tools like Azure Machine Learning prompt flow and Azure ML AI Studio to tailor LLMs to their unique needs and processes.
The complete show notes for this episode can be found at twimlai.com/go/657.
Real-Time ML Workflows at Capital One with Disha Singla - #606
Weakly Supervised Causal Representation Learning with Johann Brehmer - #605
Stable Diffusion & Generative AI with Emad Mostaque - #604
Exploring Large Language Models with ChatGPT - #603
Accelerating Intelligence with AI-Generating Algorithms with Jeff Clune - #602
Programmatic Labeling and Data Scaling for Autonomous Commercial Aviation with Cedric Cocaud - #601
Engineering Production NLP Systems at T-Mobile with Heather Nolis - #600
Sim2Real and Optimus, the Humanoid Robot with Ken Goldberg - #599
The Evolution of the NLP Landscape with Oren Etzioni - #598
Live from TWIMLcon! The Great MLOps Debate: End-to-End ML Platforms vs Specialized Tools - #597
Live from TWIMLcon! You're not Facebook. Architecting MLOps for B2B Use Cases with Jacopo Tagliabue - #596
Building Foundational ML Platforms with Kubernetes and Kubeflow with Ali Rodell - #595
AI-Powered Peer Programming with Vasi Philomin - #594
The Top 10 Reasons to Register for TWIMLcon: AI Platforms 2022!
Applied AI/ML Research at PayPal with Vidyut Naware - #593
Assessing Data Quality at Shopify with Wendy Foster - #592
Transformers for Tabular Data at Capital One with Bayan Bruss - #591
Understanding Collective Insect Communication with ML, w/ Orit Peleg - #590
Multimodal, Multi-Lingual NLP at Hugging Face with John Bohannon and Douwe Kiela - #589
Synthetic Data Generation for Robotics with Bill Vass - #588
Create your
podcast in
minutes
It is Free
20/20
The Dropout
Ten Percent Happier with Dan Harris
World News Tonight with David Muir
NEJM This Week