Ep. 43 – Beyond magic promises: Implementing productive AI w/Ilya Venger

In this episode, Noz Urbina interviews Ilya Venger, Data and AI Product Leader at Microsoft, to deliver a masterclass in practical AI implementation for business leaders. Ilya addresses the trillion-dollar question facing every executive: Should we build our own AI solution, buy off-the-shelf, or wait for the technology to mature? His answer: it depends on understanding your specific business problems, not chasing shiny technology. Key Takeaways The 80% Solution: Ilya reveals that AI systems work correctly about 80% of the time. Success isn’t about perfecting that last 20% through expensive fine-tuning – it’s about redesigning processes to work with AI’s probabilistic nature. As Noz puts it, “If you create a workflow with zero tolerance for error, you’ve designed a bad process.” The Fine-Tuning Trap: Ilya shares cautionary tales of companies spending millions to fine-tune models for specific problems (like the “six finger problem” in image generation), only to watch base models solve these issues within 18 months. His stark example: a model fine-tuned to be cheaper than GPT-4 became pointless when GPT-4’s price dropped tenfold. Data Reality Check: Both speakers agree that most organizations have “data heaps” – disconnected silos without understanding or metadata. Ilya’s metaphor: “You’ve got gold nuggets in a dark room. You need to turn on the lights first.” Organisations must understand their data landscape before implementing any AI solution. The Build vs. Buy Decision Framework: Build (Fine-tune): Only when you have extremely specific tasks with proprietary data (like recognizing manufacturing equipment or crop diseases) Buy: For most use cases, using off-the-shelf models with good system prompts and workflow design Wait: When your problem might be solved by next quarter’s model improvements What you’ll learn

• The build, buy, or wait decision framework – Clear criteria for when to fine-tune models (specific tasks with proprietary data), buy off-the-shelf solutions (most use cases), or wait for base models to improve
• Master the 80% solution – Why AI works correctly 80% of the time and three strategies to handle failures: improve the AI, modify your processes, or introduce human oversight
• Avoid the million-dollar fine-tuning trap – Real examples of why custom models become obsolete within 18 months and when fine-tuning makes sense
• Turn your “data heaps” into AI gold – How to assess and organize disconnected data silos before implementing AI, plus why most organizations fail at this critical first step
• Design systems, not magic genies – Why thoughtful system prompts and workflow design deliver 10x better ROI than chasing the latest AI model
• Handle AI’s “alien” failure modes – Understand how probabilistic systems fail differently than traditional software and build processes that expect interpretation errors
• Find your real competitive edge – Why your IP isn’t in having a custom model but in process design, context setting, and treating AI as “10,000 eager interns”
• Know when waiting beats racing – Recognise when today’s expensive problem (like the “six finger problem”) will be solved by next year’s base models