As AI becomes easier and cheaper to deploy, designers face a new challenge: deciding not just what can be automated, but what should be. Lou Rosenfeld talks with Joy Kendi of Dalberg Design — and an upcoming speaker at the Designing with AI 2026 conference — about introducing AI into high-stakes systems where design decisions can directly affect access to healthcare, services, and critical resources.
Joy shares examples from her work in public health systems, including an AI concept intended to help community health workers prioritize patient vis...
As AI becomes easier and cheaper to deploy, designers face a new challenge: deciding not just what can be automated, but what should be. Lou Rosenfeld talks with Joy Kendi of Dalberg Design — and an upcoming speaker at the Designing with AI 2026 conference — about introducing AI into high-stakes systems where design decisions can directly affect access to healthcare, services, and critical resources.
Joy shares examples from her work in public health systems, including an AI concept intended to help community health workers prioritize patient visits. What initially seemed like an obvious efficiency gain quickly raised deeper concerns around incomplete data, shifting real-world conditions, and the irreplaceable role of human judgment and community trust. Rather than treating AI as a decision-maker, Joy argues for designing systems where AI supports — but does not override — human expertise.
Their conversation also explores how AI is reshaping the role of designers themselves. Joy makes the case that designers must move “upstream” in the process, helping define boundaries, risks, trust, and governance before automation decisions are made.
What You'll Learn from this Episode:
- How Joy Kendi defines “high-stakes” design systems
- Why automation can fail in public health contexts
- The risks of relying on incomplete or outdated data
- Why AI should support human judgment, not replace it
- How designers are moving upstream in AI decision-making
- What human-centered AI governance can look like in practice
Quick Reference Guide:
0:11 - Meet Joy and learn about high-stake systems
3:26 - Joy’s backstory
6:10 - Seeing beyond the everyday facade
8:55 - Why you need the Rosenverse
11:08 - Designing responsible AI for high-stakes public health decisions
16:06 - Why human judgment still matters in AI-assisted decision-making
18:29 - Designers must shape AI boundaries, not just interfaces
22:06 - Joy’s gift for listeners
Resources and Links from Today's Episode:
Designing with AI 2026 https://rosenfeldmedia.com/designing-with-ai/
Quotes:
“When I say high stakes systems, I mean systems where the design choices carry very real consequences beyond the typical interface or screen that a lot of designers are used to.”
“The hardest question is not just can we build this tool. It’s also deciding whether we should automate some of these complex social decisions at all.”
“Even when the quality of data is good, and the data is available, there's more complexity to how human beings make decisions. A lot of that is also based on the social dynamics.”
View more
