Multimodal AI Transformation

Included Health, 2025 · Case study by Will Dzierson

Tags: UX, Prototyping, LLMs, Machine Learning, AI

Project link: https://y4qwrjnnsy2vya5v.vercel.app/

My contribution

As the lead on this initiative, I was responsible for the full prototype lifecycle — from conversation flow design and prompt engineering to TypeScript/Vercel implementation with OpenAI and ElevenLabs. I collaborated directly with clinical, product, and leadership stakeholders to validate and ship the experience, and championed AI-assisted design workflows across the team, introducing tools like v0, Cursor, Lovable, and GitHub Copilot.

The work

At Included Health, I designed and prototyped a number of AI-based experiences to explore how natural language interfaces could streamline patient intake, streamline physician workflow, reduce documentation time, and improve accessibility. The goal was to evaluate whether a conversational experience could make healthcare interactions more human, intuitive, and efficient — especially in high-friction, high-stakes environments like telehealth and care navigation.

I led the end-to-end experience design, from conversation flow mapping and prompt engineering to interface prototyping and voice synthesis integration. The prototype supported real-time, conversational data capture and retrieval, enabling users to speak naturally and receive intelligent, context-aware responses from the assistant.

To bring the prototype to life, I used Cursor and v0 to build the interface, and integrated OpenAI's language models for NLU and ElevenLabs for synthetic voice output, creating a lifelike interaction loop. I collaborated with clinical ops and engineering stakeholders to align the voice experience with real-world care workflows and regulatory considerations.

The project was always dual-purpose: redesign a broken flow, and learn something meaningful about patient preferences for AI in healthcare. We ran a structured Maze experiment to test all three experiences (original UI, chat-based LLM, and voice-based LLM) head-to-head, and voice came out ahead by a wider margin than anyone anticipated — a finding that shifted how the team thought about conversational AI across the board.

The research directly informed the design language and interaction patterns for AI experiences across several subsequent projects at Included Health, establishing a foundation for how the team approached conversational UI going forward.

Images and video