As part of a graduate studio class, Microsoft asked for concepts that demonstrate the value of Conversational User Interface (CUI) to achieve a symbiotic human-computer experience. They wanted us to explore where, how, and in what ways conversational user interfaces could support human-computer symbiosis in the near future.
I worked in a group of designers and researchers to conduct research for and through design artifacts like journey maps and prototypes. We iteratively designed and tested our conceptual interface through the course of the 2-month long project.
How can we support the experience of Emergency Medical Technicians inside ambulances through a conversational user interface (CUI)?
How can a CUI integrate disparate elements within the emergency healthcare network?
How can we improve the front-of-ambulance transport experience by designing an intelligent conversational user interface?
We looked at possible applications in autonomous vehicles. Interviews with engineers and automobile interface designers on one side, and EMTs on the other, pointed to several points of tension.
I designed the research plan as sprints. It included exploratory research to understand the context of emergency services through semi-structured interviews and ethnographic ride alongs, generative research of a co-creative nature, and iterative evaluative research. Each research sprint was preceded by a design sprint. The outcomes and questions arising from design was the center of the research activities of the following phase. We reached out to the Pittsburgh EMS Station and met with 3 EMTs. They served as our core participant group through the course of the project.
Early data collection activities included
// 16 hours of ridealong time in ambulances and stations
// 3 participatory workshops with EMTs that included the following activities:
Collaging activity to understand their ideal colleagues and benefits or perceived limitations.
Journey map of a typical call to understand breakpoints and opportunities in current processes.
Flexible modeling to explore how existing tools are used and how audio and visual interactions might be integrated.
We analyzed the data through affinity mapping and arrived at design themes. After blue-sky ideation sessions, we narrowed down to four concepts which we expressed through storyboards. These storyboards acted as provocations to arrive at final design directions.
Our design themes were based on the hypotheses that the CUI should aid safe navigation and transport, mediate coordination, and provide stress relief. We designed storyboards and a scripted conversation to demonstrate use cases and personalities.
Through testing the storyboards, we found that EMTs didn't want to deal with their stress directly but that we would have to alleviate them through other approaches in the design.
We developed a Wizard of Oz style of prototyping 3 different scenarios with 15 touchpoints to test the relevance and authenticity of the concepts, the forms of multi-modal feedback and how they would use a system like this individually, in pairs and across the larger organization of the EMS.
Transporting the patient is the main task; not treating the patient in the ambulance.
It is difficult to drive when you are breaking rules intentionally. Audio, visual and haptic indicators that you are breaking rules increases the chances of getting distracted. It is also difficult to anticipate the reaction of other drivers to the lights and siren, further complicating the act of driving.
Emotional strain is an expected part of the job but it is the unexpected events that cause stress.
EMTs make decisions based on protocols. But it is moments of decision making like route planning after receiving a dispatch call, driving, arriving at the patient's location and driving to the hospital that have a higher likelihood of unanticipated events occurring. This leads to bursts of stress throughout the day.
Decision-making logic needs to be communicated in order to build trust.
EMTs are confident and take pride in their expertise, including their independent decision-making skills. They want to understand the rationale behind certain decisions, like the best route to the hospital at a certain time of day, before taking action. They appreciate space for negotiation and conversation before starting the journey.
Knowledge of patterns in frequency, type, and instances of patient care is distributed across a team
Informal conversations at the station are the primary way of finding out what happened on someone else's shift. Learning is informal and happens on the job. Besides ambulance logs which are cumbersome to maintain, there are very little connecting teams of EMTs across locations and shifts.
Kara, the conversational agent, embodies a symbiotic relationship through co-action. Through shared logic, memory, and responsibility within an ambulance, Kara helps EMS personnel by managing the unknown risks and stresses they face.
Kara is a concept for an intelligent conversational agent in ambulances. Emergency medical technicians have clear protocols to stabilize patients. The transportation aspect that causes stress. Kara augments the foresight capabilities of medics through multimodal interactions. It doesn't just give suggestions but arrives at a decision through conversation and builds trust. It shares responsibility for the ambulance and manages shift handovers to facilitate conversations across teams. It shares contextually relevant information to prepare EMTs for any situation. Kara helps EMS personnel to achieve the shared goal of effectively transporting patients to the hospital.