Case Study · Accessibility · HMI Research

Autonomous
Rideshare

A multimodal HMI that bridges the accessibility gap — giving blind and low-vision riders the confidence, safety, and autonomy they deserve.

Role UX Researcher (team project, publication co-author)

What I Owned ·Co-designed the between-subjects protocol (3 conditions × 6 tasks)
·Conducted 8 of 24 Wizard-of-Oz sessions (moderator + post-session interview)
·Co-authored the codebook; coded 8 of 24 transcripts; resolved discrepancies through consensus with teammates
·Contributed to Kruskal-Wallis interpretation
·Co-authored the Methods and Results sections of the SAGE manuscript
·Conducted the 27-paper literature review

Duration 2.5 months

Study design Between-subjects, N = 24

Read the study Jump to results ↓

Autonomous rideshare concept vehicle with screen showing a visual impairment simulation of 20/200 acuity

TL;DR

25M Americans face transportation insufficiency due to sensory, cognitive, or motor impairments. Existing autonomous-vehicle interfaces are vision-first. We designed a multimodal HMI (rear-seat display + voice + tactile buttons) and tested it against a visual-only control. Audio restored trust, identification accuracy, and navigation performance for visually impaired riders to near-parity with sighted users (p < .05). Visual-only performed significantly worse.

Published research

This study was published in SAGE Publications (2025). A between-subjects experiment with 24 participants across three conditions, testing whether a multimodal (audio + visual) HMI restores autonomous-vehicle usability for blind and low-vision riders.

Read the full paper →

The Problem

Most autonomous HMIs are vision-first.

Imagine you were injured and didn't have a partner or friend to drive you where you needed to go. How would your life be impacted? For 25 million Americans, this is everyday reality — transportation insufficiency stemming from cognitive, sensory, and/or motor impairments.

Without reliable, usable transportation, these individuals are further left behind and isolated from society. Autonomous vehicles promise independent mobility, but most human-machine interfaces are built for sighted users first.

Blind and low-vision riders face barriers at every stage: identity verification, trip confirmation, en-route updates, unexpected events, and safe exit. We set out to design a multimodal interface that restores confidence, safety, and autonomy.

25M

Americans alone experience transportation insufficiency stemming from cognitive, sensory, and/or motor impairments — left further behind and isolated from society.

The Idea

A human-centered, multimodal HMI (audio + visual) that bridges the accessibility gap in autonomous ridesharing.

01 — Discovery

27 articles. One clear gap.

Because fully autonomous vehicles are so new and rapidly changing, we set a publication boundary of 2014 to capture the most accurate, up-to-date research. After keyword searching and abstract filtering, 27 articles were identified and read in full.

Articles reviewed in literature review

The literature review began with broad keyword searches, then was filtered by relevance and recency. The publication date boundary was set at 2014 to capture the most current research on fully autonomous vehicles.

accessible HMI accessible rideshare visually impaired HMI visual impairment design fully autonomous vehicles multimodal design

Real-World Context

Wizard-of-Oz simulation setup for usability testing

Wizard-of-Oz simulation environment used for usability testing — replicating an in-vehicle rear-seat experience with the multimodal prototype.

02 — Define

Six tasks. Every failure point mapped.

We broke down the full ridesharing process with impairment needs included — from boarding to exit. Each task became a study condition to measure trust, navigation accuracy, and satisfaction.

Task 01

Identity Verification

Speak a 4-digit ride code to confirm rider-vehicle pairing — replacing visual QR codes that blind riders cannot use.

Task 02

Trip Confirmation

Destination read back via audio. Rider verbally confirms or corrects the address before the vehicle departs.

Task 03

Driving Updates

Proactive status narration: intersections, signal changes, ETA updates — keeping riders continuously oriented.

Task 04

Unexpected Events

A special mode activates to explain hazards and evasive actions — maintaining trust when the vehicle deviates from expected behavior.

Task 05

Destination Arrival

A 3-minute pre-arrival alert prepares the rider with curbside context — orientation, surroundings, and safe disembark preparation.

Task 06

Exit Interaction

Guided, safe, step-by-step disembarkation — ensuring riders safely exit without risking traffic or disorientation.

Methodological clarification

We used the Cambridge Disability Simulator for methodological consistency across participants — recognizing that simulation captures sensory degradation but not the adaptive expertise of users with lived visual impairment. Recruitment of actual blind and low-vision riders is the highest-priority next step.

Design Principles

Principle 01

Memory Supports

Predictive cues, consistent language, and knowledge-in-the-world reduce cognitive load for riders with memory or cognitive impairments.

Principle 02

Perception Optimization

Low access cost, redundancy across modalities, and progressive disclosure ensure every rider can receive critical information regardless of vision level.

Principle 03

Attention Management

Salience, urgency mapping, and interruption handling ensure high-priority information reaches the rider at the right moment — without overwhelming.

Hypotheses

Audio feedback improves navigation confidence and route awareness for visually impaired riders.

Multimodal feedback enables visually impaired users to perform comparably to non-visually impaired riders.

Removing audio feedback decreases rider trust and situational awareness.

03 — Prototype

Three components. One multimodal system.

The prototype combined a rear-seat mounted display, a natural-voice external speaker, and high-contrast tactile hardware buttons — designed to work seamlessly together across all six trip tasks.

🖥️

Rear-Seat Display

Mounted display for route status and environmental descriptions. High-contrast visual output synced with audio narration for redundancy.

🔊

Voice — "Maple"

External speaker with a natural, calm voice character named Maple. Synchronized audio prompts across all six tasks — proactive, not reactive.

🔘

Hardware Buttons

Two high-contrast tactile buttons for physical interaction — no touchscreen required. Color-coded for disambiguation.

■ #F4B300 · ■ #B87AFF

Wizard-of-Oz prototype in testing environment

Wizard-of-Oz simulation: the prototype in a controlled test environment replicating a rear-seat AV experience.

Location of components of multimodal system

Rear-seat display.

Emergency assistance tactile button.

04 — Experiment Design

Between-subjects. Three conditions.

A between-subject design with N = 24 participants across three conditions — designed to isolate the effect of multimodal audio on trust, satisfaction, and navigation accuracy for visually impaired riders.

NVI

Condition 1 — Non-Visually Impaired

Sighted participants using the full multimodal interface — audio + visual. Baseline condition for comparison.

Condition 2 — Visually Impaired + Audio

Visually impaired participants (Cambridge Disability Simulator, 20/200 acuity blur) using the full multimodal interface — audio + visual.

VIX

Condition 3 — Visually Impaired, No Audio

Visually impaired participants using visual-only interface — no audio. Control condition to isolate the contribution of audio to the experience.

Measures & Methods

📏

Quantitative Measures

Trust, satisfaction, and navigation accuracy measured on Likert scales. Statistical analysis via Kruskal–Wallis with post-hoc testing.

🔬

Disability Simulation

Cambridge Disability Simulator standardized visual impairment at 20/200 acuity blur across all impaired conditions — ensuring consistency.

📝

Qualitative Input

Interviews and literature review confirmed preference for audio-visual redundancy. Key themes: clarity, directionality, and trust in the system voice.

Results

Audio narrowed the gap.

Across all three key measures — trust, correct vehicle identification, and navigation — the multimodal condition restored performance for visually impaired riders to near-parity with sighted users. Visual-only was significantly worse.

Finding 01

Trust & Satisfaction

VI-with-audio was still different from NVI, just less varied than VIX was. Audio narrowed the gap but didn't fully close it.

Finding 02

Correct Rideshare Identification

Audio narration restored confidence in vehicle identification for blind and low-vision riders — a task that was frequently failed in visual-only conditions.

Audio restored correct identification

Finding 03

Navigation Understanding

Proactive narration of intersections, signals, and ETA changes significantly improved route understanding and rider comfort throughout the trip.

Proactive narration improved comfort

H1 was supported

H2 was supported

H3 was supported

Hypothesis 1 results chart showing audio feedback improved navigation confidence and route awareness for visually impaired riders

H1 results — audio feedback improved navigation confidence and route awareness for visually impaired riders.

Hypothesis 2 results chart showing multimodal feedback helped visually impaired users perform comparably to non-visually impaired riders

H2 results — multimodal feedback enabled visually impaired users to perform comparably to non-visually impaired riders.

Hypothesis 3 results chart showing removing audio feedback decreased rider trust and situational awareness

H3 results — removing audio feedback decreased rider trust and situational awareness.

Radar chart comparing non-visually impaired, visually impaired, and visually impaired without audio conditions across route and navigation, destination, rideshare identification, feedback, help, and satisfaction measures

All-results radar chart — multimodal audio-visual feedback improved visually impaired riders' route awareness, destination confidence, rideshare identification, and overall satisfaction compared with the visual-only condition.

Prototype & Mockup

Unimpaired vs. impaired view of the rear-seat display — showing how visual degradation affects screen readability without audio support.

Wizard-of-Oz simulation used for usability testing — replicating the rear-seat AV environment with synchronized audio and visual output.

Full system mockup in autonomous vehicle

Full multimodal system mockup — as it would appear if fully autonomous vehicles existed and were deployed at scale.

Next Steps

Where this research goes next.

The current prototype validated the multimodal approach in a controlled Wizard-of-Oz setting. Five directions would move this toward real-world deployment.

🚗 Field trials with blind and low-vision riders in real autonomous vehicles across varied road conditions — moving beyond the simulation environment.
📳 Introduce haptics — seat vibration and localized speakers — for tri-modal redundancy. A third channel would further close the gap for riders with combined sensory impairments.
🎛️ Personalization preferences: rider control over verbosity, pace, tone, and cue frequency — allowing the system to adapt to individual needs and anxiety levels.
🛡️ Safety sandbox studies for edge cases: reroutes, obstructions, and emergency stops — the high-stakes moments where trust is most fragile and communication most critical.
📊 Replicate with a 2x2 design (impairment x audio condition) to isolate whether sighted users also benefit from audio narration or are unaffected — strengthening the causal claim about audio's role.

AutonomousRideshare