publications

2023

1. 

   Toucha11y: Making Inaccessible Public Touchscreens Accessible

   Jiasheng Li, Zeyu Yan, Arush Shah, Jonathan Lazar, and Huaishu Peng

   In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 2023

   Abs DOI Video PDF

   Despite their growing popularity, many public kiosks with touchscreens are inaccessible to blind people. Toucha11y is a working prototype that allows blind users to use existing inaccessible touchscreen kiosks independently and with little effort. Toucha11y consists of a mechanical bot that can be instrumented to an arbitrary touchscreen kiosk by a blind user and a companion app on their smartphone. The bot, once attached to a touchscreen, will recognize its content, retrieve the corresponding information from a database, and render it on the user’s smartphone. As a result, a blind person can use the smartphone’s built-in accessibility features to access content and make selections. The mechanical bot will detect and activate the corresponding touchscreen interface. We present the system design of Toucha11y along with a series of technical evaluations. Through a user study, we found out that Toucha11y could help blind users operate inaccessible touchscreen devices.

2. 

   A Framework for Active Haptic Guidance Using Robotic Haptic Proxies

   L. Niall Williams, Jiasheng Li, and C. Ming Lin

   In 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023

   Abs DOI Video PDF Project Website

   Haptic feedback is an important component of creating an immersive virtual experience. Traditionally, haptic forces are rendered in response to the user’s interactions with the virtual environment. In this work, we explore the idea of rendering haptic forces in a proactive manner, with the explicit intention to influence the user’s behavior through compelling haptic forces. To this end, we present a framework for active haptic guidance in mixed reality, using one or more robotic haptic proxies to influence user behavior and deliver a safer and more immersive virtual experience. We provide details on common challenges that need to be overcome when implementing active haptic guidance, and discuss example applications that show how active haptic guidance can be used to influence the user’s behavior. Finally, we apply active haptic guidance to a virtual reality navigation problem, and conduct a user study that demonstrates how active haptic guidance creates a safer and more immersive experience for users.

2022

1. 

   TangibleGrid: Tangible Web Layout Design for Blind Users

   Jiasheng Li, Zeyu Yan, Ebrima Haddy Jarjue, Ashrith Shetty, and Huaishu Peng

   In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, 2022

   Abs DOI Video PDF

   We present TangibleGrid, a novel device that allows blind users to understand and design the layout of a web page with real-time tangible feedback. We conducted semi-structured interviews and a series of co-design sessions with blind users to elicit insights that guided the design of TangibleGrid. Our final prototype contains shape-changing brackets representing the web elements and a baseboard representing the web page canvas. Blind users can design a web page layout through creating and editing web elements by snapping or adjusting tangible brackets on top of the baseboard. The baseboard senses the brackets’ type, size, and location, verbalizes the information, and renders the web page on the client browser. Through a formative user study, we found that blind users could understand a web page layout through TangibleGrid. They were also able to design a new web layout from scratch without the help of sighted people.

2. 

   Calico: Relocatable On-cloth Wearables with Fast, Reliable, and Precise Locomotion

   Anup Sathya, Jiasheng Li, Tauhidur Rahman, Ge Gao, and Huaishu Peng

   Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2022

   Abs DOI Video PDF

   We explore Calico, a miniature relocatable wearable system with fast and precise locomotion for on-body interaction, actuation and sensing. Calico consists of a two-wheel robot and an on-cloth track mechanism or "railway," on which the robot travels. The robot is self-contained, small in size, and has additional sensor expansion options. The track system allows the robot to move along the user’s body and reach any predetermined location. It also includes rotational switches to enable complex routing options when diverging tracks are presented. We report the design and implementation of Calico with a series of technical evaluations for system performance. We then present a few application scenarios, and user studies to understand the potential of Calico as a dance trainer and also explore the qualitative perception of our scenarios to inform future research in this space.