�Թ���

October 13, 2020

For people who are blind or visually impaired, tasks such as navigating through unfamiliar environments can be difficult. Navigation is part of a set of skills collectively called Orientation and Mobility (O&M), and researchers have shown that one’s proficiency with this set of skills is pivotal to allowing people to access social and economic opportunities in their community.

Due to its importance, engineers have created many kinds of technology to assist with tasks such as navigation, but these efforts have a mixed track record, according to Paul Ruvolo, �Թ��� Associate Professor of Computer Science. The reasons why, he says, are technological (expensive hardware and brittle algorithms) and methodological (users not being involved enough in the design process).

Ruvolo has spent the past six years addressing these shortcomings. Alongside �Թ��� students and in partnership with members of the blind community from all around the world, he’s exploring how to use emerging mobile technologies to enable radical improvements in access to physical spaces for people with visual impairments.

Technology, Accessibility, and Design

Creating assistive technology wasn’t specifically what Ruvolo set out to do. During his seven years at �Թ���, he shifted his focus from researching theoretical problems in machine learning and computer science to using tools from these same fields in concert with approaches from human-centered design to achieve real-world impact in the areas of accessibility and inclusivity. “I was initially attracted to working on problems in assistive tech because of a fairly naïve belief that whiz-bang technology would be the key to helping people,” he says.

“Being in the �Թ��� learning community, with colleagues who look at societal issues through a variety of disciplinary lenses, helped me develop an approach to my work that put the user and their experiences at the forefront and placed technology in an important but decidedly supporting role.”

�Թ���’s flexible, non-traditional criteria for evaluating faculty mean that he was freed from the typical publish-or-perish treadmill that dominates much of academia. “At �Թ���, I don’t have focus on using a particular technology because we can publish a paper on it; it’s about, how can we create systems for people that they will find truly empowering, and the interesting technological problems grow naturally from this pursuit,” says Ruvolo. “I’ve always found this to be a more compelling and fulfilling way to work and I’m grateful that �Թ��� allows me to pursue this approach.”

Ruvolo has taken dozens of �Թ��� students along with him on this journey. During summers, he’s mentored 40 undergraduate student researchers, who work on the same projects that he’s working on, as well as several students each school year. And in 2018, Ruvolo and Caitrin Lynch created a course called Technology, Accessibility, and Design. In this class, students learn historical and societal context about visual impairment and blindness, build design skills for creating accessible technologies, and work on a team-based project with a client to improve accessibility for people who are blind or visually impaired.

The collaborations fostered by the class between �Թ��� students, with their engineering and design skills, and the many people from the Boston area who work for institutions that serve the blind, enabled compelling final projects. For example, one team created a tethering system to allow a sighted guide and a runner who is blind to run ultra-marathons, and another team built a website for chefs who are blind with accessible recipes and tips for cooking without sight.

Leveraging the Smartphone

Ruvolo’s work is based in the belief that in order to have maximum impact on the quality of life of people who are blind, O&M technologyshould be cheap and easy to make available to the community. For that reason, several of his projects leverage mainstream smartphone technology.

One exciting development in smartphones that has enabled his work is the introduction of highly accurate 3D-motion sensing algorithms to support augmented reality (AR). While these features were primarily designed to support gaming experiences for sighted people (like Pokémon Go), they have the potential to be repurposed to create O&M technology for people who are blind. “Smartphone technology has come a long way, and it’s an exciting time, as every new iteration of the device changes what’s possible,” says Ruvolo.

Ruvolo’s group developed and released , a free, open-source, indoor navigation app for people who are blind and visually impaired. The app uses smartphone-based AR technology to track a person’s motion using their mobile device to allow them to record, share, and navigate routes through environments such as schools, stores, apartment buildings, and offices. The app works by leaving a virtual breadcrumb trail through the environment to a visited location. The app can then use haptic and auditory feedback to guide a user along that route at a later point in time. Routes can also be shared with other users to allow people who have never traveled a particular route to benefit from a recording made by someone else.

The first version of Clew was coded in a summer, but the work that went into it took years. It’s the first, and currently only, widely adopted AR-based smartphone app for O&M. Over the last year and a half, it’s been downloaded more than 15,000 times and is used by 1,000 unique users a month from all around the world for anything from navigating open concept offices at work to establishing a route from a hotel’s front door to the elevator, so they can follow it later on.

Taking Tech to the Next Level

Ruvolo was recently awarded a three-year, $343,800 National Science Foundation grant to further this work. His project, , builds upon Clew’s success. And while the response to Clew has been encouraging, getting feedback from users all around the world is a challenge, and there are opportunities to improve the app. In order to address this, he’s assembling a team that includes �Թ��� students, experts in the field of O&M, and dozens of co-designers—both with and without visual disabilities. “Instead of working with just a handful of people through co-design sessions at �Թ���, we are seeking to collaborate with hundreds of people from around the world to make navigation technology that works for them in their local environment. Figuring out how to do this effectively is our key research challenge,” says Ruvolo.

Co-designers will evaluate features, provide feedback, and use their smartphones to collect rich data streams. This data will be processed using new machine learning and crowdsourcing algorithms to extract information, like a person’s location and the location of nearby objects, creating a first-of-its-kind dataset to build into Clew “2.0.” and other smartphone apps.

A uniquely �Թ��� effort

Ruvolo is using the grant to involve the �Թ��� community and other educators. In addition to mentoring student researchers, he will teach more Technology, Accessibility, and Design courses. “�Թ��� students already have an accessibility mindset and they’re eager to think about assistive technology and user-centered design,” he says. “These educational experiences can help them develop an accessibility mindset.”

Ruvolo is also planning to run external faculty development workshops at �Թ���’s annual Summer Institute, which draws faculty from around the world. In the summer of 2018, Lynch and Ruvolo offered a faculty development workshop of this type. The goal of these workshops is to enable faculty to use tools to create courses that motivate their students to design for accessibility and inclusion. “Clew is an interesting place to start,” he says. “What I really hope to accomplish is raising awareness of the potential of this kind of technology. If the work that my collaborators and I do inspires someone or some company to create something that radically improves equity for people who are blind or visually impaired, I would consider that a tremendous win.”

Read a press release about this new grant.