Evolving Lookout

People who rely on accessibility apps deserve a user experience that goes beyond mere functionality; they deserve something user-centered, learnable, and even delightful. Elements like audio, video, haptics, and illustrations with alt text elevate the user experience. They can help users not just accomplish a task, but feel engaged, empowered, and respected.

I’m excited to share how we’ve approached this when evolving Lookout, an app that helps people who are blind or have low vision obtain information about their surroundings using the phone’s camera.

Graphic design is dominant in UX. We tend to prioritize the visual experience, with sound and haptics considered later in the design process, if ever. But for blind and low-vision users, we can craft beautiful design across sensory experiences.

For example, we know Lookout users don’t want to hear robotic voices speak text aloud. They prefer human-sounding voices that enrich and enhance their aesthetic experiences. We’ve also learned that some voices are clearer than others at faster speeds, which is how many people who regularly use assistive tech listen to audio.

Many of our users depend on a screen reader, which converts text and other UI elements into speech, to operate their phones. Supporting the screen reader experience is quite intensive for an app like Lookout, because the app’s active audio needs to “play nice” with the screen reader audio. We must ensure the two can coexist and still make sense to a user. And Lookout’s not a quiet app; it has a lot on its mind.

When it comes to people with some vision, beauty lies in clarity. When I choose an icon, I try to make sure it’s not convoluted. I use large, familiar symbols and aim for high contrast with the background. A contrasting background color and clearly defined borders can help users with visual impairments identify buttons even if they can’t make out the icon or text.

Lookout is designed to assist users with blindness and low vision by providing information about their surroundings through the camera. It has seven modes:

• Text: Read image text aloud
• Documents: Scan and read full pages of text
• Explore: Identify objects and text in the environment
• Currency: Identify currency denominations
• Food labels: Scan barcodes and recognize food products
• Find: Locate specific objects from a predefined list
• Images: Capture and describe images using AI

With the newest version, released in 2024, we put a lot of work into the Images mode. Previously, a user could upload an image and the app would generate a caption. Now, users can also capture images directly — using audio framing guidance if they need it. After hearing the description, they can ask follow-up questions, with answers powered by Gemini.

As lead designer, I advocated for adopting the latest Material 3 guidelines in our 2023 release — to beautify the app, bring it into alignment with other products, build on existing intuition, and take advantage of a more modern and accessible design system. At Google, the world is watching what we do — our approach, features, and design choices. If we invest in the experience of our accessibility apps, we inspire the larger ecosystem to do the same.

Before the update, Lookout’s primary navigation component was a horizontal scrolling navigation bar at the bottom of the screen. While we liked that it was always visible, the navigation bar took up a lot of space, kept growing as we released new functionality, and hid some of the mode options offscreen. We also learned that horizontal scrolling behavior can be physically challenging for some people, including screen-reader users who need to scroll with two fingers.

The Material guidelines limit navigation bars to three to five destinations. So we moved to a navigation drawer, which presents destinations in a scannable, vertical list and gives us room to increase the number of available modes.

We also leaned on Material’s guidelines for bidirectional UIs in order to launch Arabic support. In an English-speaking culture, we take for granted that people read left-to-right. Now, the app should feel more natural for somebody who has a right to left mental model.

Following the latest Material guidelines has many benefits. I’m color-blind myself, which often helps me immediately understand when contrast isn’t working. But stylistically, I rely on the design system because I know that a professional visual designer picked the color palette.

Respecting the design system can serve the usability and accessibility of the app. In the best-case scenario, the system encourages you to make the most accessible app possible. And in turn, we were able to give feedback to the Material 3 system designers about ways to enhance accessibility in their overall guidelines.

In the accessibility space, we have to take design changes seriously. We know that people rely on this technology to read a sign, read their mail, or find their way. Luckily, this field attracts compassionate professionals. Working on Lookout is meaningful and also enjoyable, because I collaborate with empathic individuals who understand the importance of codesigning with our users.

Going forward, I humbly ask that we try to observe accessibility issues in our daily lives as part of a continuous learning process. Even if you’re not a designer or developer, consider how the user experience and even the full life cycle of our products affect humanity. We all have the power and obligation to make responsible decisions, identify irresponsible ones, and pursue a better future for everyone.

We need to continue to push ourselves to prioritize accessibility and elevate the voices of users with disabilities through frequent, thoughtful, and genuine engagement. If we do this, we help give assistive tech and its users a functional and elegant experience, and we help leverage the power of tech for equality. Sometimes I joke that Lookout’s most helpful accessibility feature is that it’s free. I think there’s a lot of truth to that.

Illustrations by Arthur Vergani