Vocal accessibility for Voice User Interfaces

WHAT WE COVER

Story of VUIs

Story of VUIs

VUIs have become an integral part of our lives

Voice User Interfaces (VUIs) are fascinating, aren’t they? They’ve become an integral part of our lives, from chatting with our virtual assistants to controlling smart home devices—all with the power of our vocal cords. But, we also need to check that VUIs serve everyone, including those with disabilities in addition to offering convenience and accessibility.

Accessibility design and research has traditionally focused on physical accessibility. While in the last decade there has been some progress made on accessible human-computer interaction, it is still a relatively new field of research with limited practical applications. One area that has a huge potential implication, yet remains largely undefined, is the idea of making voice interactions accessible (Vocal Accessibility (VA)), especially given the rapid development of new vocal technologies in recent years. As a result, everyday tasks that have traditionally relied on a largely visual communication – such as banking, navigation, and learning – provide rich territory and opportunity for innovation with new forms of accessibility.

Let’s dive into some considerations for designing accessible VUIs:

VUIs should be linguistically flexible. They encounter diverse accents, dialects, and unique speech patterns. Imagine a VUI that understands both a Texan drawl and a posh British accent—now that’s inclusive!
But it’s not just about regional differences. Consider users with physical or vocal impairments. Maybe someone has a speech impediment or uses an augmentative communication device. Our VUIs should be ready to listen and understand them all.

Digital accessibility is universal and not just a niche concern. It’s all about making technology compatible for everyone, whether you’re an 85-year-old elderly or a multitasking working wife who wants to change the background music with a simple “Hey Alexa, next track.”
So, when designing VUIs, think beyond the obvious. It’s not just about disability; it’s about creating an inclusive experience for all.

VUIs shouldn’t inadvertently create new obstacles. Imagine a scenario where someone with a hearing impairment struggles to use a voice-controlled system because it lacks visual feedback. Not cool, right?
Developers and designers must ensure that VUIs don’t replace one barrier with another. Let’s break down walls, not build them!

Some users might need more time to process information. Offering adjustable response speeds can be a game-changer. It’s like having a patient friend who waits for you to catch up in a conversation.

Complex tasks or responses can leave anyone scratching their head. For users who require more context, provide options for detailed explanations. It’s like adding footnotes to your VUI conversation.

Remember, VUIs are like friendly companions—they should listen, adapt, and make life easier for everyone. So, whether you’re asking Siri for the rain conditions or telling Alexa to lower the light intensity, we should always be sure that these voice interfaces are as friendly as a hot cup of coffee on a chilly day!

Let’s understand the scenario of VUIs with an ongoing communication between two friends Stella and Jon

They are trying to develop certain VUI standards for their company. They are discussing the Challenges, Importance of VUIs and Design Approaches. Let's take a look into their conversation.

Stella:

Designing for voice-enabled experiences requires a deep understanding of how humans communicate, think, and interact. It involves crafting user interfaces that any person can access but also for people with special abilities, as there are certain challenges faced by them while accessing the VUIs.

Jon:

What do you mean by that Stella?

Challenges

Visual imparairment

Complete loss of sight (complete blindness), partial loss of sight (partial blindness) and inability to see or differentiate certain colors (color blindness)

They cannot see any visual feedback given by the device indicating it is listening (blue ring light of Amazon’s Alexa, colored dots by Google Home, small circular animation for Siri)
Display prompts given on the screen cannot be seen & need to be read out.
Speech rate of voice interface in speech readers is terribly slow.
When the device responds with ‘okay’ ‘done’ etc., it cannot be understood by the user what action got completed.
The device designs are not intuitive.

Hearing Impairment

Complete deafness, hard of hearing and, deaf and mute

It is difficult to perceive or hear sounds coming from voice assistants.
Users who are deaf not mute will not be able to hear the agents prompts.
It may be difficult to listen to the agent at average volumes.
Users with hearing impairment cannot hear the earcon.

Cognitive & Neurological disorders

Trouble remembering, learning, understanding - dyslexia, Alzheimer's, autism

Long task journeys & complex interactions are difficult to complete.
Technical jargon, complex language & long prompts are difficult to understand.
Response time limits imposed by device may not be sufficient for user to complete utterance.
Pauses, repeated words, different pronunciations may confuse VUI.
Users with dementia may forget the term for waking up the VUI
Users may not understand the agent clearly due to the speech rate.

Speech Impairments

Inability to speak clearly, have an adequate speech rate, not speak words clearly and loudly

Uttering clear sentences with adequate voice intensity may be difficult for the users, often required by voice interfaces.
Pauses in between utterances, repeated words in utterances and different pronunciations may confuse the VUI.
Response time limit for an utterance may lapse if the user requires more time to formulate an utterance or has slow speech.

Motor Disabilities

Unclear or broken speech with low clarity

Users may have difficulty in accessing any buttons provided on device
Users may face inability to access a screen as multimodal option for their varying range of assistive devices.

Stella:

These are some scenarios faced by people with disabilities. There are some generic problem statements as well:

Activation of a skill, general weather checks, changing reminders and e-commerce shopping
Insufficient time to speak
Voice agent sending links to the mobile app to continue action breaking the flow.
Excessive use of earcons and sounds difficult to understand.
Multiple instances where the device would be speaking and fail to pause when asked to stop, repeatedly
Grammar errors are caused such as when asked ‘empty my shopping cart,’ the device failed to understand while ‘clear my shopping cart’ was understood.
Lack of carrying the context of a conversation.
Device not picking the difference between ‘b’ and ‘d’ due to similar sounds

Jon:

Got it Stella. So, how do you think we should go about it?

Importance of Voice enabled experiences

Stella:

Jon, we must first try to understand what the importance of a voice enabled systems for people with disabilities and then focus on what can be the design approach for enhancing experiences..

Jon:

Okay Stella, I think the importance lies in the accessibility, convenience, intuitiveness, personalization and a multimodal approach of the system.

Stella:

I agree Jon.

Provides an alternate option of input for those with limited mobility, enabling them to interact with technology with ease.
Allows individuals with visual impairments to access information and control devices hands-free.
Offers an accessible and inclusive interface for persons with disability.

Saves time and effort by enabling users to perform tasks and access information without the need for manual input.
Allows for seamless control of smart home devices, appliances, and entertainment systems using voice commands.
Reduces physical straining due to persistent use of traditional input methods.

Creating a more intuitive and user-friendly experience by leveraging natural language processing to act and react to voice commands.
Reduces the need for complicated navigation through menus and settings, making it easier for users to realize their goals.
Provides a more natural and conversational interaction, reducing the learning curve for users.

Allows for customization of voice commands and preferences to suit individual needs and preferences.
Enables users to create personalized routines and macros to streamline frequently performed tasks.
Improves accuracy and reduces the need for repetition by adapting to the user's voice and language patterns.

Combines voice input with other modalities, such as touch, gesture, or eye-tracking, to provide a more comprehensive and flexible interaction experience.
Enables users to switch between input methods based on their needs and preferences, ensuring that they can always find the most suitable option.
Providing multiple avenues for interaction and control augments the overall user experience.

By incorporating these key aspects, voice enabled systems can significantly improve the accessibility, usability, and overall quality of life for differently abled individuals, empowering them to interact with technology more independently and confidently

Stella:

Now that we know how important the design of such systems is, let us discuss on the design approaches for such systems.

Jon:

Certainly!

Design Approach

Opening patterns could include greetings (such as ‘hello’, ‘good morning’); news checks (such as ‘anything new?’, ‘what’s up’), welfare checks (such as ‘how are you’, ‘how’s it going’), name introductions or self-identifications (such as ‘I'm Dave’), identification requests (such as ‘may I know your name?’)

Closing a conversation can be done by farewell or by disengagement. The agent can end the conversation by asking the user’s name such as ‘Before you go, may I know your name?’, acknowledgement of help such as ‘thanks’ or just say farewell ‘bye’

Specific to the pace of speech, emphasis, pauses, sound icons (that are also called earcons) and other sound characteristics that enable better understanding of a conversation.

A comma plus a 350 millisecond pause (instead of a question mark or full stop) should be added after each list item so that each item in a list can be distinguished.

Earcons can be used to convey when the agent is awake and listening, when the agent has stopped listening, when volume has been changed, or mic is switched off.

An understandable & short set of options must be presented to the users, instead of longer and complex lists.

Extended telling is like story telling where more than one turn is taken to complete the sequence. It is like a step-by-step narration using continuers like ‘first,’ ‘then,’ ‘lastly.’ The agent responds step-by step and continues when the user responds by using continuers such as ‘uh-huh,’ ‘ok,’ ‘then’ etcetera. Continuers help the user to control the pace of the story.

Feedback can be given to users in 3 ways: Implicit, explicit, no feedback.

Explicit feedback: ‘you said, I want to buy tulips. Is that correct?’

Implicit: ‘Ok, tulips, which color would you like?’

No feedback: ‘Ok’

Explicit & implicit feedback are important when financial transactions are occurring, or essential information is being input.

Feedback patterns also include the earcon sounds given by the voice agents to inform the user when they are listening and awake. Short acknowledgements can also be used in-between to assure that the interface is listening.

Comparative study patterns can be provided to enable the user to differentiate products or services based on their parameters. Comparative study inquiries ask the agent to create comparisons.

Examples: 'which is better for health, curd or milk?'; 'compare the two laptops in my cart' or 'Compare the size of these two mobiles.'

Cookie patterns inspired from graphical interfaces. To ensure privacy in using applications with voice interfaces or applications on a voice device, many personal details such as voice recordings, location may be accessed.

Unless clearly agreed to by the user, when any application is started, if it is accessing personal information or using cookies, the user must be given a choice to agree to it before proceeding, instead of assuming that they have agreed to it.

When a user has done an action repeatedly and frequently, it becomes a reflex and later just slips under the user’s consciousness.

On area of good defaults observed within voice interfaces was while shopping, where the default delivery address and payment method were automatically picked up from the account.

Since voice is an invisible medium, while keeping default choices for user, they should be confirmed.

Stella:

But there must be some design guidelines as well. Can you propose anything on these lines Jon?

Jon:

Let me take a dig into it for the multimodal experience.

Design Guidelines

Conversational components for a multimodal experience

The conversational components for a multimodal experience are: Spoken prompt, the display prompt on screen and clickable chips (chips are suggestions on how user can continue the conversation). The device with voice interfaces can be grouped by the components used for the response:

01.

Conversation on a speaker or headphones can only have spoken prompt.

02.

For conversations in a car or a smart display, spoken prompts, visuals and chips can be used.

03.

For conversations on a TV, Laptop, mobile; spoken prompt, display prompts, visuals and chips can be used.

Design guidelines for various elements

How content must be formatted on the display screen

Title: Establish a context for what is being displayed. Example: ‘Results for ‘Cow’s Milk Cheese’
Lists: Provide the user with at least 30 seconds to review and tap the displayed options. Let the voice agent tell the user “Let me know if you’d like to hear more.” While introducing a list, provide minimal and required information only. Avoid prompts like “You can say the name of the cheese or tap on the item” as they are instructive. Do not ask a question before presenting the list, since the user may begin speaking right after the question.
Since list items are like phrases, no punctuation is required, but if a list of questions is presented, do add a question mark in the list item’s display.
Specific Entity Search: In cases where a user’s search results in a specific result or one result only, for example a recipe, make sure that at least the title and image can be viewed from a distance. Divide information between the spoken and display prompt. The core of the message could be conveyed with prompts while the visuals can add supplemental information and details.
Content Priority: Present list items according to priority. Use visuals as supplementary information and avoid reading out exactly what is written on the screen.
Opening and closing a skill: An opening welcome prompt and closing prompt could be used to help the user orient themselves. Example: ‘Welcome to Cat facts. This also lets the users know that this skill or action had been invoked. While opening, suggestions for intents may be provided.
Null Result: When no results are found, help the user move forward by providing displayed suggestions such as Try: ‘Alexa tell me about Gouda cheese;’ but keep in mind that these displayed hints will be unavailable on a voice only interface.

Avoid asking any questions after the task has been completed as this frustrates the users. A user can control the screen display by speaking the following actions.

Scroll up/down/left/right.
Page up/down
Next/previous

Do not nest prompts within a list since these are difficult to select by voice. Avoid instructions such as "touch the screen". Do not give prompts that promote barge-in such as “When you hear the option you want, just say it.”

When reopening a skill for a repeat user, consider opening it with a display from where they left off in the last session. Prompt with a question to let the user know when it is their turn to speak. Include these questions in display prompts as well. Let the display prompts be condensed versions of spoken prompts, with the core of the message remaining the same. Responses for the user could be provided as chips. Chips must have the options provided in the agent verbal prompts.

Keep the voice and tone of spoken and display prompt consistent.

For help, examples can be provided for what the user can say. Avoid adding hints at the bottom of the screen when the voice agent is listening, since these might get covered in the blue light or audio wave patterns displayed there. Provide hints at the bottom when the screen is static.

For a list, the agent must reply in a conversational manner, but the displayed list must be organized. The user can either tap on the item or say the name of the item. Encourage users to select from a list but let them proceed via voice as well. Lists can be vertical or horizontal, along with scrolling feature.

Vertical lists are beneficial for long lists such as lists of sample utterances, lists of food items, timetables etc.
In horizontal lists, there is scope for visuals to take precedence. When unique images can help the user choose items such as movies, books, videos; provide a horizontal list.

Even when the visuals provide the best answer, make sure the prompts still carry the core of the message. Use the visuals to provide additional detail.

Typography: Text font sizes for a range from 3 to 10 feet. This will be different for devices such as a mobile phone. Do not use all uppercase or large blocks of text as these are hard to read. For long blocks of text, let users scroll by touch. Use different text styles to create content hierarchy.
Images and colors: Do not display any foreground or important content in background images. Keep background images with a 70% black layer opacity so that the image and text on it have sufficient contrast. Avoid using background images with text as the foreground text will clash with it. Remember that in dark-mode, all-black images are tough to view and in light mode, all-white images are tough to view.
Voice buttons (actions): Displayed phrases that act as voice buttons, for example ‘view pairings with earrings’ are actions that need to standalone in their own space. Display these prominently above the fold, in a line of their own, and not within body text. The displayed voice button and the utterance needed to activate must be the same. Action links must be differentiated from body text, buy use different of color, text weight etc. and all action links must be usable by voice.

General recommendations

Let the display add supplemental information and not replicate the voice experience.

Are you simply reading what is written on the screen? Do not do that. Instead, use consistent terminology, visuals, and labelling through the display screens. A screen display must launch within 2 seconds of the verbal prompt. Group the content thoughtfully so that user does not have to oscillate between the display prompt and visual. When multiple things are to be displayed, focus on one touchpoint at a time. For example, for checking a day’s routine, the user says, "What's my day like?". An online assistant may respond with details about the environment conditions, weather, date & time, commute conditions, and news content. Each type of content may again take up the full screen and is presented sequentially rather than all showing up at once on the screen.

Adoption Strategies and Accessibility

Make their onboarding procedure insightful and easy to follow
Use interactional tutorials to guide users through setup and features
Provide clear directions and value proposition to help users quickly comprehend the product's benefits

Get insights on user behaviors, preferences, and challenges
Segment or cluster the users based on their goals and tailor their experiences based on their journey
Understand the users and what drives them to accept and continue using the product

Offer various support channels like email, chat, instant messaging, and dedicated teams
Personalize support to address individual user needs and provide them timely assistance
Help users conquer obstacles and boost their confidence in using the product

Reward users who reach specific adoption milestones with discounts, upgrades, etc.
Implement a loyalty program where existing users earn rewards for referrals
Collaborate with influencers who align with the target audience to promote authentically

Onboarding should be an ongoing process that adapts as the company and user base evolves
Leverage data to identify friction points and make iterative improvements
Ensure onboarding remains relevant and valuable for all user segments

Define clear, trackable goals for your adoption strategy
Select key metrics like conversion rate, adoption rate, time to value
Implement a tracking plan to measure baseline metrics and gauge impact of changes
Review results, adjust, and repeat the process

Start by writing prompts for a voice-only interface, then think about how a screen might change the interaction. Design with voice-first in mind and use the screen or display to enhance the experience. If possible, make sure that everything on the screen can be activated with voice. The display screen could be made accessible with screen readers or screen magnifiers. (Echo spot and Echo show provide accessibility for their display devices with an included screen reader called VoiceView and a screen magnifier). At least keep the main activities of the application navigable by voice.
The key is to deeply understand your users, provide exceptional experiences, offer personalized support, and continuously optimize based on data. By implementing these strategies, you can drive higher product adoption, retention, and growth for your business.

Conclusion

Prioritize accessibility, convenience, personalization, and a multimodal approach

The design of voice user interfaces for disabled individuals must prioritize accessibility, convenience, personalization, and a multimodal approach. By focusing on these key areas, we can create inclusive VUIs that empower users with disabilities, enhance their interaction with technology, and improve their overall quality of life. As technology continues to advance, it is essential to keep the needs of all users at the forefront of VUI design, fostering a more inclusive digital landscape. It is important to adopt the design patterns while designing for accessibility of voice user interfaces (VUI). In Summary, some of the ways to adopt Vocal Accessibility(VA) of VUIs, are as follows:

While designing task flows and journeys of any activity that will be done with voice, think about brevity of agent responses and prepare the agent to being able to answer a variety of responses. The agent can be trained with users who have speech impairment as well.
Testing device design parameters (from buttons on the device, to intuitive gestures) and task flows of the activities that can be done with voice, with a diverse range of users having different abilities. A heuristic evaluation of a voice application can also be done, like the way it is done for graphical interfaces.
While designing for voice user interfaces, verbalize the script and read it out aloud. The script sounds different when read and when said out aloud.

Incorporating screens or displays as a fallback modality in voice user interfaces is essential for creating accessible, user-friendly systems. The display and the voice interface must work in sync, such as, if three options are being read out, those three options must be visible on the screen as well. The voice interface and the graphical interface must not be mutually exclusive. Their functionality should overlap and enhance the experience. A graphical interface is required for people with hearing impairment. The integration of visual elements alongside voice commands enhances the overall experience, ensuring that users can interact with technology effectively, regardless of their abilities. By prioritizing accessibility, convenience, and personalization, designers can create VUIs that empower all users, fostering inclusivity and enhancing the quality of life for individuals with special abilities.

About the Authors

Dr. Charudatta Jadhav

Scientist, Accessibility Research & Innovation, TCS

Dr. Charudatta Jadhav is Principal Scientist and Heads the Accessibility Research & Innovation at TCS. He has over 35 years of extensive experience and has been a key architect and lead consultant in establishing the accessibility practice within TCS. He is nominated as Advisory Board Member on United Nations WIPO – ABC initiative or Accessible Books Consortium. Besides, he is advisory and member of working groups on many national and international Standards bodies. He has also filed fifty-one patents, thirty-four of which have been granted and has also published 50 plus Scientific White Papers, 7 book Chapters and other publications. Among the many awards he has received, few mentions, Recipient of National Award for Best Disabled Employee, conferred on him by then President of India Hon. Dr. APJ Abdul Kalam.

Farajana Shaikh

Principal Innovation Evangelist, R&I , TCS

With nearly 25 years of experience in the field of customer experience, accessibility, and design, Farajana Shaikh is a Principal Innovation Evangelist at Tata Consultancy services Research and Innovation unit. She is a Certified Professional in Accessibility Core Competencies (CPACC) by the International Association of Accessibility Professionals (IAAP). As Inclusive thinker and accessibility strategist, Farajana works with several TCS global clients advising them in their accessibility to inclusion journey. She is a strong believer that accessibility leads to inclusion and inclusion leads to innovation which in turn lead to growth, transformation, and brand experience. She dreams of equal opportunities to every single person on the earth and has a strong belief that technology has a power to transform the lives of everyone but especially people with disabilities.

Dr. Arpita Sen

Innovation Evangelist, TCS

Dr. Arpita Sen is an Innovation Evangelist at TCS. She has nearly 19 years of experience in evangelism and product management, covering a range of technologies and domains across telecom, insurance, healthcare, biopharma, retail, accessibility and the internet of things. She seeks to work in ensuring technology is accessible to all and is passionate about its advocacy. She aims to raise awareness and drive positive change in the accessibility space with emerging technologies.

Contact

TCS is here to make a difference through technology.

We deliver excellence and create value for customers and communities.

Extraordinary expertise leads to remarkable results.

Want to be a global change-maker? Join our team.

Find the latest news about TCS in our Newsroom

Recent Press releases

Recent News

Recent recognitions

Upcoming events

TCS works hand in hand with world-leading investors.

WHAT WE COVER

Story of VUIs

Accommodate Different Speech Styles

Digital Accessibility Matters

Avoid New Barriers

Adjustable Response Speeds

Detailed Explanations

Challenges

Visual imparairment

Hearing Impairment

Cognitive & Neurological disorders

Speech Impairments

Motor Disabilities

Importance of Voice enabled experiences

Design Approach

Conversation Patterns

Sound Patterns

Extended Telling Pattern and Lists

Feedback Patterns

Search & Inquiries Patterns

Cookie Pattern for Privacy & Security

Human Behavioral Patterns

Design Guidelines

Conversational components for a multimodal experience

Design guidelines for various elements

Content Design

Navigational Design

Prompt Design

Error Handling & Help

Lists

Visual Specifications

General recommendations

Adoption Strategies and Accessibility

Create an Exceptional First-Time User Experience

Know Your Users

Provide Personalized Support

Incentivize Adoption

Continuously Improve Onboarding

Measure and Optimize

Implement

Conclusion

About the Authors

Dr. Charudatta Jadhav

Farajana Shaikh

Dr. Arpita Sen

Accessibility Adjustments