In today’s modern society, technology is all around us and part of everyday life. Although some may feel that technology and artificial intelligence (AI) raises questions of ethics or argue our world may be too dependent on them, there is no denying the progress AI has made in accessibility. As AI continues to improve and expand in its capabilities, it offers an increasing benefit to the deaf and hard of hearing community. AI makes the world more accessible for individuals who are deaf and hard of hearing through services like captioning and speech recognition to positively impact how they consume media, learn and communicate in person and over the phone or internet.
Technological advances and AI have made media and communication more accessible than ever, these advances are not without their flaws and still have progress to be made. In the United States, thanks to rules established by the Federal Communication Commission (FCC), it is mandated that all video program distributors, including cable operators and broadcasters, provide closed captioning for TV programs. While this greatly helps to make TV shows and movies more accessible, there are still no standards in place for content on Youtube, Instagram or other similar platforms, which makes up an increasing amount of video content consumed. Additionally, many captions that are provided for may still be lacking as auto-generated captions are often inaccurate or not in sync with the audio track. Another area lacking in accessibility is customer service centers that are phone-only or automated systems which require the caller to navigate the system using voice only. As technology and AI continue to improve and allow for broader use of features such as speech to text and improved speech recognition, hopefully strides will be made to allow those who are hard of hearing easier access for greater independence and ease of use.
While AI is a concept that a majority of people have been introduced to or are familiar with, the term machine learning is likely more abstract. Machine learning is responsible for a majority of technology and automations encountered daily, from chatbots to suggestions for what movies or videos to watch and even autonomous vehicles. A subfield or application of artificial intelligence, machine learning is what allows a computer system to learn and develop intelligence. The computer system is able to continue to learn and improve on its own without direct programming by utilizing a process of mathematical models of data. Within machine learning there are various models and approaches that programmers can choose from, all of which impact how the model functions and its accuracy.
As artificial intelligence capabilities continue to expand, there are more applications to help facilitate communication between individuals who are deaf and hard of hearing and hearing individuals. AI has made in-person communication more accessible for the deaf community through advancements in sound isolation and lip reading algorithms. Researchers at Google have been able to create an AI application that can isolate a single individual’s voice from other speakers and background noise. Similarly to how a person without hearing loss or other difficulties can focus their attention on a particular speaker to better hone in on what they are saying, this technology is able to do the same to make it easier for the hard of hearing to hear more clearly, especially in a crowded or busy setting. Google is also responsible for creating an AI that is able to generate captions for those who rely on lip reading. Relying on algorithms, the AI system is now outperforming professional lip-readers, translating 46.8% of words accurately. While this technology is still learning and improving, future application to hearing aids and other assistive devices could greatly improve speech recognition for users.
Automatic speech recognition (ASR) allows for audio input to be converted into easy to read captions. In order to generate accurate captions, ASR relies on AI and machine learning to decipher what is being said and attribute context. ASR is used in a variety of situations to translate speech to text, both for in-person conversations as well as phone conversations and video calls. Another application of ASR is auto-generated captions on applications like TikTok, Instagram and Youtube which allow deaf and hard of hearing viewers to enjoy content. This same technology can also be used to transcribe in-person conversations through various applications like LiveTranscribe, displaying captions of what is being said to facilitate clearer communication. Applications like InnoCaption utilize ASR to provide real-time captions for phone calls to make phone conversations more accessible for the hard of hearing.
Artificial intelligence also plays a key role in making the classroom more accessible for deaf and hard of hearing students, making it easier for these students to perform their best academically. Colleges and universities like the Rochester Institute of Technology in Upstate New York have utilized automatic speech recognition to provide live-captions of lectures. As the professor wears a headset during class, captions are generated through Microsoft Translator and appear to the attending students. This allows deaf students to receive information at the same time as their hearing peers and allows them to focus on taking in the information rather than trying to keep up with the pace of the lecture.
Although AI is a key component of all captioning, as technology continues to advance closed captions are now becoming more applicable to those who use sign language as well. Closed captioning personalization technology has now been developed by the company Cochlear. Not only does this technology transcribe audio in real-time, but it also translates sign language next to the text. Similarly, a Netherlands based company also developed an app called GynoSys which is able to translate sign language into captions and even speech.
The abundance of video content uploaded to platforms like Youtube, Instagram and TikTok can leave many deaf viewers who rely on sign language feeling excluded. New AI technology aims to make media more accessible to the deaf community through the introduction of new avatars that are capable of sign language. There are approximately 70 million people who use sign language around the world and over 300 different sign languages. There is no universal sign language, as each sign language is based on the unique dialect and culture of the region it originates from. Sign language consists of more than just hand signs as well, also being composed of body movements and facial expressions to communicate emotion and turn a phrase into a question.
A UK-based startup company, Robotica, has created digital avatars to serve as sign language interpreters. CEO of Robotica, Adrian Pickering, saw a need for more sign language interpreters than are available and believes that with the amount of content being churned out, machine translation is the best way to bring equality of access to the deaf community. Robotica’s digital signers currently know British sign language (BSL), and are in the process of learning American and Italian sign language - with more to come. As sign language translation remains in the early stages, there are not any systems that allow users to directly translate from American sign language to BSL or other sign languages, or to translate sign language into any foreign language. However, researchers are currently hard at work to create systems capable of translating their individual regions’ sign language.
Voice assistants like Google’s Alexa and iPhone’s Siri can be found in most households and are an increasingly commonplace part of everyday life. However, voice assistants play a much larger role than simplifying daily tasks, and researchers are now using this technology to assist those with hearing loss. The same technology behind voice assistants has been modified to deliver a more specialized hearing experience. Leading hearing implant provider, Cochlear, has patented their own AI-based voice assistant called FOX. The technology behind FOX uses speech perception and patient outcome tests in its fitting optimization algorithm, in order to give patients an improved outcome and better tailor their cochlear implant to meet their needs.
Just as AI generates human sounding voice to respond to commands given to voice assistants, these same capabilities are used for text-to-speech functions. Text-to-speech features allow for improved accessibility for deaf individuals who are unable to voice for themselves or have difficulty being understood due to a deaf accent. Being able to utilize text-to-speech allows for easier navigation through automated systems and makes phone conversations accessible, or can facilitate communication in cases when a sign language interpreter may not be available. New applications like Voiceitt are utilizing AI to create software that can be trained to recognize, learn and predict speech for individuals with limited speech and unique speech patterns. This technology encompasses multiple functions of speech recognition and AI, as Voiceitt is able to support spoken communication by recognizing, saying and displaying phrases in real-time, in-person conversations. Artificial intelligence allows for a variety of applications of voice recognition used to facilitate communication and improve daily life for those who have accessibility needs.
Advancements in artificial intelligence play an important role in creating a more accessible and safer world for individuals who are deaf and hard of hearing. Applications of AI in developments in captioning and sign language translation provide greater equity of opportunity in both education and the workplace. As AI continues to learn, there is no shortage of its applications. Beyond facilitating conversations, AI is also allowing companies like Wavio to engineer sound recognition technology to allow deaf and hard of hearing individuals to “see” sounds around them. The sound recognition software can be implemented into smart devices to not only alert the user of the sound, but also correctly identify the noise. Such applications can be instrumental in helping to keep users safe, but also a benefit for deaf or hard of hearing parents to alert them if their children fall or are in distress. More applications of artificial intelligence continue to be engineered to allow those who are hard of hearing improved access to information and entertainment, helping to improve inclusivity and equal opportunity. As AI continues to grow in capabilities and becomes an increasingly integral part of daily life, the hope is that accessibility will also become more commonplace.
InnoCaption provides real-time captioning technology making phone calls easy and accessible for the deaf and hard of hearing community. Offered at no cost to individuals with hearing loss because we are certified by the FCC. InnoCaption is the only mobile app that offers real-time captioning of phone calls through live stenographers and automated speech recognition software. The choice is yours.
InnoCaption proporciona tecnología de subtitulado en tiempo real que hace que las llamadas telefónicas sean fáciles y accesibles para la comunidad de personas sordas y con problemas de audición. Se ofrece sin coste alguno para las personas con pérdida auditiva porque estamos certificados por la FCC. InnoCaption es la única aplicación móvil que ofrece subtitulación en tiempo real de llamadas telefónicas mediante taquígrafos en directo y software de reconocimiento automático del habla. Usted elige.
