Challenge

Giving Voice to The Voiceless: our journey of creating an AI for Tunisian Sign Language Transcription 

Our Challenge

Deafness is widespread around the world. According to the World Health Organization (WHO), there are currently more than 430 million people living with disabling hearing loss. It is also expected that by 2050, this number could be over 700 million. In Tunisia, there are over 60,000 deaf people according to the African Sign Languages Resource Center. The impacts of deafness are broad and can be profound. The main issue we are dealing with is essentially the lack of communication tools and the inability to deliver information to deaf people in Tunisia.

This inconvenience has proven to be disastrous as it severely impacted the education rate for them. Nearly 95% of deaf people are uneducated or quit school at the elementary level. Besides, they often find themselves discriminated against simply for their difference. The majority of people barely recognize sign language and won’t be able to understand it without the presence of an expert. Imagine being easily misunderstood, unable to communicate with other people and yet no one makes an effort to help you out. Deaf people have found themselves to be facing a disadvantage in the Tunisian society and that is why WE, a team of 6 students at ESPRIT, have decided to help this community and create an easier, more fluid communication tool for them.

Our Solution

Since deaf people do not have that many options for communicating with a hearing person, and all of the alternatives do have major flaws, interpreters aren’t usually available, and also could be expensive, our solution is to create an easy-to-use innovative mobile application that works by placing a smartphone in front of the user while the app translates gestures or sign language into text and speech.it can translate the Tunisian sign language as quick as the person speaks. we will be using neural networks and computer vision to recognize the video of Tunisian sign language speaker, and then smart algorithms translate it into text or speech.

Why Our Project Matters

The first SDG that we are striving for is excellent education by providing an inclusive educational tool for the 95% illiterate deaf people who were unable to participate in this trip due to very poor communication options. Thanks to our mobile app, teachers will be able to easily understand and interact with their deaf students without struggling with the translation of sign language anymore. The second one is reduced inequality because as long as our systems aren’t designed to be used by individuals who speak TSL, this class’s life in our nation is a nightmare. Even the most basic activities for any given individual will be impossible for any deaf person to fulfill. That’s why in this application we are focusing on guaranteeing equality among all citizens. Finally, the third SDG aims to eliminate poverty. As long as deaf individuals have access to education, their integration into the labor force will be considerably simpler, allowing them to cover the costs of at least their basic needs and lifting them out of poverty.  

 

Building the Dataset

As a first step, we have decided to limit ourselves to the context of a medical consultation. We’ve worked with a general practitioner that described the medical consultation. And using that description, we’ve extracted a list of words that we will use for our dictionary.

We’ve then sent the words to Tunisian Sign Language interpreters working with ATILS (Association Tunisienne des Interprètes en Langues des Signes) whom we’ve reached through the president. These interpreters sent us back the sign for each term we’ve picked.

In addition, we’ve used the “Medical Dictionary in Tunisian Sign Languages” which was made by AVST (Association Voix du Sourd Tunisienne) in cooperation with the French Institute. This booklet contains 160 words used in the context of a medical consultation with their respective signs. Through the help of our doctor, we’ve sampled 60 relevant words as a starting size for our vocabulary (we will be going back to add the rest of the words once we have a minimal viable product: a working classifier model with a certain accuracy threshold to be determined).

This is the repo of our project: https://github.com/Tunisian-Sign-Language-Transcription/building-dataset.git