Harnessing Deep Learning to Arabic Sign Language Recognition

Communication is a crucial aspect of human life and society. It enables us to connect, share ideas, and foster mutual understanding. However, there are individuals who are unable to communicate through spoken words, such as those who are deaf and mute and rely on sign language for communication, especially in education. Unfortunately, sign language is not widely understood, especially in Arabic-speaking countries. Especially, in the emerging cross cultural environment in the Middle eastern countries, this is becoming even challenging for the said individuals. Additionally, poor lighting conditions, the inherent environment noise, and hand obstructions make the detection process even challenging. There are limited studies in literature emphasizing the said issues with cross cultural focus and in a real time situation. To address this challenge, this initiative aims to use computer vision and deep learning to bridge the communication gap and facilitate mutual comprehension with a sustainable solution. The primary goal of this research is to serve Arabic-speaking regions by breaking down communication barriers and enabling seamless communication for the deaf community. After conducting a thorough literature review, the study has shortlisted and investigated CNN and YOLOv8 on the Arabic sign language dataset, as they show promise in solving similar problems. The CNN model achieved an accuracy of 98.88%, whereas the YOLOv8 model achieved precision and recall values of 97%, respectively. These results are quite acceptable and enhanced compared to similar studies in the literature. Moreover, a mobile application has been developed for sign language recognition on the go. Thus, the study is a potential contribution to assist such differently abled people in the Arabic regions.