Dataset Information

Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model.

ABSTRACT: Every one of us has a unique manner of communicating to explore the world, and such communication helps to interpret life. Sign language is the popular language of communication for hearing and speech-disabled people. When a sign language user interacts with a non-sign language user, it becomes difficult for a signer to express themselves to another person. A sign language recognition system can help a signer to interpret the sign of a non-sign language user. This study presents a sign language recognition system that is capable of recognizing Arabic Sign Language from recorded RGB videos. To achieve this, two datasets were considered, such as (1) the raw dataset and (2) the face-hand region-based segmented dataset produced from the raw dataset. Moreover, operational layer-based multi-layer perceptron "SelfMLP" is proposed in this study to build CNN-LSTM-SelfMLP models for Arabic Sign Language recognition. MobileNetV2 and ResNet18-based CNN backbones and three SelfMLPs were used to construct six different models of CNN-LSTM-SelfMLP architecture for performance comparison of Arabic Sign Language recognition. This study examined the signer-independent mode to deal with real-time application circumstances. As a result, MobileNetV2-LSTM-SelfMLP on the segmented dataset achieved the best accuracy of 87.69% with 88.57% precision, 87.69% recall, 87.72% F1 score, and 99.75% specificity. Overall, face-hand region-based segmentation and SelfMLP-infused MobileNetV2-LSTM-SelfMLP surpassed the previous findings on Arabic Sign Language recognition by 10.970% accuracy.

SUBMITTER: Podder KK

PROVIDER: S-EPMC10459624 | biostudies-literature | 2023 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model.

Podder Kanchon Kanti KK Ezeddin Maymouna M Chowdhury Muhammad E H MEH Sumon Md Shaheenur Islam MSI Tahir Anas M AM Ayari Mohamed Arselene MA Dutta Proma P Khandakar Amith A Mahbub Zaid Bin ZB Kadir Muhammad Abdul MA

Sensors (Basel, Switzerland) 20230814 16

Every one of us has a unique manner of communicating to explore the world, and such communication helps to interpret life. Sign language is the popular language of communication for hearing and speech-disabled people. When a sign language user interacts with a non-sign language user, it becomes difficult for a signer to express themselves to another person. A sign language recognition system can help a signer to interpret the sign of a non-sign language user. This study presents a sign language ...[more]

PMID: 37631693

Similar Datasets

Project description:BackgroundIndividuals with hearing impairments may face hindrances in health care assistance, which may significantly impact the prognosis and the incidence of complications and iatrogenic events. Therefore, the development of automatic communication systems to assist the interaction between this population and health care workers is paramount.ObjectiveThis study aims to systematically review the evidence on communication systems using human-computer interaction techniques developed for deaf people who communicate through sign language that are already in use or proposed for use in health care contexts and have been tested with human users or videos of human users.MethodsA systematic review will be performed based on a literature search in MEDLINE, Web of Science, ACM, and IEEE Xplore as well as top-tiered conferences in the area to identify relevant studies. The inclusion criteria are the description of the development of a sign language recognition system in a health care context and the testing with human users. Independent investigators (LFRO, LRV, and LMMSR) will screen eligible studies, and disagreements will be solved by a senior researcher (MSM). The included papers will undergo full-text screening. A PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) flow diagram will be presented to visually summarize the screening process, ensuring clarity and transparency in presenting the results. Additionally, a comprehensive chart table will be constructed to consolidate essential data related to the key variables extracted from the studies. These results will be meticulously analyzed and presented descriptively, offering insightful interpretations of the information encapsulated within the table.ResultsA preliminary search was performed in April 2024. Researchers concluded the study selection by July 2024. Data extraction, synthesis, report, and recommendations are expected to be finished by February 2025.ConclusionsThis systematic review will identify human-machine systems that enable communication in health services involving deaf patients, presenting the framework that includes usability and application in human contexts. We will present a comprehensive panel of findings, highlighting systems used to tackle communication barriers and offer a narrative comparison of current implementation practices.International registered report identifier (irrid)PRR1-10.2196/55427.

Dataset Information

Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model.

Publications

Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets