Dynamic BdSL Digit Recognition using Hybrid 3DCNN+BiLSTM Model

Ishat Salsabil Silvia; Kamrul Hasan Talukder

doi:10.36948/ijfmr.2024.v06i03.20993

Dynamic BdSL Digit Recognition using Hybrid 3DCNN+BiLSTM Model

Author(s)	Ishat Salsabil Silvia, Kamrul Hasan Talukder
Country	Bangladesh
Abstract	Sign language is used as a way of communication by deaf and mute individuals. However, due to the limited number of people who understand sign language, integrating them into society is challenging. Approximately 6.9% of Bangladesh's population and 5% of the world’s population suffer from speech impediments. Individuals with this condition cannot hear what others are saying or communicate verbally, thus sign language must be relied upon. In recent years, sign language recognition has gained attention due to its necessity. However, a scarcity of publicly available dynamic gesture datasets for Bangladeshi Sign Language (BdSL) exists. Dynamic gestures, which contain both spatial and temporal information, are more useful in real-life applications. The classification of dynamic gestures requires more data than static gestures. In this research, 11 numeral Bengali digit signs are aimed to be classified. Data has been collected from the “SignBD-Word” dataset, which contains extracted RGB human body pose keypoints skeleton data. When compared to other dynamic sign language datasets and data requirements for training deep neural networks for dynamic gestures, the data per class in this dataset is found to be insufficient. A hybrid model architecture is proposed in this research to recognize dynamic gestures using lightweight 3DCNN and bidirectional LSTM layers for classifying gestures from the motion patterns of human body pose keypoints skeleton data after experimenting on various models. It has been observed that combining 3DCNN with the pre-trained DenseNet-201 and the BiLSTM model increases real-time accuracy by 4.54%. To the best of our knowledge, this is the first approach to combine 3DCNN with DenseNet-201 for action recognition. Also, one of the earliest investigations on dynamic BdSL hand gesture digit recognition using dynamic data. Additionally, different pre-trained models as base feature extractors have been evaluated.
Keywords	BdSL, Sign language, Deep learning, Transfer learning, Body joints skeleton, Action recognition
Field	Computer > Artificial Intelligence / Simulation / Virtual Reality
Published In	Volume 6, Issue 3, May-June 2024
Published On	2024-06-17
Cite This	Dynamic BdSL Digit Recognition using Hybrid 3DCNN+BiLSTM Model - Ishat Salsabil Silvia, Kamrul Hasan Talukder - IJFMR Volume 6, Issue 3, May-June 2024. DOI 10.36948/ijfmr.2024.v06i03.20993
DOI	https://doi.org/10.36948/ijfmr.2024.v06i03.20993
Short DOI	https://doi.org/gt2cb3

View / Download PDF File

E-ISSN 2582-2160

doi

CrossRef DOI is assigned to each research paper published in our journal.

IJFMR DOI prefix is
10.36948/ijfmr

Downloads

Research Paper Format Copyright Permission Form and Undertaking Form Cover Page Vol 6 Isu 6 Cover Page Vol 6 Isu 5 Cover Page Vol 6 Isu 4

All research papers published on this website are licensed under Creative Commons Attribution-ShareAlike 4.0 International License, and all rights belong to their respective authors/researchers.

CC-BY-SA

About IJFMR Fees & Payment Current Issue Publication Archive	Submit Research Paper Track Submission Status Publication Guidelines Publication Ethics Peer Review & Plagiarism	Join as a Reviewer Editors & Reviewers Reviewer Referral Program Get Reviewer Membership Certi.	Website/Journal Policies Usage Policy Content Policies Privacy Policy

Contact Us		+91-9898-5948-55	editor@ijfmr.com

International Journal For Multidisciplinary Research

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal

Dynamic BdSL Digit Recognition using Hybrid 3DCNN+BiLSTM Model

Share this