Research Experiences
Researcher
Systems and Software Lab (SSL),
Islamic University of Technology (IUT), Dhaka, Bangladesh
Apr'21 - Jul'23
My work in this lab is based on Sign Language Recognition (SLR). I have been working on Bangla Sign Language (BdSL) Recognition using depth information. Since my affiliation with this lab, I have published one paper on Bangla Sign Digits in an International Conference as the first-author and has one journal article under preparation. Recently, I, along with my thesis supervisor and co-supervisor have received a research grant for an ongoing project on Generating Bangla Sign Alphabets.
Research Grants
UIU Research Grant
Institute for Advanced Research (IAR),
United International University (UIU), Dhaka, Bangladesh
Title : Generating Bangla Sign Alphabet Dataset using Depth Information
Grant Reference : UIU-IAR-01-2022-SE-37
Awarded on : June 01, 2022
Received as : Co-Investigator
Grant Period : 12 Months (From 01 June, 2022 to 31 May, 2023)
Grant Amount : BDT 475,000.00
Journal Articles
[1] Rayeed, S. M., Sidratul Tamzida Tuba, Hasan Mahmud, Mumtahin Habib Ullah Mazumder Md, Saddam Hossain Mukta Md, and Kamrul Hasan Md. "BdSL47: A Complete Depth-based Bangla Sign Alphabet and Digit Dataset." Data in Brief (2023): 109799. doi: 10.1016/j.dib.2023.109799
Title : BdSL47: A complete depth-based Bangla sign alphabet and digit dataset
Authors : S M Rayeed, Sidratul Tamzida Tuba, Hasan Mahmud, Md. Mumtahin Habib Ullah Mazumder, Md. Saddam Hossain Mukta, Md. Kamrul Hasan
Publication Date : November 21, 2023
Published by : Elsevier
Status : Available Online
DOI : 10.1016/j.dib.2023.109799
Abstract :
Sign Language Recognition (SLR) is crucial for enabling communication between the deaf-mute and hearing communities. Nevertheless, the development of a comprehensive sign language dataset is a challenging task due to the complexity and variations in hand gestures. This challenge is particularly evident in the case of Bangla Sign Language (BdSL), where the limited availability of depth datasets impedes accurate recognition. To address this issue, we propose BdSL47, an open-access depth dataset for 47 one-handed static signs (10 digits, from ০ to ৯; and 37 letters, from অ to ँ) of BdSL. The dataset was created using the MediaPipe framework for extracting depth information. To classify the signs, we developed an Artificial Neural Network (ANN) model with a 63-node input layer, a 47-node output layer, and 4 hidden layers that included dropout in the last two hidden layers, an Adam optimizer, and a ReLU activation function. Based on the selected hyperparameters, the proposed ANN model effectively learns the spatial relationships and patterns from the depth-based gestural input features and gives an F1 score of 97.84 %, indicating the effectiveness of the approach compared to the baselines provided. The availability of BdSL47 as a comprehensive dataset can have an impact on improving the accuracy of SLR for BdSL using more advanced deep-learning models.
Conference Papers
[1] S M Rayeed, Gazi Wasif Akram, Sidratul Tamzida Tuba, Golam Sadman Zilani, Hasan Mahmud, Md. Kamrul Hasan, "Bangla sign digits recognition using depth information," Proc. SPIE 12084, Fourteenth International Conference on Machine Vision (ICMV 2021), 120840P (4 March 2022); doi: 10.1117/12.2623400
Title : Bangla sign digits recognition using depth information
Authors : S M Rayeed, Gazi Wasif Akram, Sidratul Tamzida Tuba, Golam Sadman Zilani, Hasan Mahmud, Md. Kamrul Hasan
Conference Name : Fourteenth International Conference on Machine Vision (ICMV 2021)
Conference Date : Nov 08 - 12, 2021
Publication Date : March 4, 2022
Conference Location : Rome, Italy (Virtually)
Published by : SPIE Digital Library
Status : Available Online
DOI : 10.1117/12.2623400
Abstract :
Sign Language Recognition (SLR) targets on interpreting the sign language into text or speech, so as to facilitate the communication between deaf-mute people and ordinary people. The task has broad social impact, but is still very challenging due to the complexity and large variations in hand actions. Existing dataset for Sign Language Recognition (SLR) in Bangla Sign Language (BdSL) is based on RGB images. Recent research on sign language recognition has shown better recognition accuracy using depth-based features. In this paper, we present a complete dataset for Bangla sign digits from Zero (Shunno in Bangla) to Nine (Noy in Bangla) using MediaPipe, a cross-platform depth-map estimation framework. The proposed method can utilize hand skeleton joint points containing depth information in addition to x, y coordinates from RGB images only. To validate the effectiveness of our proposed approach, we have run MediaPipe on a benchmark American Sign Language (ASL) dataset. Running different classifiers in our proposed dataset we got 98.65% using Support Vector Machine (SVM). Moreover, we compared our dataset with the existing Bangla digit dataset Ishara Bochon using deep learning based approach and achieved significantly higher accuracy.
Published Datasets
[1] S. M. Rayeed, (2022). BdSL47 : A complete dataset of sign alphabet and digits of Bangla Sign Language (BdSL) using depth information via MediaPipe (Version V1). Harvard Dataverse. doi: 10.7910/DVN/EPIC3H
Collaborators : S M Rayeed, Sidratul Tamzida Tuba, Gazi Wasif Akram, Raiyan Ahmed
Published by : Harvard Dataverse
Publication Date : Nov 08 - 12, 2021
Status : Available Online
DOI : 10.7910/DVN/EPIC3H
Description :
The dataset contains 47000 RGB input images of 47 signs (10 digits, 37 letters) of Bangla Sign Language. The images have been processed via MediaPipe framework, which is designed to detect predefined 21 hand key-points from a sample and provide normalized x & y coordinate values and an estimated depth value. The 3D coordinate values were stored in .csv files (1 file contains information of 100 image sample of the same sign). The dataset contains 470 .csv files in total, and 47000 corresponding output images with hand key-points being detected.