CFP last date
30 June 2025
Call for Paper
July Edition
JAAI solicits high quality original research papers for the upcoming July edition of the journal. The last date of research paper submission is 30 June 2025

Submit your paper
Know more
The week's pick
Responsive image
Optimal Distributed Generator Sizing and Placement by Modified Particle Swarm Optimization (MPOS) Algorithm for Minimization of Distribution Power Loss

Anshu Sharma Rajesh Kumar Rai Ghizal Ansari
Reseach Article

Development of Computational Models for Automatic Indian Sign Language (ISL) Learning and Gesture Recognition

by Mehuli Mukherjee
journal cover thumbnail

Journal of Advanced Artificial Intelligence
Foundation of Computer Science (FCS), NY, USA
Volume 1 - Number 7
Year of Publication: 2025
Authors: Mehuli Mukherjee
10.5120/jaai202433

Mehuli Mukherjee . Development of Computational Models for Automatic Indian Sign Language (ISL) Learning and Gesture Recognition. Journal of Advanced Artificial Intelligence. 1, 7 ( May 2025), 12-17. DOI=10.5120/jaai202433

@article{ 10.5120/jaai202433,
author = { Mehuli Mukherjee },
title = { Development of Computational Models for Automatic Indian Sign Language (ISL) Learning and Gesture Recognition },
journal = { Journal of Advanced Artificial Intelligence },
issue_date = { May 2025 },
volume = { 1 },
number = { 7 },
month = { May },
year = { 2025 },
pages = { 12-17 },
numpages = {9},
url = { https://jaaionline.phdfocus.com/archives/volume1/number7/development-of-computational-models-for-automatic-indian-sign-language-isl-learning-and-gesture-recognition/ },
doi = { 10.5120/jaai202433 },
publisher = {Foundation of Computer Science (FCS), NY, USA},
address = {New York, USA}
}
%0 Journal Article
%1 2025-05-31T02:21:20+05:30
%A Mehuli Mukherjee
%T Development of Computational Models for Automatic Indian Sign Language (ISL) Learning and Gesture Recognition
%J Journal of Advanced Artificial Intelligence
%V 1
%N 7
%P 12-17
%D 2025
%I Foundation of Computer Science (FCS), NY, USA
Abstract

Indian Sign Language (ISL) is a crucial task in the areas of computer vision and pattern recognition, having its own grammar, syntax, vocabulary, and several unique linguistic attributes. It has wide applications in different aspects but the environment, background image resolution, modalities, and datasets affect the performance a lot. Over 466 million people are speech or hearing impaired, and 80% of them are semiilliterate or illiterate according to the WHO (World Health Organization) reports. So far discrete sign language videos are used to train the models. An enhancement opportunity has been proposed, where publicly available continuous videos can be considered and then using masked model algorithms to achieve a continuous conversation of speech/audio. Several deep neural networks methodologies are used which combines different methods for hand movement tracking, feature extraction, encoding and decoding and machine learning modeling. The current state of art is also leveraged for translating into different Indian languages. Finally, a comparative study is proposed to compare between widely used American Sign Languages.

References
  1. W. H. Organization. (2018) Deafness and hearing loss. [Online]. Available: https://www.who.int/news-room/ fact-sheets/detail/deafness and-hearing-loss.
  2. S. Aly and W. Aly, “Deeparslr: A novel signer-independent deep learning framework for isolated arabic sign language gestures recognition,” IEEE Access, vol. 8, pp. 83 199–83 212, 2020.
  3. R. Rastgoo, K. Kiani, and S. Escalera, “Hand sign language recognition using multi-view hand skeleton,” Expert Systems with Applications, vol. 150, p. 113336, 2020.
  4. K. Tripathi and N. B. G. Nandi, “Continuous Indian sign language gesture recognition and sentence formation,” Procedia Computer Science, vol. 54, pp. 523–531, 2015.
  5. M. Hassan, K. Assaleh, and T. Shanableh, “Multiple proposals for continuous arabic sign language recognition,” Sensing and Imaging, vol. 20, no. 1, p. 4, 2019.
  6. S. Albanie, G. Varol, L. Momeni, T. Afouras, J. S. Chung, N. Fox, and A. Zisserman, “Bsl-1k: Scaling up co-articulated sign language recognition using mouthing cues,” in European Conference on Computer Vision. Springer, 2020, pp. 35–53.
  7. O. Aran, I. Ari, L. Akarun, B. Sankur, A. Benoit, A. Caplier, P. Campr, A. H. Carrillo et al., “Signtutor:An interactive system for sign language tutoring,” IEEE MultiMedia, vol. 16, no. 1, pp. 81–93, 2009.
  8. F. S. Botros, A. Phinyomark, and E. J. Scheme, “Electromyography based gesture recognition: Is it time to change focus from the forearm to the wrist?” environments, vol. 14, p. 15, 2020.
  9. R. Fatmi, S. Rashad, and R. Integlia, “Comparing an, svm, and hmm based machine learning methods for American sign language recognition using wearable motion sensors,” in 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC). IEEE, 2019, pp. 0290–0297.
  10. S. Wei, X. Chen, X. Yang, S. Cao, and X. Zhang, “A component-based vocabulary-extensible sign language gesture recognition framework,” Sensors, vol. 16, no. 4, p. 556, 2016.
  11. S. Kim, J. Kim, S. Ahn, and Y. Kim, “Finger language recognition based on ensemble artificial neural network learning using armband emg sensors,” Technology and Health Care, vol. 26, no. S1, pp. 249–258, 2018.
  12. M. J. Cheok, Z. Omar, and M. H. Jaward, “A review of hand gesture and sign language recognition techniques,” International Journal of Machine Learning and Cybernetics, vol. 10, no. 1, pp. 131–153, 2019.
  13. S. Kim, Y. Ji, and K.-B. Lee, “An effective sign language learning with object detection based roi segmentation,” in 2018 Second IEEE International Conference on Robotic Computing (IRC). IEEE, 2018, pp. 330– 333.
  14. M. P. Paulraj, S. Yaacob, M. S. bin Zanar Azalan, and R. Palaniappan, “A phoneme based sign language recognition system using skin color segmentation,” in 2010 6th International Colloquium on Signal Processing & its Applications. IEEE, 2010, pp. 1–5.
  15. A. S. Ghotkar and G. K. Kharate, “Vision based real time hand gesture recognition techniques for human computer interaction,” Int J Comput Appl, vol. 70, no. 16, pp. 1–8, 2013.
  16. J.-H. Shin, J.-S. Lee, S.-K. Kil, D.-F. Shen, J.-G. Ryu, E.-H. Lee, H.-K. Min, and S.-H. Hong, “Hand region extraction and gesture recognition using entropy analysis,” IJCSNS International Journal of Computer Science and Network Security, vol. 6, no. 2A, pp. 216– 222, 2006.
  17. R. Akmeliawati, F. Dadgostar, S. Demidenko, N. Gamage, Y. C. Kuang, C. Messom, M. Ooi, A. Sarrafzadeh, and G. SenGupta, “Towards real-time sign language analysis via markerless gesture tracking,” in 2009 IEEE Instrumentation and Measurement Technology Conference. IEEE, 2009, pp. 1200–1204.
  18. X. Wang, M. Xia, H. Cai, Y. Gao, and C. Cattani, “Hidden-markovmodels-based dynamic hand gesture recognition,” Mathematical Problems in Engineering, vol. 2012, 2012.
  19. P. Li, T. Zhang, and A. E. Pece, “Visual contour tracking based on particle filters,” Image and Vision Computing, vol. 21, no. 1, pp. 111–123, 2003.
  20. M. Al-Hammadi, G. Muhammad, W. Abdul, M. Alsulaiman, M. A. Bencherif, and M. A. Mekhtiche, “Hand gesture recognition for sign language using 3dcnn,” IEEE Access, vol. 8, pp. 79 491–79 509, 2020.
  21. S. Khalid, T. Khalil, and S. Nasreen, “A survey of feature selection and feature extraction techniques in machine learning,” in 2014 science and information conference. IEEE, 2014, pp. 372–378.
  22. S. Jiang, B. Lv, W. Guo, C. Zhang, H. Wang, X. Sheng, and P. B. Shull, “Feasibility of wrist-worn, real-time hand, and surface gesture recognition via semg and imu sensing,” IEEE Transactions on Industrial Informatics, vol. 14, no. 8, pp. 3376–3385, 2018.
  23. Y. Yao and C.-T. Li, “Hand posture recognition using surf with adaptive boosting,” in British Machine Vision Conference, 2012.
  24. K. K. Gurjal P, “Real time hand gesture recognition using sift.” Int J Electron Electr Eng, vol. 2, no. 3, 2012.
  25. S. Z. Gurbuz, A. C. Gurbuz, E. A. Malaia, D. J. Griffin, C. Crawford, M. M. Rahman, E. Kurtoglu, R. Aksu, T. Macks, and R. Mdrafi, “American sign language recognition using rf sensing,” IEEE Sensors Journal, 2021.
  26. S. Jiang, B. Lv, W. Guo, C. Zhang, H. Wang, X. Sheng, and P. B. Shull, “Feasibility of wrist-worn, real-time hand, and surface gesture recognition via semg and imu sensing,” IEEE Transactions on Industrial Informatics, vol. 14, no. 8, pp. 3376–3385, 2018.
  27. M. Tariq, A. Iqbal, A. Zahid, Z. Iqbal, and J. Akhtar, “Sign language localization: Learning to eliminate language dialects,” in 2012 15th International Multitopic Conference (INMIC). IEEE, 2012, pp. 17–22.
  28. W. Ahmed, K. Chanda, and S. Mitra, “Vision based hand gesture recognition using dynamic time warping for indian sign language,” in 2016 International Conference on Information Science (ICIS). IEEE, 2016, pp. 120–125.
  29. N. Kumar, “Sign language recognition for hearing impaired people based on hands symbols classification,” in 2017 International Conference on Computing, Communication and Automation (ICCCA). IEEE, 2017, pp. 244–249.
  30. R. A. Kadhim and M. Khamees, “A real-time american sign language recognition system using convolutional neural network for real datasets,” TEM Journal, vol. 9, no. 3, p. 937, 2020.
  31. C. K. Lee, K. K. Ng, C.-H. Chen, H. C. Lau, S. Chung, and T. Tsoi, “American sign language recognition and training method with recurrent neural network,” Expert Systems with Applications, vol. 167, p. 114403, 2021.
  32. O. M. Sincan and H. Y. Keles, “Autsl: A large scale multi-modal turkish sign language dataset and baseline methods,” IEEE Access, vol. 8, pp. 181 340–181 355, 2020.
  33. G. Yuan, X. Liu, Q. Yan, S. Qiao, Z. Wang, and L. Yuan, “Hand gesture recognition using deep feature fusion network based on wearable sensors,” IEEE Sensors Journal, vol. 21, no. 1, pp. 539–547, 2020.
  34. Sridhar, Advaith, Rohith Gandhi Ganesan, Pratyush Kumar, and Mitesh Khapra. "Include: A large scale dataset for indian sign language recognition." In Proceedings of the 28th ACM international conference on multimedia, pp. 1366-1375. 2020.
  35. http://islrtc.nic.in/
  36. https://indiansignlanguage.org/
  37. Cao, Zhe, Tomas Simon, Shih-En Wei, and Yaser Sheikh. "Realtime multi-person 2d pose estimation using part affinity fields." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 72917299. 2017.
  38. Chen, Tianqi, and Carlos Guestrin. "Xgboost: A scalable tree boosting system." In Proceedings of the 22nd acmsigkdd international conference on knowledge discovery and data mining, pp. 785-794. 2016.
  39. Meyer, David, Friedrich Leisch, and Kurt Hornik. "The support vector machine under test." Neurocomputing 55, no. 1-2 (2003): 169-186.
  40. Pal, Mahesh. "Random forest classifier for remote sensing classification." International journal of remote sensing 26, no. 1 (2005): 217222.
  41. Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. "Bert: Pre-training of deep bidirectional transformers for language understanding." arXiv preprint arXiv:1810.04805(2018).
Index Terms

Computer Science
Information Sciences

Keywords

Indian Sign Language Deep Neural Networks Machine Learning

Powered by PhDFocusTM