An Air-Written Real-Time Multilingual Numeral String Recognition System Using Deep Convolutional Neural Networks
- 1 Department of Computer Science, Dr. Vishwanath Karad MIT World Peace University, Pune, MH, India
- 2 Symbiosis Institute of Computer Studies and Research (SICSR), Symbiosis International (Deemed University), Pune, MH, India
Abstract
Air-writing is a modern practice for free-space writing of characters or words with hand or finger movements. In the new era, hand gestures are commonly used for Human-Computer Interaction (HCI) or controlling machines in several applications. Writing with a finger or holding any object in hand and 3-D movement in the air is helpful for several applications. However, it is hard to build simulated environments due to the complex structure of the hand and regulate the joints. Palm detection is another way to reduce the complexity of hand detection. Deep learning-based techniques enhance object detection tasks with excellent performance in this task. Convolutional Neural Networks (CNN) are the powerful frameworks used in detecting palms facing several challenges in this task. Therefore, we propose a novel approach using hand landmark detection to identify palms. Our proposed CNN-based air-writing recognition model is intended to detect and recognize numbers in 2 different languages. We present the CNN-based 2-D model to detect air-writing in real-time video. The proposed models were implemented on our developed datasets for two languages, Devanagari and English, with 99.55% accuracy. Our contribution is focused on the air-writing interface using a fingertip to write the numbers instead of traditional input devices. In addition, different air-writing gestures are introduced to control the writing activity. Therefore, our proposed model integrates detection, recognition, and control of air-writing activity. Hence, we achieved precise detection and management of palm movements and accurate recognition of air-written numbers. The results are promising and valuable for real-time multilingual numeral string recognition.
DOI: https://doi.org/10.3844/jcssp.2024.1712.1722
Copyright: © 2024 Meenal Jabde, Chandrashekhar Himmatrao Patil, Amol D. Vibhute and Shankar Mali. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
- 558 Views
- 282 Downloads
- 0 Citations
Download
Keywords
- Air-Writing
- Palm Detection
- Handwriting Recognition
- HCI
- CNN