Pengenalan Pola Huruf Hijaiyyah dengan Metode CNN untuk Bahasa Isyarat Arab
DOI:
https://doi.org/10.52436/1.jpti.1172Kata Kunci:
Bahasa Isyarat Arab, Convolutional Neural Network, Hijaiyah, Transfer Learning, TunarunguAbstrak
Bahasa Isyarat Arab (Arabic Sign Language/ArSL) merupakan sarana komunikasi utama bagi penyandang tunarungu, termasuk dalam pembelajaran Al-Qur’an. Namun, keterbatasan teknologi dalam mengenali bahasa isyarat secara otomatis menjadi hambatan serius terhadap akses pendidikan agama yang inklusif. Penelitian ini bertujuan untuk mengenali pola huruf hijaiyyah dalam ArSL dengan memanfaatkan metode Convolutional Neural Network (CNN) melalui pendekatan transfer learning dan fine-tuning pada empat arsitektur pralatih, yaitu MobileNetV2, EfficientNetB0, VGG16, dan ResNet50. Dataset yang digunakan terdiri dari 7.856 citra RGB tangan yang mewakili 31 huruf hijaiyyah, yang dibagi menjadi data pelatihan, validasi, serta pengujian. Evaluasi dilakukan menggunakan metrik accuracy, precision, recall, F1-score, serta efisiensi komputasi berdasarkan ukuran model dan waktu inferensi. Hasil penelitian memperlihatkan bahwa ResNet50 memperoleh akurasi tertinggi sebesar 98,35%, diikuti MobileNetV2 (97,84%), EfficientNetB0 (97,71%), dan VGG16 (97,07%). Meskipun demikian, MobileNetV2 memiliki ukuran model terkecil dan kecepatan inferensi tercepat, sehingga paling sesuai untuk implementasi pada perangkat dengan keterbatasan sumber daya. Analisis confusion matrix juga menunjukkan kesalahan klasifikasi terutama pada huruf yang memiliki kemiripan visual, seperti dal–dzal dan ta–tha. Penelitian ini menegaskan efektivitas CNN berbasis transfer learning dalam pengenalan huruf hijaiyyah bahasa isyarat Arab serta memberikan kontribusi nyata terhadap pengembangan sistem pembelajaran agama yang lebih inklusif bagi penyandang tunarungu.
Unduhan
Referensi
B. Pamungkas and H. Hermanto, “Tahapan Belajar Al Qur’an Menggunakan Huruf Hijaiyah Isyarat bagi Anak dengan Hambatan Pendengaran,” Jurnal Pendidikan Kebutuhan Khusus, vol. 6, no. 1, pp. 34–41, Apr. 2022, doi: 10.24036/jpkk.v6i1.621.
H. A. AbdElghfar et al., “QSLRS-CNN: Qur’anic sign language recognition system based on convolutional neural networks,” The Imaging Science Journal, vol. 72, no. 2, pp. 254–266, Feb. 2024, doi: 10.1080/13682199.2023.2202576.
A. Setiawan, “Intelligent Systems For Disabilities Technology For A More Inclusive Life,” 2025. doi: 10.2139/ssrn.5062273.
N. M. Alharthi and S. M. Alzahrani, “Vision Transformers and Transfer learning Approaches for Arabic Sign Language Recognition,” Applied Sciences, vol. 13, no. 21, p. 11625, Oct. 2023, doi: 10.3390/app132111625.
J. Sánchez, J. Reyes-Rojas, and J. Alé-Silva, “What Is Known about Assistive Technologies in Distance and Digital Education for Learners with Disabilities?” Education Sciences, vol. 14, no. 6, p. 595, May 2024, doi: 10.3390/educsci14060595.
S. A. Alhulays, “The Role Of Assistive Technology in Supporting Communication and Academic Access For Deaf Students: A Qualitative Study At Gallaudet University,” Baileo: Jurnal Sosial Humaniora, vol. 2, no. 1, pp. 97–106, Sep. 2024, doi: 10.30598/baileofisipvol2iss1pp97-106.
S. Al Ahmadi, F. Muhammad, and H. Al Dawsari, “Enhancing Arabic Sign Language Interpretation: Leveraging Convolutional Neural Networks and Transfer learning,” Mathematics, vol. 12, no. 6, p. 823, Mar. 2024, doi: 10.3390/math12060823.
R. El Kharoua and X. Jiang, “Deep learning Recognition for Arabic Alphabet Sign Language RGB Dataset,” Journal of Computer and Communications, vol. 12, no. 03, pp. 32–51, 2024, doi: 10.4236/jcc.2024.123003.
I. D. Mienye, T. G. Swart, G. Obaido, M. Jordan, and P. Ilono, “Deep Convolutional Neural Networks in Medical Image Analysis: A Review,” Information, vol. 16, no. 3, p. 195, Mar. 2025, doi: 10.3390/info16030195.
Z. S. Bilal, A. Gargouri, H. F. Mahmood, and H. Mnif, “Comparison of Collective Diverse Arabic Sign Language Dataset,” Journal of Wireless Mobile Networks, Ubiquitous Computing, and Dependable Applications, vol. 15, no. 4, pp. 133–150, Dec. 2024, doi: 10.58346/JOWUA.2024.I4.009.
A. Hussain, S. Ul Amin, M. Fayaz, and S. Seo, “An Efficient and Robust Hand Gesture Recognition System of Sign Language Employing Finetuned Inception-V3 and Efficientnet-B0 Network,” Computer Systems Science and Engineering, vol. 46, no. 3, pp. 3509–3525, 2023, doi: 10.32604/csse.2023.037258.
P. Poladiya, D. Suresh, P. Gulhane, M. A. Ajmal, and P. Kosamkar, “Sign Language Detection Using Deep learning,” in Proc. 3rd Int. Conf. for Innovation in Technology (INOCON), IEEE, Mar. 2024, pp. 1–6, doi: 10.1109/INOCON60754.2024.10512307.
B. Alsharif, A. S. Altaher, A. Altaher, M. Ilyas, and E. Alalwany, “Deep learning Technology to Recognize American Sign Language Alphabet,” Sensors, vol. 23, no. 18, p. 7970, Sep. 2023, doi: 10.3390/s23187970.
M. Balat, R. Awaad, H. Adel, A. B. Zaky, and S. A. Aly, “Advanced Arabic Alphabet Sign Language Recognition Using Transfer learning and Transformer Models,” arXiv preprint, Oct. 2024. [Online]. Available: https://arxiv.org/abs/2410.00681
M. H. Ismail, S. A. Dawwd, and F. H. Ali, “Static hand gesture recognition of Arabic sign language by using deep CNNs,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 24, no. 1, pp. 178–188, Oct. 2021, doi: 10.11591/ijeecs.v24.i1.pp178-188.
Y. Brianorman and R. Munir, “Perbandingan Pre-Trained CNN: Klasifikasi Pengenalan Bahasa Isyarat Huruf Hijaiyah,” Jurnal Sistem Informasi Bisnis, vol. 13, no. 1, pp. 52–59, Jul. 2023, doi: 10.21456/vol13iss1pp52-59.
U. Fadlilah, A. K. Mahamad, and B. Handaga, “The Development of Android for Indonesian Sign Language Using Tensorflow Lite and CNN: An Initial Study,” Journal of Physics: Conference Series, vol. 1858, no. 1, p. 012085, Apr. 2021, doi: 10.1088/1742-6596/1858/1/012085.
M. F. Nurnoby, E.-S. M. El-Alfy, and H. Luqman, “Evaluation of CNN Models with Transfer learning for Recognition of Sign Language Alphabets with Complex Background,” in Proc. Int. Conf. on Innovation and Intelligence for Informatics, Computing and Technologies (3ICT), IEEE, Dec. 2020, pp. 1–6. doi: 10.1109/3ICT51146.2020.9311989.
R. Temkar, S. Jagtap, K. Jadhav, and M. Deshmukh, “Real-time Sign Language Recognition Using MobileNetV2 and Transfer learning.”
R. Poojary, R. Raina, and A. Kumar Mondal, “Effect of data-augmentation on fine-tuned CNN model performance,” IAES International Journal of Artificial Intelligence (IJ-AI), vol. 10, no. 1, p. 84, Mar. 2021, doi: 10.11591/ijai.v10.i1.pp84-92.
H. Imaduddin, F. Yusfida Ala, A. Fatmawati, and B. A. Hermansyah, “Comparison of transfer learning method for COVID-19 detection using convolution neural network,” Bulletin of Electrical Engineering and Informatics, vol. 11, no. 2, pp. 1091–1099, Apr. 2022, doi: 10.11591/eei.v11i2.3525.
M. R. Sadik et al., “Computer Vision Based Bangla Sign Language Recognition Using Transfer learning,” in Proc. 2nd Int. Conf. on Data Science and Information System (ICDSIS), IEEE, May 2024, pp. 1–7. doi: 10.1109/ICDSIS61070.2024.10594269.
M. A. A. Mosleh, A. Assiri, A. H. Gumaei, B. F. Alkhamees, and M. Al-Qahtani, “A Bidirectional Arabic Sign Language Framework Using Deep learning and Fuzzy Matching Score,” Mathematics, vol. 12, no. 8, p. 1155, Apr. 2024, doi: 10.3390/math12081155.
I. Goodfellow, Y. Bengio, and A. Courville, Deep learning. Cambridge, MA: MIT Press, 2016.
F. Chollet, Deep learning with Python, 2nd ed. Shelter Island, NY: Manning Publications, 2021.
E. Stevens, L. Antiga, and T. Viehmann, Deep learning with PyTorch. Shelter Island, NY: Manning Publications, 2020.
M. Al-Barham, A. Alsharkawi, M. Al-Yaman, M. Al-Fetyani, A. Elnagar, A. A. SaAleek, and M. Al-Odat, “RGB Arabic Alphabets Sign Language Dataset,” arXiv preprint arXiv:2301.11932, Jan. 2023, doi: 10.48550/arXiv.2301.11932.
N. Trung Tuyen, N. Thanh Dang, D. Hon Minh, and T. Q. Nguyen, “Enhancing human activity recognition model performance through KMeans-based stratified data splitting,” Measurement and Control, Jan. 2025, doi: 10.1177/00202940241312873.
H. Imaduddin, I. C. Utomo, and D. A. Anggoro, “Fine-tuning ResNet-50 for the classification of visual impairments from retinal fundus images,” International Journal of Electrical and Computer Engineering (IJECE), vol. 14, no. 4, p. 4175, Aug. 2024, doi: 10.11591/ijece.v14i4.pp4175-4182.
D. Grießhaber, J. Maucher, and N. T. Vu, “Fine-Tuning BERT for Low-Resource Natural Language Understanding via Active Learning,” in Proc. 28th Int. Conf. on Computational Linguistics, Stroudsburg, PA: ICCL, 2020, pp. 1158–1171, doi: 10.18653/v1/2020.coling-main.100.
W. N. Ismail, H. A. Alsalamah, and E. A. Mohamed, “Genetic-Efficient Fine-Tuning with Layer Pruning on Multimodal Covid-19 Medical Imaging,” Neural Computing and Applications, vol. 36, no. 6, pp. 3215–3237, Feb. 2024, doi: 10.1007/s00521-023-09194-5.
A. Davila, J. Colan, and Y. Hasegawa, “Comparison of Fine-Tuning Strategies for Transfer Learning in Medical Image Classification,” Image and Vision Computing, vol. 146, p. 105012, Jun. 2024, doi: 10.1016/j.imavis.2024.105012.
