Eksplorasi Model Hybrid Transformer-Latent Semantic Analysis (LSA) Untuk Pemahaman Konteks Teks Berita Berbahasa Indonesia
DOI:
https://doi.org/10.52436/1.jpti.662Kata Kunci:
Latent Semantic Analysis, Model Hybrid, Pemrosesan Bahasa Alami, Teks Berita, TransformerAbstrak
Kemajuan teknologi informasi meningkatkan konsumsi berita digital, menuntut sistem Natural Language Processing (NLP) yang efisien dalam memahami bahasa Indonesia. Namun, kompleksitas morfologi bahasa Indonesia menyulitkan model NLP konvensional dalam menangkap makna semantik secara akurat. Model deep learning seperti Transformer unggul dalam menangkap hubungan semantik lokal, sementara Latent Semantic Analysis (LSA) memahami hubungan semantik global melalui reduksi dimensi. Namun, Transformer membutuhkan sumber daya komputasi besar, sedangkan LSA cenderung kehilangan konteks sintaksis. Penelitian ini mengusulkan model hybrid yang mengintegrasikan Transformer dan LSA untuk meningkatkan pemahaman teks berita Indonesia serta mengevaluasi performanya dibandingkan model individu dan deep learning yang lebih kompleks. Evaluasi menggunakan Accuracy, F1-Score, BLEU Score, ROUGE, dan Perplexity. Model hybrid mencapai akurasi 0.510760 dan F1-Score 0.520486, lebih baik dari LSA dan Transformer, tetapi masih tertinggal dari BERT dan GPT. Meski demikian, model hybrid lebih efisien secara komputasi dibandingkan model deep learning yang lebih kompleks. Penelitian ini berkontribusi pada pengembangan NLP bahasa Indonesia dengan pendekatan yang lebih ringan. Implikasi penelitian menunjukkan perlunya dataset lebih besar dan teknik embedding lebih maju. Penelitian selanjutnya dapat mengeksplorasi integrasi model hybrid dengan BERT atau GPT, serta teknik embedding lain seperti word2vec atau fastText untuk meningkatkan pemahaman semantik.
Unduhan
Referensi
C. Primasiwi, M. I. Irawan, and R. Ambarwati, “Key Performance Indicators for Influencer Marketing on Instagram:,” presented at the 2nd International Conference on Business and Management of Technology (ICONBMT 2020), Surabaya, Indonesia, 2021. doi: 10.2991/aebmr.k.210510.027.
T. Guarda, J. Balseca, K. García, J. González, F. Yagual, and H. Castillo-Beltran, “Digital Transformation Trends and Innovation,” IOP Conf. Ser. Mater. Sci. Eng., vol. 1099, no. 1, p. 012062, Mar. 2021, doi: 10.1088/1757-899X/1099/1/012062.
A. Dewandaru, D. H. Widyantoro, and S. Akbar, “Event Geoparser with Pseudo-Location Entity Identification and Numerical Extraction in Indonesian News Corpus,” Aug. 14, 2020, MATHEMATICS & COMPUTER SCIENCE. doi: 10.20944/preprints202008.0263.v1.
A. F. Aji et al., “One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Dublin, Ireland: Association for Computational Linguistics, 2022, pp. 7226–7249. doi: 10.18653/v1/2022.acl-long.500.
M. Hahn, “Theoretical Limitations of Self-Attention in Neural Sequence Models,” Trans. Assoc. Comput. Linguist., vol. 8, pp. 156–171, Dec. 2020, doi: 10.1162/tacl_a_00306.
T. Wolf et al., “Transformers: State-of-the-Art Natural Language Processing,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Online: Association for Computational Linguistics, 2020, pp. 38–45. doi: 10.18653/v1/2020.emnlp-demos.6.
D. F. O. Onah, E. L. L. Pang, and M. El-Haj, “A Data-driven Latent Semantic Analysis for Automatic Text Summarization using LDA Topic Modelling,” in 2022 IEEE International Conference on Big Data (Big Data), Osaka, Japan: IEEE, Dec. 2022, pp. 2771–2780. doi: 10.1109/BigData55660.2022.10020259.
D. Patterson et al., “Carbon Emissions and Large Neural Network Training,” 2021, arXiv. doi: 10.48550/ARXIV.2104.10350.
Y. Li and A. Risteski, “The Limitations of Limited Context for Constituency Parsing,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online: Association for Computational Linguistics, 2021, pp. 2675–2687. doi: 10.18653/v1/2021.acl-long.208.
C. W. Schmidt et al., “Tokenization Is More Than Compression,” Oct. 07, 2024, arXiv: arXiv:2402.18376. doi: 10.48550/arXiv.2402.18376.
M. Amien, “Sejarah dan Perkembangan Teknik Natural Language Processing (NLP) Bahasa Indonesia: Tinjauan tentang sejarah, perkembangan teknologi, dan aplikasi NLP dalam bahasa Indonesia,” Mar. 28, 2023, arXiv: arXiv:2304.02746. doi: 10.48550/arXiv.2304.02746.
R. Rianto, A. B. Mutiara, E. P. Wibowo, and P. I. Santosa, “Improving the Accuracy of Text Classi?cation using Stemming Method, A Case of Non-formal Indonesian Conversation”.
H. T. Y. Achsan, H. Suhartanto, W. C. Wibowo, D. A. Dewi, and K. Ismed, “Automatic Extraction of Indonesian Stopwords,” Int. J. Adv. Comput. Sci. Appl., vol. 14, no. 2, 2023, doi: 10.14569/IJACSA.2023.0140221.
G. Garrido-Bañuelos, Mpho Mafata, and A. Buica, “Exploring the use of Latent Semantic Analysis (LSA) to investigate wine sensory profiles,” 2024, doi: 10.13140/RG.2.2.32030.96328.
T. Xiao and J. Zhu, “Introduction to Transformers: an NLP Perspective,” Nov. 29, 2023, arXiv: arXiv:2311.17633. doi: 10.48550/arXiv.2311.17633.
T. Q. Nguyen, K. Murray, and D. Chiang, “Data Augmentation by Concatenation for Low-Resource Translation: A Mystery and a Solution,” Jul. 02, 2021, arXiv: arXiv:2105.01691. doi: 10.48550/arXiv.2105.01691.
A. Galassi, M. Lippi, and P. Torroni, “Attention in Natural Language Processing,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 10, pp. 4291–4308, Oct. 2021, doi: 10.1109/TNNLS.2020.3019893.
M. Pagliardini, A. Mohtashami, F. Fleuret, and M. Jaggi, “DenseFormer: Enhancing Information Flow in Transformers via Depth Weighted Averaging,” Mar. 21, 2024, arXiv: arXiv:2402.02622. doi: 10.48550/arXiv.2402.02622.
K. M. Kahloot and P. Ekler, “Algorithmic Splitting: A Method for Dataset Preparation,” IEEE Access, vol. 9, pp. 125229–125237, 2021, doi: 10.1109/ACCESS.2021.3110745.
Ž. Ð. Vujovic, “Classification Model Evaluation Metrics,” Int. J. Adv. Comput. Sci. Appl., vol. 12, no. 6, 2021, doi: 10.14569/IJACSA.2021.0120670.
“CLINICAL DETERIORATION PREDICTION IN BRAZILIAN HOSPITALS BASED ON ARTIFICIAL NEURAL NETWORKS AND TREE DECISION MODELS,” in Proceedings of the 15th International Conference on ICT, Society and Human Beings (ICT 2022), the 19th International Conference Web Based Communities and Social Media (WBCSM 2022) and 14th International Conference on e-Health (EH 2022), IADIS Press, Jul. 2022. doi: 10.33965/ICT_WBC_EH2022_202204L024.
J. Wieting, T. Berg-Kirkpatrick, K. Gimpel, and G. Neubig, “Beyond BLEU: Training Neural Machine Translation with Semantic Similarity,” Sep. 14, 2019, arXiv: arXiv:1909.06694. doi: 10.48550/arXiv.1909.06694.
A. Bharadwaj, A. Srinivasan, A. Kasi, and B. Das, “Extending The Performance of Extractive Text Summarization By Ensemble Techniques,” in 2019 11th International Conference on Advanced Computing (ICoAC), Chennai, India: IEEE, Dec. 2019, pp. 282–288. doi: 10.1109/ICoAC48765.2019.246854.
J. Roh, S.-H. Oh, and S.-Y. Lee, “Unigram-Normalized Perplexity as a Language Model Performance Measure with Different Vocabulary Sizes,” Nov. 26, 2020, arXiv: arXiv:2011.13220. doi: 10.48550/arXiv.2011.13220.
