IMPLEMENTASI ALGORITMA CONVOLUTIONAL NEURAL NETWORK (CNN) PADA EVALUASI STRATEGI PROMPT ENGINEERING TERHADAP AKURASI VISUAL QUESTION ANSWERING (VQA)

Authors

  • Muhammad Riyan Universitas Islam Sumatera Utara, Medan, Indonesia Author
  • Khairuddin Nasution Universitas Islam Sumatera Utara, Medan, Indonesia Author
  • Darjat Saripurna Universitas Islam Sumatera Utara, Medan, Indonesia Author

DOI:

https://doi.org/10.71282/jurmie.v3i11.2726

Keywords:

Visual Question Answering, Convolutional Neural Network, Transformer, Prompt Engineering, VQA Accuracy.

Abstract

Visual Question Answering (VQA) is the task of answering natural language questions about the content of an image. This study aims to identify the prompt engineering strategy with the highest VQA accuracy and to test whether the accuracy differs significantly among strategies. The model is a CNN–Transformer architecture with ResNet-50 as the visual encoder, BERT as the text encoder, and a cross-attention fusion layer. Five strategies were evaluated: zero-shot, few-shot, chain-of-thought, instruction-based, and role-prompting. The data consist of 8,000 questions from the VQA v2.0 validation set selected by stratified random sampling. Each strategy was trained with five random seeds and evaluated using VQA Accuracy and One-Way ANOVA. The results show that the instruction-based strategy achieved the highest mean accuracy of 37.54%, while the other strategies ranged from 36.63% to 37.17%. However, the differences were not statistically significant (p = 0.2897). The instruction-based strategy was then implemented in a web application prototype.

Downloads

Download data is not yet available.

References

Amelia, W. (2023). Implementasi Jaringan Syaraf Tiruan Sebagai Sistem Diagnosis Stunting Balita Menggunakan Metode Backpropagation Di Puskesmas Brebes. Jurnal Minfo Polgan, 12(2), 1874–1883. https://doi.org/10.33395/jmp.v12i2.13044

Arwansyah, A., Sy, H., & Susanto, C. (2026). PROSIDING SEMINAR ILMIAH SISTEM INFORMASI DAN TEKNOLOGI INFORMASI Pusat Penelitian dan Pengabdian pada Masyarakat (P3M) Universitas Dipa Makassar Implementasi Mixture of Experts (MoE) Berbasis CNN dan Vision Transformer untuk Klasifikasi Citra. 15(2), 95–104. https://doi.org/10.36774/sisiti.v15i2.2524

Cibro, E., & Yahfizham. (2024). Mengidentifikasi Dasar Algoritma Pemrograman. Jurnal Ilmiah Sistem Informasi Dan Ilmu Komputer, 4(1), 194–206. https://doi.org/10.55606/juisik.v4i1.763

Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., & Parikh, D. (2017). Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering. (CVPR), 6904–6913. http://arxiv.org/abs/1612.00837

Grace, S. J., & Gunawan, D. (2025). PERBANDINGAN CNN, RESNET50, DAN VISION TRANSFORMER UNTUK KLASIFIKASI KANKER PAYUDARA BERBASIS WEB. Rabit : Jurnal Teknologi Dan Sistem Informasi Univrab, 10(2), 945–956. https://doi.org/10.36341/rabit.v10i2.6420

Hanifah, A., Wakhidatus Sholikah, R., & Hari Ginardi, R. (2024). Visual Question Answering Bahasa Indonesia Berbasis Deep Learning untuk Pembelajaran Visual Anak TK Indonesian Visual Question Answering Based on Deep Learning for Kindergarten’s Visual Learning. In Februari (Vol. 23, Number 1).

Haromain, I., Munir, S., & Rahmah, A. (2025). Analisa Prompt Engineering pada Large Language Model dengan Retrieval-Augmented Generation untuk Informasi Obat dan Vitamin. Indonesian Journal Computer Science, 4(2), 144–153. https://jurnal.bsi.ac.id/index.php/ijcs/article/view/10005

itsmariodias. (2023). VQA Validation Dataset [Data set]. Kaggle. https://www.kaggle.com/datasets/itsmariodias/vqa-validation-dataset

Kurniawati, L., Priyanto, D., Ningsih, N. S., Syahrir, M., & Rismayati, R. (2025). Perbandingan Metode Berbasis Decision Tree dalam Deteksi Penyakit Paru-Paru. Jurnal Bumigora Information Technology (BITe), 7(1), 51–62. https://doi.org/10.30812/bite.v7i1.4909

Mulyawan, R., Sunyoto, A., & Muhammad, A. H. (2023). INTERNATIONAL JOURNAL ON INFORMATICS VISUALIZATION journal homepage : www.joiv.org/index.php/joiv INTERNATIONAL JOURNAL ON INFORMATICS VISUALIZATION Pre-Trained CNN Architecture Analysis for Transformer-Based Indonesian Image Caption Generation Model. www.joiv.org/index.php/joiv

Nalendra Zaradiva, M., & Dewi, I. A. (2026). Perbandingan Kinerja Model ResNet-50 dan MobileNetV2 untuk Klasifikasi Citra Menggunakan Metode CNN. DNS-DIGITAL NEXUS SYSTEMATIC JOURNAL, 02(2), 13–22. https://www.kaggle.com/datasets/aryashah2k/breast-

Pamungkas, W. G., Wardhana, M. I. P., Sari, Z., & Azhar, Y. (2023). Leaf Image Identification: CNN with EfficientNet-B0 and ResNet-50 Used to Classified Corn Disease. Jurnal RESTI, 7(2), 326–333. https://doi.org/10.29207/resti.v7i2.4736

Rahmawati, R., Teknik, E., & Salsabila, A. S. (2026). Evaluasi Teknik Prompt Engineering pada LLM Berukuran Kecil untuk Klasifikasi Sentimen Bahasa Indonesia. DNS-DIGITAL NEXUS SYSTEMATIC JOURNAL, 02(2), 1–7.

Rizky, M. A. (2025). Analisis Efektivitas Dua Jenis Gaya Prompt dalam Model LLM Berbasis RAG. Jurnal Komtika (Komputasi Dan Informatika), 9(1), 76–86. https://doi.org/10.31603/komtika.v9i1.13488

Setyadi, R. A., Rahman, S., Manurung, D., Hasanah, M., & Indrawati, A. (2024). IMPLEMENTASI TRANSFER LEARNING UNTUK KLASIFIKASI PENYAKIT PADA DAUN CABAI MENGGUNAKAN CNN. Jurnal Teknologi Informasi, 5(2), 304–315. https://doi.org/10.46576/djtechno

Shafelbilyunazra, A., & Prasetya, D. D. (2025). Pengaruh Few-shot Learning pada Kinerja LLM untuk Ekstraksi Entitas Iklan Lowongan Kerja. Jurnal Pustaka AI (Pusat Akses Kajian Teknologi Artificial Intelligence), 5(2), 418–427. https://doi.org/10.55382/jurnalpustakaai.v5i2.1069

Wijaya, I. N. S. W., Seputra, K. A., & Dewi, N. P. N. P. (2025). FINE TUNNING MODEL INDOBERT UNTUK ANALISIS SENTIMEN BERITA PARIWISATA INDONESIA. Jurnal Pendidikan Teknologi Dan Kejuruan, 22(2), 195–204. https://www.detik.com/search/searchall?query=wisata&siteid=3&sortby=time&fromdatex=01/01/2022&

Yahya, K., Moeis, D., Thamrin, M., Yunarti, S., Felicia Watratan, A., & Mallu, S. (2026). Analisis Perbandingan Efektivitas Zero-Shot vs Chain-of-Thought Prompting dalam Meningkatkan Presisi Informasi pada LLM Berbasis Dokumen PDF. RIGGS: Journal of Artificial Intelligence and Digital Business, 5(1), 2394–2402. https://doi.org/10.31004/riggs.v5i1.6524

Downloads

Published

17-11-2026

How to Cite

IMPLEMENTASI ALGORITMA CONVOLUTIONAL NEURAL NETWORK (CNN) PADA EVALUASI STRATEGI PROMPT ENGINEERING TERHADAP AKURASI VISUAL QUESTION ANSWERING (VQA). (2026). Jurnal Riset Multidisiplin Edukasi, 3(11), 389-410. https://doi.org/10.71282/jurmie.v3i11.2726

Most read articles by the same author(s)

1 2 > >> 

Similar Articles

1-10 of 147

You may also start an advanced similarity search for this article.