IMPLEMENTASI HAND TRACKING BERBASIS MEDIAPIPE SEBAGAI METODE KONTROL PADA GAME SIMULASI DELIVERY “TOFU DELIVER”
DOI:
https://doi.org/10.71282/jurmie.v3i8.2526Keywords:
Unity, MediaPipe, Computer Vision, Hand Tracking, Game SimulationAbstract
Advances in computer vision technology enable human-computer interaction without relying on conventional input devices such as keyboards or mice. One prominent technology being widely developed is MediaPipe-based hand tracking, which is capable of mapping hand positions in real-time using an RGB camera. This research implements a hand tracking system using Python and MediaPipe as an alternative control method in a delivery simulation game titled Tofu Deliver. The game was developed using the Unity engine and features a core gameplay concept centered around the production and distribution of tofu to customers. The system architecture incorporates two control modes—keyboard and hand tracking—integrated via TCP socket communication between Unity and Python. Evaluation was conducted to test the accuracy of gesture detection, system responsiveness, and user experience aspects. The implementation results indicate that the hand tracking system successfully drives the vehicle in real-time with optimal response levels, thereby offering a more natural interaction preference compared to conventional control schemes.
Downloads
References
Alvin, A., Husna Shabrina, N., Ryo, A., & Christian, E. (2021). Hand Gesture Detection for American Sign Language using K-Nearest Neighbor with Mediapipe. Ultima Computing : Jurnal Sistem Komputer, 13(2).
Chen, J., Zhao, S., Meng, H., Cheng, X., & Tan, W. (2022). An interactive game for rehabilitation based on real-time hand gesture recognition. Frontiers in Physiology, 13. https://doi.org/10.3389/fphys.2022.1028907
D’SOUZA, S. M., TEJA, K. S. D., GEETHIKA REDDY, V., REDDY, N. L., & T, P. K. (2022). GAME AUTOMATION USING MACHINE LEARNING. IJARCCE, 11(12). https://doi.org/10.17148/ijarcce.2022.111246
Filius Alsaro Ginting, R., & Elvira Messe, F. (2025). PENGEMBANGAN GAME EDUKASI PUZZLE BERBASIS COMPUTER VISION HAND TRACKING UNTUK ANAK USIA DINI. https://doi.org/10.52972/hoaq.vol16no1
J. Kotwal, A. R. N. S. B. C. P. K. dan V. K. (2025). Gesture Recognition-Based Game Using Media Pipe.
Jo, B. J., Kim, S. K., & Kim, S. K. (2023). Enhancing Virtual and Augmented Reality Interactions with a MediaPipe-Based Hand Gesture Recognition User Interface. Ingenierie Des Systemes d’Information, 28(3), 633–638. https://doi.org/10.18280/isi.280311
Katiyar, A. K., Choi, J., & Ahn, J. H. (2025). Recent advances in CMOS-compatible synthesis and integration of 2D materials. In Nano Convergence (Vol. 12, Number 1). Korea Nano Technology Research Society. https://doi.org/10.1186/s40580-025-00478-1
Shreenethi, A. M. , V. S. , & J. R. (2025). Hand Gesture Game Controller Using Machine Learning.
Sole, P. T., Nangude, P. K., Gond, N. K., Patil, K. A., & Raut, E. (2023). Gesture Controlled Recognition Based Game Using Mediapipe and PyGame. International Journal of Advanced Research in Computer and Communication Engineering Impact Factor, 8. https://doi.org/10.17148/IJARCCE.2023.121007
Sung, G., Sokal, K., Uboweja, E., Bazarevsky, V., Baccash, J., Bazavan, E. G., Chang, C.-L., & Grundmann, M. (2021). On-device Real-time Hand Gesture Recognition. http://arxiv.org/abs/2111.00038
Yaseen, Kwon, O. J., Kim, J., Jamil, S., Lee, J., & Ullah, F. (2024). Next-Gen Dynamic Hand Gesture Recognition: MediaPipe, Inception-v3 and LSTM-Based Enhanced Deep Learning Model. Electronics (Switzerland), 13(16). https://doi.org/10.3390/electronics13163233
Zheng, Y., Shishi, C., & High School, B. (2023). Gesture Recognition Based on MediaPipe for Computer Game Control. https://doi.org/10.47297/wspciWSP2516-252721.20230702
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Andrea Ramadhan Cahya Aryana, Ina Sholihah Widiati, Sri Widiyanti (Author)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
