INSYST: Journal of Intelligent System and Computation

Hand Sign Virtual Reality Data Processing Using Padding Technique

Teja Endra Eng Tju — 2024-10-02

This study focuses on addressing the challenges of processing hand sign data in Virtual Reality environments, particularly the variability in data length during gesture recording. To optimize machine learning models for gesture recognition, various padding techniques were implemented. The data was gathered using the Meta Quest 2 device, consisting of 1,000 samples representing 10 American Sign Language hand sign movements. The research applied different padding techniques, including pre- and post-zero padding as well as replication padding, to standardize sequence lengths. Long Short-Term Memory networks were utilized for modeling, with the data split into 80% for training and 20% for validation. An additional 100 unseen samples were used for testing. Among the techniques, pre-replication padding produced the best results in terms of accuracy, precision, recall, and F1 score on the test dataset. Both pre- and post-zero padding also demonstrated strong performance but were outperformed by replication padding. This study highlights the importance of padding techniques in optimizing the accuracy and generalizability of machine learning models for hand sign recognition in Virtual Reality. The findings offer valuable insights for developing more robust and efficient gesture recognition systems in interactive Virtual Reality environments, enhancing user experiences and system reliability. Future work could explore extending these techniques to other Virtual Reality interactions.

Implementation of Hand Gesture Recognition as Smart Home Devices Controller

Stanley Dewangga — 2024-10-15

Some current virtual assistant products such as Alexa, Siri and Google Home facilitate features to control smart home devices through voice input, which has become increasingly popular in recent years. In addition to voice input, smart home devices can also be monitored and controlled through smartphones or computers using applications that provide users with flexibility. However, both control systems are less efficient, as they consume time and voice input utilization may sometimes not be recognized in crowded conditions. Therefore, this research introduces an application to recognize real-time hand gestures and utilize them for a new control system that provides time and energy efficiency. This application processes images using the Mediapipe framework, generating hand landmark outputs. These landmark outputs are utilized to determine the combination of raised or lowered fingers, which is then used to control smart home devices. The application is developed with ESP32 and ESP01s modules as data receivers from gesture recognition, and ESP32-CAM for image acquisition. Meanwhile, the gesture recognition computation process is executed on a Raspberry Pi 3 Model B. The gesture recognition application achieves good accuracy at 88%, but may experience occasional failures for certain gestures. However, the response time generated by the smart home control system is still relatively long, averaging 7.88 seconds.

Thesis Defense Scheduling Optimization Using Harris Hawk Optimization

Kevin Setiono — 2024-10-31

This research discusses how the Harris Hawk Optimization (HHO) algorithm handles scheduling problems. The scheduling of thesis defenses at the Institut Sains dan Teknologi Terpadu Surabaya (ISTTS) is a complex issue because it involves the availability of lecturers, teaching/exam schedules, lecturer preferences, and limited room and time availability. The scheduling constraints in this research are divided into two categories: Hard Constraints and Soft Constraints. Hard constraints must not be violated, including each lecturer's unique availability, conflicts, and existing exam or teaching schedules. Soft constraints, on the other hand, include preferences for specific days or rooms for the defense. The complexity of scheduling due to these two types of constraints leads to longer scheduling times and an increased likelihood of human error. To automate and optimize this process, the author employs the HHO algorithm. HHO is inspired by the behavior of the Harris Hawk, known for its intelligence and ability to coordinate while hunting. The results of the HHO algorithm are translated into a slot meter, which helps to map the solution to available time slots. The HHO algorithm can generate schedules that comply with 90% of the hard constraints at ISTTS. Evolutionary algorithms typically have high complexity and computational time; in this case, the researcher experimented with multiprocessing. Multiprocessing improved the computational time by up to 39%.