An App To Provide Gesture and Voice Control for Liquid Galaxy.
- Key Features - How to Use - Download - Work Done - Credits - License
-
Gesture Recognition:
- Utilizes ML Kit and Mediapipe for real-time and accurate hand gesture detection.
- Advanced machine learning algorithms interpret hand movements for precise control over the Liquid Galaxy Rig.
-
Fluid Gesture Mapping:
- Maps distinct gestures to specific commands, enhancing control intuitiveness.
- Common gestures like hand movements are associated with meaningful actions in the Liquid Galaxy environment.
-
Real-time Interaction:
- Instant responsiveness for seamless navigation and manipulation of Liquid Galaxy content through gestures.
-
Voice Recognition:
- Integrates advanced voice recognition for accurate interpretation of user commands.
- Enables hands-free control of the Liquid Galaxy Rig via voice inputs.
-
Customizable Voice Commands:
- Define personalized voice commands for seamless Liquid Galaxy navigation.
- Easy-to-use trigger phrases for users such as Move, Fly To, Zoom, Orbit, and much more!
-
Intuitive UI Design:
- User-friendly interface with straightforward navigation for effortless gesture and voice control.
- Prioritizes visual appeal and ease of use to enhance user experience.
-
Comprehensive Help Section:
- Extensive help and support resources catering to all user levels.
- Voice and Gesture samples to assist users in navigating the application with ease.
-
Sample Commands and Gestures:
- Diverse collection of voice samples and gestures showcasing app capabilities.
- Highlights applications of gesture and voice controls in the Liquid Galaxy environment.
To start using the app, follow these steps:
-
Open the app on your device.
-
Navigate to the "Settings" section.
-
Provide the following details to connect to a Liquid Galaxy system:
- IP Address
- Username
- Password
- Number of Screens
-
Once connected, you can use the app in two ways:
- Voice Recognition: Access the "Voice" section to control the Liquid Galaxy Rig using voice commands.
- Gesture Recognition: Navigate to the "Gesture" section and use intuitive hand gestures to control the system.
You can download the latest version of the app from the following sources:
- Worked on the initial build of the LG Gesture And Voice Control project.
- Developed the UI for the landing page of the LG Gesture And Voice Control App.
- Implemented the Splash Screen for the app.
- Established the LG Rig connection using the ssh2 package.
- Researched common LG functions like orbit, fly to, and LG commands for shutdown, reboot, relaunch, and KML clearing.
- Added screen overlay for logos to be displayed on the LG view.
- Enhanced the User Interface (UI) of the Help and Menu pages for the LG Gesture And Voice Control App.
- Worked on connecting LG Rig using the ssh2 package.
- Implemented UI changes for the connection screen and About page.
- Created a button that changes color on successful LG Rig connection.
- Added error detection functions for SSH connection in case of misconnection.
- Implemented additional alert messages and dialog boxes on successful LG Rig connection.
- Developed a state manager for the app and integrated it with LG Rig.
- Finished implementing the state manager for the LG Gesture And Voice Control App.
- Added KML commands for changing the LG Rig's location when connected to the app.
- Created Splash Screen and Logos screen to be displayed when connected to the LG Rig via the app.
- Began working on the AI architecture and proper routes for the app after connecting to LG Rig.
- Added required dependencies to LG Rig for voice recognition purposes.
- Integrated voice recognition functionalities into the app to understand human voice.
- Highlighted specific voice commands made to the app.
- Altered permissions and build to accommodate LG Rig changes, updated GitHub progress.
- Worked on connecting voice commands to appropriate KMLs.
- Created functions for LG Rig rebooting, relaunching, and shutting down from the app.
- Developed and integrated a State Manager for the LG App.
- Connected the LG Rig to the Statemanager for better control.
- Added KML commands for connecting voice commands to appropriate LG Rig actions.
- Created UI for the connection status of the app to the LG Rig.
- Successfully connected the LG Rig to the app.
- Created functions for voice commands related to LG Rig movements.
- Updated UI for voice detection page and Settings Page.
- Debugged and fixed errors in the app based on Lab Testers' feedback.
- Created Playstore material for the app and uploaded it to Drive.
- Added remaining commands for LG Rig orbit to the app.
- Tested application and analyzed gesture detector performance.
- Optimized gesture detector and analyzed remaining bugs.
- Worked on optimizing gesture recognition for better performance.
- Implemented remaining error fixes and optimization in the gesture detector.
- Tested optimized gesture recognition on a local machine.
- Adjusted gesture detector speed to improve performance.
- Worked on the integration of gesture detection with the app.
- Debugged and fixed errors mentioned by Lab Testers.
- Made UI improvements for the Home Page and About Page.
- Debugged and fixed errors mentioned by Lab Testers.
- Continued updating documentation and ensuring completeness of code.
- Continued debugging and addressing any remaining errors.
- Analyzed the performance of gesture detection.
- Tested app thoroughly and updated GitHub with progress.
- Completed integration of gesture detection and voice commands with the app.
- Finished updating documentation and rechecked for any missing codes or dependencies.
- Continued working on debugging and finalizing the app.
- Focused on optimizing gesture detection and UI elements.
- Resolved any remaining bugs or issues.
- Made final adjustments to the app's functionalities.
- Completed final testing and ensured smooth operation.
- Prepared for the closing and finalization phase.
- Reviewed the entire project for any last-minute errors.
- Ensured that all features were working correctly.
- Finalized and submitted the project.
I extend my heartfelt gratitude to the following individuals, organization as well as to the Lab Testers at Lleida Lab for their invaluable contributions to this app:
- Liquid Galaxy Labs
- Organization Administrator: Andreu Ibanez
- Main Mentor: Merul Dhiman
- Sub Mentors: Gabry and Alejandro Illán Marcos
Your guidance and support have been crucial in completing this project.
This project is licensed under the Apache License 2.0.
