a virtual assistant that helps users improve their conversation skills through voice practice sessions
Munio is a virtual assistant that functions as an English teacher to help users enhance their conversational skills through voice practice sessions. The application's main dependency is Generative AI, which it uses for audio analysis, generating phrases, and providing feedback. In this project, we utilize Gemini AI to fulfill these requirements.
The application offers two modes: sessions and conversations.
In this mode, users will answer random phrases based on context and level requested by themselves. After each lesson and at the end of the session, they will receive an overall feedback, helping them understand how to improve.
- Phrase generation using Gemini AI, based on context and level requested by the user;
- Audio analysis using Gemini AI, providing feedback about the user's speaking and pronunciation;
- Audio upload using Google Cloud Storage;
- Session overall feedback using Gemini AI.
In this mode, users will have a realistic dialogue with an AI (using Text to Speech) generated based on context and level requested by them. At the end of the conversation, they will receive a general feedback that presents ways for improve their conversation skills.
- Realistic dialogue generated by Gemini AI, based on context and level requested by the user;
- Audio analysis using Gemini AI, providing feedback about the user's speaking and pronunciation;
- Audio upload using Google Cloud Storage;
- Text to Speech using Google Cloud Text to Speech;
- Conversation overall feedback using Gemini AI.
To improve user experience, we use Websockets (Socket.io) for a more natural real-time interaction.
This is a public version of the frontend-end application and the backend-end can be found here: web-server.
npm ci# development
npm run start
# production
npm run build
# serve the application
serve -s dist/web-client/browser
- Back-end (Integration with AI, Authentication, Uploads, etc): web-server
- Markdown to HTML: Markdown-it
- Audio Visualization: WaveSurfer
- Audio Recording: RecordRTC
- Framework: Angular 18
- UI: Angular Material 3
- Websockets: Socket.io
- Author - Gabriel Sena
- Website - https://munio.cloud