Botagoz App

Android application for AI assistant designed specifically to empower individuals with visual impairments, which provides users with a seamless and supportive experience in navigating their surroundings.

Demo

Untitled.video.-.Made.with.Clipchamp.mp4

Algorithm

An Android phone sends the photo and the question to the FastAPI backend. It appends the question to the chat log. Then, a classifier based on ChatGPT analyzes whether it needs visual context. If so, it uses BLIP via the Replicate.com API and processes it. If not, it answers the question as it is. To see the implementation, please refer to the backend repository.

Features

• image capturing
• image processing
• speech recognition

Stack

Language: Kotlin

Framework: Native Android UI

Architecture: MVVM

Libraries

• Native Camera API
• Speech Recognizer
• Text-to-Speech

Screenshots

Speech Recognition	Responding	Answer is displayed and voiced

This application was developed for the GDSC Solution Challenge 2023. You can watch our demo video here