Yanni and I are freshman at Cornell and MIT respectively. A few years ago, we were going to the same high school when COVID hit and the classroom became our bedroom. Teachers struggled to replicate the interactivity of the classroom and learning outcomes out our school suffered. We realized how important the experience of "raising you hand" was to the classroom environment. Fast forward to today and Yanni and I are studying Artificial Intelligence in college. We observed the public excitement over LLM's and their ability to answer questions. While brainstorming for this hackathon, we realized we could harness LLM's to bring the interactivity of the classroom to every laptop around the world. We envision future where students of all backgrounds have access to a personalized AI assistant.
Pupil is a chrome extension that links to an AI chatbot which answers questions about educational videos. Whenever the student pauses the video Pupil will ask them if they have any questions. The AI has access to transcript information about the video, so its responses consider the section of the video the student is currently watching. Therefore, Pupil's responses are relevant and accurate.
We build a simple Chrome extension that redirects users to a NextJS WebApp. Here, the user can watch an embedded video and interface with the chat.
We ran out of time to implement the voice controls. Initially we hoped to allow the user to speak questions using their mic and receive answers through AI voice (assembly realtime + huggingface TTS). We believe this conversational paradigm is important and we will be implementing it shortly after the hackathon.
We're proud to have worked on a project with contributes to something we are personally passionate about. We're excited to keep working on this project.
We learned how to build a cohesive chat experience using LLM's and develop chrome extensions.
We will be submitting to the Future Finders Government challenge, a $1 million U.S. Department of Education challenge to reimagine career navigation for adult learners. We're confident we can convey our vision that AI learning assistants have a place in the future of US education.