Stormhacks Hackathon
Getting started Problem Main Features
Many students aren’t the fastest notetakers and the act of note taking can make lectures less engaging as the student is distracted from participating in class discussions and active thinking during lectures. Thus students are forced to go back to the lecture notes/recording to gather information that has been missed; creates gaps in the learning of students. Moreover our software can show students how what they are learning is relevant in the "real world" and how they might use it in their future careers.
You’ll never have to multitask listening to the professor and copying down information on slides ever again.
Live audio and visual transcription provided by AssemblyAI is used to transcribe the speaker and keywords are picked out to generate topic maps. OpenCV enables us to capture the student’s screen and send it to Tesseract for optical character recognition. Notes are then placed on an HTML doc for the student
As the professor is teaching whats happening is that we are looking for keywords such as nouns and key technical terms the professor mentions using machine learning.
Did the professor leave some things out? Three-link deep Wikipedia linking will help fill in knowledge gaps.
Wikipedia’s API is used to collect relevant info and topics. These keywords will then be used to perform a search through Wikipedia (web scrape and collect links)
Create a network of potentially related topics; makes it easier for student to look at certain topics that they aren't familiar with.
Not sure how what you’re learning is relevant? Automated Knowledge Maps show relation between old and new knowledge.
The software will use computer vision, look for things diagrams, equations and some bold ideas and then it will put a timestamp of where it was taken and the picture of the formula
The program will then create an html page where all the links and notes (diagrams/equations) are displayed.
noteAI is built using Python, AssemblyAI, OpenCV, and HTML. Wikipedia links are gathered using web scraping through Wikipedia APIs. Using the AssemblyAI live transcribing, the text is parsed, and subject-specific words. Computer vision and optical object detection then compare slides of the lecture material to look for important formulas and keywords. HTML was then used to combine the equations, resources, and the topic map for the lecture to create a user-friendly experience for the student.