Skip to content

coalaroot/Pytorch-challenge-side-project

Repository files navigation

Real-time Sign Language Interpreter

We are a team of volunteers from PyTorch Scholarship Challenge from Facebook hosted by Udacity creating awesome Deep Learning models.

Dataset

Sign Language Dataset

File Description

  • main.py
    • Entry point of the application for detecting sign language gesture
  • collect_data.py
    • Entry point of the application for creating new sign language data

Setup

Run the following command to install all required python packages

pip install -r requirements.txt

How to Detect Gesture

Run the following command to install all required python packages

python main.py
Instructions
  • Place your hand on the screen around the green boxes as shown on the figure below
  • Press 'z' button to start hand detection
  • Press 'z' button once again to stop hand detection
  • Press ESC button to exit the application

Hand

How to Add new image data

See the sign language number gesture in https://github.com/ardamavi/Sign-Language-Digits-Dataset

Run the following command to install all required python packages

python collect_data.py
Instructions
  • Press 'z' on your keyboard to start detecting hand
  • Press 0-9 on your keyboard to change active label
  • Press 'c' on your keyboard to capture a single data
  • Press 's' on you keyboard to save all the captured data into file, the file will be located in the current application as X_{datetime}.npy and Y_{datetime}.npy
  • Press 'd' on your keyboard to delete last recorded data
  • Press ESC button to exit the application

Watch the example of adding new image data on https://youtu.be/lEJL5Xflwjo

References

[1] Kaggle Example Script

[2] Finger Detection

[3] Hand Detection

Notes

We also developed a sign language interpretation android application. Please check our repository on https://github.com/forfireonly/SignLanguage

More about our project

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published