Barcodes are placed on nearly every product, but what if they aren’t needed? I explored the potential use of artificial intelligence to recognize supermarket items without the need for a barcode. I constructed WaveOut, an AI-powered self-checkout system designed to be both user-friendly and efficient. Simply wave your items at the camera and the AI will recognize it and add it to your cart. This can vastly improve the efficiency of self-checkouts and change the future of shopping.
View Code
·
Report Bug
·
Request Feature
Table of Contents
From August 2020 to March 2021, I worked on a large-scale project called WaveOut, an AI-powered self-checkout system that allowed customers to wave the products at a camera to add them to their cart. I began this project in a semester-long course at my high school called Catalyst, but I continued to work on the project after completing the course.
I started this project due to a flaw I observed in unmanned retail stores. Unmanned retail stores have emerged as a promising solution in cities in developed countries where high labor cost is an issue. Without human labor, the store can safely operate 24/7. Due to this, several unmanned convenience stores have sprouted up near me and were featured heavily in the news as the answer to retail in future smart cities.
Yet, a problem I have noticed is that the self-checkout systems for almost all stores are barcode or RFID-based. Every item needed a label printed or placed to allow barcode/RFID scanners to identify the objects. While this works, sticking a label on every single product is time-consuming. I observed the store workers at the unmanned store named Octobox near my home manually tagging every item with RFID tags, defeating the purpose of unmanned operations. This observation led me to build this self-checkout system so that labels are no longer needed. My program uses the camera on the computer to recognize the object and add it to your cart.
During this journey, not only did I learn so many new skills when it comes to coding, including Artificial Intelligence, JavaScript, HTML, CSS, Google TensorFlow, and Paypal API, but I also experienced how sophisticated and difficult it is to make a working self-checkout. When using a self-checkout, it seems remarkably simple: scan a barcode, pay, and leave. But programming all these modules AND incorporating a reliable object recognition neural network is difficult and time-consuming. However, after eight months, the final result made it rewarding.
I initially tested the system with objects at home. However, starting in August 2020, I began working with Ascencio Pte Ltd, a company that runs school uniforms and bookshop operations in more than 50 local schools in Singapore, to test and implement my self-checkout system. The initial test was with 14 random items from the store, ranging from school uniforms to pencils to glue. After conducting three trials with the 14 different products, the system recognized the current item 95.2% of the time. Due to the high percentage, the company was pretty satisfied with the results.
To get a local copy up and running, follow the instructions below:
This project requires no prerequisites
- Ensure all prerequisites are installed correctly
- Clone the repo
git clone https://github.com/kea-roy/WaveOut.git
- Open
WaveOut.appor if it is not compatible with your operating system, directly openindex.htmlin a browser.
See below for a video demo of the project
WaveOut_demo_video.mp4
See the open issues for a full list of proposed features (and known issues).
Kea-Roy Ong - ko353@cornell.edu
Project Link: https://github.com/kea-roy/WaveOut