PSA Cards Grading Module

Introduction

PSA Cards Grading Module is an AI model built for grading the quality of trading cards of 4 main aspects: Centering, Corners, Edges, and Surface. It uses one of the instances of transfer learning technique, fine-tuning, to construct the model using baseline models such as VGG16, ResNet, or MobileNet.

The current model, built on VGG16, provides two seperate network flow to learn different aspects of the image. Then, all information is gathered as a single 1-D layer, and it is propagated through several layers before outputing a grade for an aspect. Before feeding the images to the model, different layers of preprocessing are adopted (including cropping the card content using U-Net) to transform the images into a more "learnable" format.

Based on previous evaluation, it can be seen that there are some very substantial results as the average difference between the target grade and the output from the model is about 0.5 (on the scale of 10).

Project structure

Before diving in the project, we will describe the current structure of this project, along with some important folders that you should notice.

- .log: Containing logs of the most recent training session
- analysis: jupyter notebooks of analysis of grading cards and the model's result. To see how grading models have performed, prefer to the "analysis/validation_analysis/result_visualization.ipynb".
- checkpoint: checkpoints of the models. For the model that assists in cropping card, it is stored in "checkpoint/cropper". As for the grading models, it is in "checkpoint/vgg16_grader_M" (with M as the score aspect).
- data: Containing images of cards. This is the folder that is used for storing card image for training the grading models. The images are stored in "data/[id]/[back and front].jpg (with id is the id of the card). Furthermore, this folder also contains a grades.csv, which is a list of scores of each aspect or the cards in the folder data.
- models: Model class.
- task: A list of available tasks written in Python.
- unet_labeled: Containing images of cards, together with labels for the cropping model (U-Net). Images are stored in "unet_labeled/[test or train folder]/[id]/[back and front].jpg, and the annotations (generated by Labelme annotation tool) are stored in "unet_labeled/[test or train]/[id]/[back and front].json.
- utils: Other utilities function written in Python.

Updating the model

We list of several existing tasks to retrain the model with new images. Currently, there are 4 main tasks:

• Extracting preprocessed data for cropper model.
• Extracting preprocessed data for grading models.
• Training cropping model from preprocessed data.
• Training grading models from preprocessed data.

The commands for each of these tasks are as follows:

To preprocess data for cropper model
python3 -m task.extract_cropper_data \
    --origin_img_height [integer, defaults as 3147] \ # Origin height of the image
    --origin_img_width [integer, defaults as 1860]\ # Origin width of the image

To preprocess data for grading models
python3 -m task.extract_grade_data \
    --train_directory [string, default as 'data'] \ # directory containing card images.
    --grade_path [string, default as '[train_directory]/grades.csv'] \ # csv file path containing grades of card images
    --origin_img_height [integer, default as 3147] \ # height of the original trading card image.
    --origin_img_width [integer, default as 1680] \ # width of the original trading card image.
    --model_score_type [list of strings] \ #score type to extract data. Currently it supports Centering, Surface, Edges, and Corners.
    --enable_ray [boolean flag] # add this flag if you want to preprocess data using with multiplee processes. 
Data will be saved in preprocessed_data/[score_type]

To train cropping model
python3 -m task.train_cropper \
    --origin_img_height [integer, default as 3147] \ # original height of the image
    --origin_img_width [integer, default as 1680] \ # original width of the image
    --epochs [integer, default as 80] \ # number of training rounds
    --batch_size [integer, default as 16] \ # batch size
    --val_ratio [float, default as 0.25] # ratio of the validation dataset

python3 -m task.train_grader \
    --grade_path [string, default as 'data/grades.csv'] \ # csv file path containing grades of card images
    --batch_size [integer, default as 32] \ # batch size for dataset
    --epochs [integer, default as 50] \ # Number of training rounds for the model.
    --val_ratio [float, default as 0.25] \ # ratio of the validation dataset
    --learning_rate [float, default as 0.001] \ # learning rate for the model. Default as 0.001
    --model_score_type [list of strings] \ # List of score types. Currently it supports Centering, Surface, Edges, and Corners.
    --clean_log [boolean flag] \ # enable this flag if log data needs to be cleaned
    --clean_checkpoints [boolean flag] \ # enable this flag if model checkpoints need to be cleaned

Installation and usage

Below are the steps required to install and run the grading components:

• Install Python based on your OS.

• Install neccessary libraries:

pip install -r requirements.txt

• Make sure that the checkpoints are available in folder "checkpoint" (which includes "cropper", "vgg16_grader_Centering", "vgg16_grader_Corners", "vgg16_grader_Edges", and "vgg16_grader_Surface").
• Run this command to start Django web server:

python3 manage.py runserver

• From then, you can go to this page to upload your image and get graded by the AI models.