Spatio-Temporal Wound Stage Classification

University of California - Santa Cruz

Student: Alex Salman

Last modified: June 3, 2022

Project Description

This project proposes an architecture for classifying wound healing stages of a series of wound images. We generate a series of consecutive wound image frames and feed them to a 2D convolutional neural network combined with long short-term memory unit and a 3D convolutional neural network to learn spatio-temporal features associated with the healing trajectory. We also visualize the saliency maps to identify features the model is extracting. Both models can extract visual features related to wound healing and have relatively high classification accuracy.

Project Features

1. Generate videos from images using openCV
2. Pad different length generated videos to have the same length (16 frames)
3. Name vidoe after the name of the last frame appended
4. Normalize array of frames to values between 0-1
5. Split data to 75% training, 12.5% validation, and 12.5% test
6. Learn spatio-temporal features using 2D-CNN+LSTM and 3D-CNN
7. Has two versions for each model for high and low resolution images
8. Visualize features extracted using saliency maps

Dataset

Wound image dataset

Processed dataset repository

Labels

Hemostasis, Inflammatory, Proliferative, and Maturation

Models

See the following .ipynb files in this reposetory:

2D-CNN+LSTM for Splint Crop

2D-CNN+LSTM for Circle Crop

3D-CNN for Splint Crop

3D-CNN for Circle Crop

Models Graphs

2D-CNN + LSTM (Splint Crops)

2D-CNN + LSTM (Circle Crops)

3D-CNN (Splint Crops)

3D-CNN (Circle Crops)

Saliency_Maps

2D-CNN+LSTM Circle Crop

3D-CNN Splint Crop

Shell Commands

Installing required packages

pip3 install -r requirements.txt