Fitzpatrick17k_Training

Fitzpatrick17k_Training

Directory containing the code for reproducing our benchmark experiment results.

Directory Structure

• Model training and evaluation code:

  • run.sh: Run script for all the benchmark experiments on Fitzpatrick17k-C.
  • train.py: The training and evaluation script, adopted from the original file, with modifications as listed in the paper and detailed below.
  • utils.py: Utility functions for various training-related tasks.

• Hyperparameter search code:

  • hparamsearch_train.py: The script for hyperparameter search over the choices of number of training epochs, optimizer, and learning rate, over all the seven holdout sets, each repeated 3 times.
  • hparamsearch_run.sh: SLURM script for the hyperparameter search.
  • hparamsearch_logs.csv: The log file containing the results of the hyperparameter search.
  • hparamsearch_analyze_logs.py: The script for analyzing the results of the hyperparameter search and returning the best hyperparameters and their corresponding results.

Modifications to train.py

Our training code is built upon the original train.py by Groh et al. with the following modifications:

• We use separate partitions for validation (i.e., picking the best performing model across training epochs based on the highest validation accuracy ) and for testing (i.e., for reporting the final Fitzpatrick17k-C benchmarks).
• We change the optimizer to SGD (from Adam in the original code), and train for 100 epochs with a learning rate of 0.001, weight decay of 0.001, and momentum of 0.9, and mini-batch size of 32.
• We use 🤗 Accelerate for mixed-precision training.
  • This can easily be disabled by setting mixed_precision to no on L#22.

Additional edits to the code that do not affect the functionality:

• We use comet_ml for experiment management and tracking.
• The flatten() function has been moved to utils.py with other utility functions for better organization.