This downloads the WMT16 dataset and extracts it into a new folder called wmt16
. If the folder wmt16
exists, it will skip the process.
bash download_data.sh
Here is an example of tuning three models in a row:
python tune_layers.py -m bert-base-uncased roberta-base albert-base-v2
The results would be appended to best_layers_log.txt
.
The last three lines of best_layers_log.txt
would be
'bert-base-uncased': 9, # 0.692518813886652
'roberta-base': 10, # 0.7062886932674598
'albert-base-v2': 9, # 0.6682362357086912
which shows the model name, the best number of layers, and the pearson correlation with human judgement.
These can be copied and pasted into model2layers
in bert_score/utils.py
.