To reproduce all the results from our EACL 2021 paper, all one has to do is run:
git clone https://github.com/machamp-nlp/machamp.git
cd machamp
git reset --hard b81c539a7e29b2364cf1269b3ef7b9114e2dba10
pip3 install requirements.txt
# now collect all hidden datasets (this is the hardest part),
# or be ok with leaving them out of the experiments
./scripts/runAll.sh
In practice however, this would take very very long. So we suggest to pick the experiment
you are interested in from the runAll.sh
file, and manually run it multi-threaded with e.g. parallel
or in SLURM
.
The experiments are numbered, so you can see which scripts to run for each part in `runAll.sh.
We also provide all the predictions of our model in a .tar.gz
archive, it can be downloaded
at: http://itu.dk/people/robv/data/machamp/preds.tar.gz
All scripts starting with 5.
can be used to generate the tables/graphs in the paper.