qrels binary file?

[harryhoch]

I'm interested in trying this tool out, but I run into a problem running input_fields_experiment.py. it blows up at the end with a FileNotFound Error: No such file or directory: 'data/external/qrels2021_binary.txt'

It looks like prepare_data.py doesn't pull this file down. Where do I find it?

thanks.

[WojciechKusa]

Hi @harryhoch, thanks for your interest!
The graded qrels (0 - irrelevant, 1 - excluded, 2 - included) file for the year 2021 can be downloaded from ir_datasets here. To get the qrels2021_binary.txt file, you need to convert all the rows marked as "excluded" to 0 and all those marked as "included" to 1. I'll try to add that script in the future.

[harryhoch]

thanks. I'm not sure when/if I'll be able to look at this, but I will try.