-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve default evaluation setting #107
Conversation
… qrels in `qrels/qrels.401-450.txt`
Synch with master
@arjenpdevries I'll let @r-clancy look at this in case he's in the middle of something else. |
I'll merge this, but for the rest of the official run through I'll keep P@30 in my local copy of jig so I can confirm P@30 numbers (which is what most people have in their README now). |
@arjenpdevries would you be willing to change over to P@30 in OldDog to be consistent with others? The primary metrics we've decided on is NDCG@20, AP, and P@30, IIRC... |
Definitely! |
After this merge we will find other values because of |
Probably should only merge the - c modification then? |
Yeah, let's just do the |
Ok 👌 later tonight |
Address evaluation metrics to be reported on all topics (
-c
flag),re: #105
And, report P@5 by default, just like the "official" evaluation at OSIRRC 2019.