Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ranking prediction are all same value #6955

Closed
1a1a11a opened this issue May 12, 2021 · 2 comments · Fixed by #8822
Closed

Ranking prediction are all same value #6955

1a1a11a opened this issue May 12, 2021 · 2 comments · Fixed by #8822
Labels
LTR Learning to rank

Comments

@1a1a11a
Copy link

1a1a11a commented May 12, 2021

I am using the XGBoost C API to do some ranking tasks and I have tried to use rank:pairwise, rank:map, and rank:ndcg as the objective, however, when I use rank:map, and rank:ndcg to train and predict, all the predicted values are constant, but it is not a problem when I use rank:pairwise. Does anyone know what could be wrong in my setup? Or is there a better way other than digging into the source code? Thank you!

@1a1a11a
Copy link
Author

1a1a11a commented May 12, 2021

I have checked that the relevance in the training data are all from 0 to 1, and the inference data are not constant. I think the training has some problems, the metric does not change at all from the first iteration to 10 iterations (while it changes if I use
regression or rank:pairwise.

@trivialfis trivialfis added the LTR Learning to rank label May 13, 2021
@creat89
Copy link

creat89 commented May 30, 2021

For NDCG, and I'm guessing MAP too, the relevance scores must be integers. See:

#6352 (comment)

or:

https://discuss.xgboost.ai/t/very-large-ndcg-result/1712

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
LTR Learning to rank
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants