Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NDCG@10 fails when relevant documents are expressed in gain -> docs format #35

Closed
agazzarini opened this issue Jul 23, 2018 · 1 comment
Assignees
Labels
bug Something isn't working
Milestone

Comments

@agazzarini
Copy link
Member

Describe the bug
NDCG@10 fails when relevant documents are expressed in gain -> docs format

"relevant_documents": { "3": ["1658090", "8927398273", "9879878"] }
To Reproduce
Create a judgment set using the gain -> docs format for expressing judgments.

Expected behavior
The format should be correctly parsed by all RRE components

StackTrace
Caused by: java.lang.NullPointerException at io.sease.rre.core.domain.metrics.impl.NDCGAtTen.lambda$idealDcg$0(NDCGAtTen.java:75) at java.util.stream.Collectors.lambda$groupingBy$45(Collectors.java:907) at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) at java.util.Iterator.forEachRemaining(Iterator.java:116) at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) at io.sease.rre.core.domain.metrics.impl.NDCGAtTen.idealDcg(NDCGAtTen.java:75) at io.sease.rre.core.domain.metrics.impl.NDCGAtTen.access$200(NDCGAtTen.java:24) at io.sease.rre.core.domain.metrics.impl.NDCGAtTen$1.value(NDCGAtTen.java:61) at io.sease.rre.core.domain.DomainMember.lambda$null$4(DomainMember.java:120) at java.util.Optional.ifPresent(Optional.java:159)

@agazzarini agazzarini added the bug Something isn't working label Jul 23, 2018
@agazzarini agazzarini added this to the 1.0 milestone Jul 23, 2018
@agazzarini agazzarini self-assigned this Jul 23, 2018
@agazzarini
Copy link
Member Author

Fixed. The patch provides also some useful logging messages during the evaluation process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant