Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AUROC for binary task, if thresholds is set, results in an error #1641

Closed
ahmed-alhindawi opened this issue Mar 21, 2023 · 1 comment · Fixed by #1642
Closed

AUROC for binary task, if thresholds is set, results in an error #1641

ahmed-alhindawi opened this issue Mar 21, 2023 · 1 comment · Fixed by #1642
Labels
bug / fix Something isn't working help wanted Extra attention is needed
Milestone

Comments

@ahmed-alhindawi
Copy link

🐛 Bug

The AUROC metric for a binary task has an optional thresholds argument. It documents that if it is set to an int, then that number of bins is set, otherwise if its a List of floats, then the AUROC at those thresholds is calculated. This is so that a faster calculation can occur. However, if the thresholds argument is set to either of those things, then an error occurs:

RuntimeError: "bincount_cuda" not implemented for 'Float'

To Reproduce

Steps to reproduce the behavior...

import torchmetrics
import torch

labels = torch.Tensor([0, 1]).cuda()
probs = torch.Tensor([0.1, 0.9]).cuda()

metric = torchmetrics.AUROC(task="binary", thresholds=20).cuda()
print(metric(probs, labels))

of note, removing .cuda() also results in an error:

RuntimeError: "bincount_cpu" not implemented for 'Float'

Expected behavior

The AUROC should be calculated along the fast O(n_thresholds) rather than the O(n_samples)

Environment

Installed from Conda with the following other relevant libraries:

  • TorchMetrics 11.4 (and 11.3.1)
  • Pytorch 1.13.0
  • Python 3.10
@ahmed-alhindawi ahmed-alhindawi added bug / fix Something isn't working help wanted Extra attention is needed labels Mar 21, 2023
@github-actions
Copy link

Hi! thanks for your contribution!, great first issue!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug / fix Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants