Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Plotting of metrics #1328

Merged
merged 75 commits into from
Dec 23, 2022
Merged

Feature: Plotting of metrics #1328

merged 75 commits into from
Dec 23, 2022

Conversation

SkafteNicki
Copy link
Member

@SkafteNicki SkafteNicki commented Nov 10, 2022

What does this PR do?

Fixes #652
Fixes #1265

This PR adds basic plotting functionality to torchmetrics. Idea is that every modular implementation should have an plot method that can create a basic plot of the metric. All plot methods should have the signature:

def compute(self) -> TypeT:
    ...

def plot(self, val: Optional[TypeT, Sequence[TypeT]])
    val = val | self.compute()
    ...

meaning that if no value is provided, self.compute is called. Additionally, either a single value can be provided or a list of values to plot a metric as it has progressed. Please try executing the examples/plotting for examples.

This PR is meant as an proof-on-concept and when it (hopefully) gets merge we can follow up with adding the functionality to other metrics.

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@SkafteNicki SkafteNicki added the enhancement New feature or request label Nov 10, 2022
@SkafteNicki SkafteNicki added this to the v0.11 milestone Nov 10, 2022
@SkafteNicki
Copy link
Member Author

Executing examples/plotting.py should give something like this:

Single metric value
Figure_a

Single metric value, one per class
Figure_b

Multiple metric values
Figure_c

Multiple metric values, one per class
Figure_d

src/torchmetrics/classification/accuracy.py Outdated Show resolved Hide resolved
src/torchmetrics/utilities/plot.py Outdated Show resolved Hide resolved
src/torchmetrics/utilities/plot.py Outdated Show resolved Hide resolved
src/torchmetrics/utilities/plot.py Outdated Show resolved Hide resolved
@justusschock
Copy link
Member

Something more general: Do we want to support smoothing? Especially for multiple timepoints this is often done to get a more general idea of the progress instead of spiky individual values (See tensorboard for example).

@SkafteNicki
Copy link
Member Author

Something more general: Do we want to support smoothing? Especially for multiple timepoints this is often done to get a more general idea of the progress instead of spiky individual values (See tensorboard for example).

I would be fine with supporting this :)
We would need to add scipy as an dependency, but thats the only "problem".

@mergify mergify bot removed the has conflicts label Dec 22, 2022
@Borda Borda disabled auto-merge December 22, 2022 17:32
@Borda Borda enabled auto-merge (squash) December 22, 2022 17:33
@mergify mergify bot added the ready label Dec 22, 2022
@Borda Borda merged commit a336f97 into master Dec 23, 2022
@Borda Borda deleted the feature/plot branch December 23, 2022 02:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request ready
Projects
None yet
5 participants