Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Detection: panoptic quality #929

Merged
merged 75 commits into from
Feb 14, 2023

Conversation

niberger
Copy link
Contributor

@niberger niberger commented Apr 4, 2022

What does this PR do?

Add the panoptic quality metric, as defined in the reference paper.
This implementation differ from the reference implementation of the COCO dataset on the following points:

  • The categories have to be declared at metric initialization.
  • Any undefined category will be considered as an unlabelled (void) region.
  • The input format for panoptic images are Tensor of shape (height, width, 2), where the first number of each pixel is the category id and the second number is the instance id.
  • No support for group of instances (crowds in COCO).

Fixes #50
closes #1435

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@codecov
Copy link

codecov bot commented Apr 4, 2022

Codecov Report

Merging #929 (781fe83) into master (61c2385) will decrease coverage by 51%.
The diff coverage is 93%.

Additional details and impacted files
@@           Coverage Diff            @@
##           master    #929     +/-   ##
========================================
- Coverage      88%     37%    -51%     
========================================
  Files         213     216      +3     
  Lines       10987   11139    +152     
========================================
- Hits         9660    4146   -5514     
- Misses       1327    6993   +5666     

@Borda Borda changed the title [WIP] Feature/50 panoptic quality Detection: panoptic quality Apr 4, 2022
Copy link
Member

@SkafteNicki SkafteNicki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few comments

docs/source/references/modules.rst Outdated Show resolved Hide resolved
docs/source/references/modules.rst Outdated Show resolved Hide resolved
torchmetrics/detection/panoptic_quality.py Outdated Show resolved Hide resolved
torchmetrics/functional/detection/panoptic_quality.py Outdated Show resolved Hide resolved
torchmetrics/functional/detection/panoptic_quality.py Outdated Show resolved Hide resolved
torchmetrics/functional/detection/panoptic_quality.py Outdated Show resolved Hide resolved
torchmetrics/functional/detection/panoptic_quality.py Outdated Show resolved Hide resolved
torchmetrics/functional/detection/panoptic_quality.py Outdated Show resolved Hide resolved
torchmetrics/functional/detection/panoptic_quality.py Outdated Show resolved Hide resolved
torchmetrics/functional/detection/panoptic_quality.py Outdated Show resolved Hide resolved
@mergify mergify bot added the has conflicts label Feb 7, 2023
@marcocaccin
Copy link
Contributor

Hi @Borda, I just reached out on PTL's Slack to offer help if it's useful to have another pair of hands to bring this PR to the finish line ;)
Quick question: Metric.update() is expected to work on inputs with a batch dimension, right? This implementation works for individual samples only.

@mergify mergify bot removed the has conflicts label Feb 13, 2023
@mergify mergify bot added the ready label Feb 13, 2023
@Borda Borda enabled auto-merge (squash) February 14, 2023 01:05
@justusschock
Copy link
Member

Hey @marcocaccin

yes, metrics should always work with batches.
Help would be very welcome.

Borda and others added 3 commits February 14, 2023 09:51
Co-authored-by: Nicki Skafte Detlefsen <skaftenicki@gmail.com>
@Borda Borda merged commit a4bcb12 into Lightning-AI:master Feb 14, 2023
@justusschock
Copy link
Member

@marcocaccin we merged it for now. Feel free to do another PR to add batched-support and also maybe more tests :)

@ChristophReich1996
Copy link
Contributor

Hi @niberger, I just wanted to ask if you have verified that your PQ implementation matches the results of the COCO PQ implementation? Thanks :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Metrics] Panoptic Quality
6 participants