Merge branch 'docs/myst-parser' of https://github.com/PyTorchLightnin…

…g/metrics into docs/myst-parser

.codecov.yml

@@ -48,4 +48,3 @@ comment:
   require_changes: false
   behavior: default  # update if exists else create new
   # branches: *
-

.github/CONTRIBUTING.md

@@ -1,7 +1,7 @@
 # Contributing
 
 Welcome to the Torchmetrics community! We're building largest collection of native pytorch metrics, with the
-goal of reducing boilerplate and increasing reproducibility. 
+goal of reducing boilerplate and increasing reproducibility.
 
 ## Contribution Types
 
@@ -24,7 +24,7 @@ We are always looking for help implementing new features or fixing bugs.
 
 3. Submit a PR!
 
-_**Note**, even if you do not find the solution, sending a PR with a test covering the issue is a valid contribution and we can 
+_**Note**, even if you do not find the solution, sending a PR with a test covering the issue is a valid contribution and we can
 help you or finish it with you :]_
 
 ### New Features:
@@ -42,7 +42,7 @@ help you or finish it with you :]_
 
 ### Test cases:
 
-Want to keep Torchmetrics healthy? Love seeing those green tests? So do we! How to we keep it that way? 
+Want to keep Torchmetrics healthy? Love seeing those green tests? So do we! How to we keep it that way?
 We write tests! We value tests contribution even more than new features. One of the core values of torchmetrics
 is that our users can trust our metric implementation. We can only guarantee this if our metrics are well tested.
 
@@ -59,9 +59,9 @@ To build the documentation locally, simply execute the following commands from p
 ### Original code
 
 All added or edited code shall be the own original work of the particular contributor.
-If you use some third-party implementation, all such blocks/functions/modules shall be properly referred and if 
+If you use some third-party implementation, all such blocks/functions/modules shall be properly referred and if
 possible also agreed by code's author. For example - `This code is inspired from http://...`.
-In case you adding new dependencies, make sure that they are compatible with the actual Torchmetrics license 
+In case you adding new dependencies, make sure that they are compatible with the actual Torchmetrics license
 (ie. dependencies should be _at least_ as permissive as the Torchmetrics license).
 
 ### Coding Style

.github/ISSUE_TEMPLATE/bug_report.md

@@ -25,7 +25,7 @@ Steps to reproduce the behavior:
 
 #### Code sample
 
-<!-- Ideally attach a minimal code sample to reproduce the decried issue. 
+<!-- Ideally attach a minimal code sample to reproduce the decried issue.
 Minimal means having the shortest code but still preserving the bug. -->
 
 ### Expected behavior

.github/ISSUE_TEMPLATE/documentation.md

@@ -12,7 +12,7 @@ assignees: ''
 For typos and doc fixes, please go ahead and:
 
 1. Create an issue.
-2. Fix the typo.   
+2. Fix the typo.
 3. Submit a PR.
 
 Thanks!

.github/PULL_REQUEST_TEMPLATE.md

@@ -2,14 +2,14 @@
 
 - [ ] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
 - [ ] Did you read the [contributor guideline](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/.github/CONTRIBUTING.md), Pull Request section?
-- [ ] Did you make sure to update the docs?   
-- [ ] Did you write any new necessary tests?  
+- [ ] Did you make sure to update the docs?
+- [ ] Did you write any new necessary tests?
 
 ## What does this PR do?
 Fixes # (issue).
 
-## PR review    
-Anyone in the community is free to review the PR once the tests have passed.     
+## PR review
+Anyone in the community is free to review the PR once the tests have passed.
 If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
 
 ## Did you have fun?

.github/workflows/ci_test-conda.yml

@@ -52,7 +52,7 @@ jobs:
     - name: Update Environment
       run: |
         conda info
-        conda install pytorch=${{ matrix.pytorch-version }} cpuonly
+        conda install mkl pytorch=${{ matrix.pytorch-version }} cpuonly
         conda list
         pip --version
         pip install --requirement requirements.txt --upgrade-strategy only-if-needed --quiet

.github/workflows/ci_test-full.yml

@@ -20,6 +20,8 @@ jobs:
         python-version: [3.6, 3.8, 3.9]
         requires: ['minimal', 'latest']
         exclude:
+          - python-version: 3.6
+            requires: 'latest'
           - python-version: 3.9
             requires: 'minimal'
 

.github/workflows/docs-check.yml

@@ -97,4 +97,3 @@ jobs:
           name: docs-results-${{ runner.os }}-${{ matrix.python-version }}-${{ matrix.requires }}
           path: docs/build/html/
         if: success()
-

.github/workflows/docs-deploy.yml

@@ -56,4 +56,4 @@ jobs:
           FOLDER: docs/build/html # The folder the action should deploy.
           CLEAN: true # Automatically remove deleted files from the deploy branch
           TARGET_FOLDER: docs # If you'd like to push the contents of the deployment folder into a specific directory
-        if: success()
+        if: success()

.pre-commit-config.yaml

@@ -15,36 +15,39 @@
 default_language_version:
   python: python3.8
 
+ci:
+  autofix_prs: true
+  autoupdate_commit_msg: '[pre-commit.ci] pre-commit suggestions'
+
 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v2.3.0
+    rev: v3.4.0
     hooks:
-      - id: trailing-whitespace
       - id: end-of-file-fixer
+      - id: trailing-whitespace
 
-  - repo: local
+  - repo: https://github.com/PyCQA/isort
+    rev: 5.8.0
     hooks:
       - id: isort
         name: imports
         entry: isort
-        args: [--settings-path, ./pyproject.toml]
-        language: system
-        types: [python]
+        language: python
+        args: [--settings-path, "./pyproject.toml"]
 
+  - repo: https://github.com/pre-commit/mirrors-yapf
+    rev: v0.29.0
+    hooks:
       - id: yapf
         name: formatting
         entry: yapf
-        args: [ --parallel ]
-        language: system
-        types: [python]
+        args: [--parallel]
+        language: python
 
+  - repo: https://github.com/PyCQA/flake8
+    rev: 3.9.1
+    hooks:
       - id: flake8
         name: PEP8
         entry: flake8
-        language: system
-        types: [python]
-
-  - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v0.790
-    hooks:
-      - id: mypy
+        language: python

CHANGELOG.md

@@ -5,13 +5,46 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 
-## [0.3.0] - 2021-04-DD
+## [unReleased] - 2021-MM-DD
 
 ### Added
 
-- Added `BootStrapper` to easely calculate confidence intervals for metrics ([#101](https://github.com/PyTorchLightning/metrics/pull/101))
+
+### Changed
+
+
+### Deprecated
+
+
+### Removed
+
+
+### Fixed
+
+- Fixed auc calculation and add tests ([#197](https://github.com/PyTorchLightning/metrics/pull/197))
+
+
+- Fixed loading persisted metric states using `load_state_dict()` ([#202](https://github.com/PyTorchLightning/metrics/pull/202))
+
+
+## [0.3.1] - 2021-04-21
+
+- Cleaning remaining inconsistency and fix PL develop integration (
+    [#191](https://github.com/PyTorchLightning/metrics/pull/191),
+    [#192](https://github.com/PyTorchLightning/metrics/pull/192),
+    [#193](https://github.com/PyTorchLightning/metrics/pull/193),
+    [#194](https://github.com/PyTorchLightning/metrics/pull/194)
+)
+
+
+## [0.3.0] - 2021-04-20
+
+### Added
+
+- Added `BootStrapper` to easily calculate confidence intervals for metrics ([#101](https://github.com/PyTorchLightning/metrics/pull/101))
+- Added Binned metrics  ([#128](https://github.com/PyTorchLightning/metrics/pull/128))
 - Added metrics for Information Retrieval ([(PL^5032)](https://github.com/PyTorchLightning/pytorch-lightning/pull/5032)):
-    * Added `RetrievalMAP` ([#5032](https://github.com/PyTorchLightning/pytorch-lightning/pull/5032))
+    * Added `RetrievalMAP` ([PL^5032](https://github.com/PyTorchLightning/pytorch-lightning/pull/5032))
     * Added `RetrievalMRR` ([#119](https://github.com/PyTorchLightning/metrics/pull/119))
     * Added `RetrievalPrecision` ([#139](https://github.com/PyTorchLightning/metrics/pull/139))
     * Added `RetrievalRecall` ([#146](https://github.com/PyTorchLightning/metrics/pull/146))
@@ -23,7 +56,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
     * Added `PearsonCorrcoef` ([#157](https://github.com/PyTorchLightning/metrics/pull/157))
     * Added `SpearmanCorrcoef` ([#158](https://github.com/PyTorchLightning/metrics/pull/158))
     * Added `Hinge` ([#120](https://github.com/PyTorchLightning/metrics/pull/120))
-- Added Binned metrics  ([#128](https://github.com/PyTorchLightning/metrics/pull/128))
 - Added `average='micro'` as an option in AUROC for multilabel problems ([#110](https://github.com/PyTorchLightning/metrics/pull/110))
 - Added multilabel support to `ROC` metric ([#114](https://github.com/PyTorchLightning/metrics/pull/114))
 - Added testing for `half` precision ([#77](https://github.com/PyTorchLightning/metrics/pull/77),
@@ -34,6 +66,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Added `__getitem__` as metric arithmetic operation ([#142](https://github.com/PyTorchLightning/metrics/pull/142))
 - Added property `is_differentiable` to metrics and test for differentiability ([#154](https://github.com/PyTorchLightning/metrics/pull/154))
 - Added support for `average`, `ignore_index` and `mdmc_average` in `Accuracy` metric ([#166](https://github.com/PyTorchLightning/metrics/pull/166))
+- Added `postfix` arg to `MetricCollection` ([#188](https://github.com/PyTorchLightning/metrics/pull/188))
 
 ### Changed
 
@@ -54,9 +87,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
-- Fixed when `_stable_1d_sort` to work when n >= N ([PL^6177](https://github.com/PyTorchLightning/pytorch-lightning/pull/6177))
+- Fixed when `_stable_1d_sort` to work when `n>=N` ([PL^6177](https://github.com/PyTorchLightning/pytorch-lightning/pull/6177))
 - Fixed `_computed` attribute not being correctly reset ([#147](https://github.com/PyTorchLightning/metrics/pull/147))
-- Fixed to blau score ([#165](https://github.com/PyTorchLightning/metrics/pull/165))
+- Fixed to Blau score ([#165](https://github.com/PyTorchLightning/metrics/pull/165))
+- Fixed backwards compatibility for logging with older version of pytorch-lightning ([#182](https://github.com/PyTorchLightning/metrics/pull/182))
 
 
 ## [0.2.0] - 2021-03-12

LICENSE

@@ -186,7 +186,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright 2018-2021 William Falcon
+   Copyright 2020-2021 PytorchLightning team
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

Makefile

@@ -27,4 +27,3 @@ docs: clean
 env:
 	pip install -r requirements.txt
 	pip install -r requirements/test.txt
-

README.md

@@ -96,7 +96,7 @@ For the single GPU/CPU case:
 ``` python
 import torch
 # import our library
-import torchmetrics 
+import torchmetrics
 
 # initialize metric
 metric = torchmetrics.Accuracy()
@@ -109,14 +109,14 @@ for i in range(n_batches):
 
     # metric on current batch
     acc = metric(preds, target)
-    print(f"Accuracy on batch {i}: {acc}")    
+    print(f"Accuracy on batch {i}: {acc}")
 
 # metric on all batches using custom accumulation
 acc = metric.compute()
 print(f"Accuracy on all data: {acc}")
 ```
 
-Module metric usage remains the same when using multiple GPUs or multiple nodes. 
+Module metric usage remains the same when using multiple GPUs or multiple nodes.
 
 <details>
   <summary>Example using DDP</summary>
@@ -165,7 +165,7 @@ def metric_ddp(rank, world_size):
             # metric on current batch
             acc = metric(preds, target)
             if rank == 0:  # print only for rank 0
-                print(f"Accuracy on batch {i}: {acc}")    
+                print(f"Accuracy on batch {i}: {acc}")
 
         # metric on all batches and all accelerators using custom accumulation
         # accuracy is same across both accelerators
@@ -174,7 +174,7 @@ def metric_ddp(rank, world_size):
 
         # Reseting internal state such that metric ready for new data
         metric.reset()
-    
+
     # cleanup
     dist.destroy_process_group()
 
@@ -197,7 +197,7 @@ from torchmetrics import Metric
 class MyAccuracy(Metric):
     def __init__(self, dist_sync_on_step=False):
         # call `self.add_state`for every internal state that is needed for the metrics computations
-	# dist_reduce_fx indicates the function that should be used to reduce 
+	# dist_reduce_fx indicates the function that should be used to reduce
 	# state from multiple processes
 	super().__init__(dist_sync_on_step=dist_sync_on_step)
 
@@ -240,7 +240,7 @@ acc = torchmetrics.functional.accuracy(preds, target)
 * [AveragePrecision](https://torchmetrics.readthedocs.io/en/latest/references/modules.html#averageprecision)
 * [AUC](https://torchmetrics.readthedocs.io/en/latest/references/modules.html#auc)
 * [AUROC](https://torchmetrics.readthedocs.io/en/latest/references/modules.html#auroc)
-* [F1](https://torchmetrics.readthedocs.io/en/latest/references/modules.html#f1) 
+* [F1](https://torchmetrics.readthedocs.io/en/latest/references/modules.html#f1)
 * [Hamming Distance](https://torchmetrics.readthedocs.io/en/latest/references/modules.html#hamming-distance)
 * [ROC](https://torchmetrics.readthedocs.io/en/latest/references/modules.html#roc)
 * [ExplainedVariance](https://torchmetrics.readthedocs.io/en/latest/references/modules.html#explainedvariance)
@@ -252,19 +252,19 @@ acc = torchmetrics.functional.accuracy(preds, target)
 And many more!
 
 ## Contribute!
-The lightning + torchmetric team is hard at work adding even more metrics. 
+The lightning + torchmetric team is hard at work adding even more metrics.
 But we're looking for incredible contributors like you to submit new metrics
 and improve existing ones!
 
-Join our [Slack](https://join.slack.com/t/pytorch-lightning/shared_invite/zt-f6bl2l0l-JYMK3tbAgAmGRrlNr00f1A) 
+Join our [Slack](https://join.slack.com/t/pytorch-lightning/shared_invite/zt-f6bl2l0l-JYMK3tbAgAmGRrlNr00f1A)
 to get help becoming a contributor!
 
 ## Community
 For help or questions, join our huge community on [Slack](https://join.slack.com/t/pytorch-lightning/shared_invite/zt-f6bl2l0l-JYMK3tbAgAmGRrlNr00f1A)!
 
 ## Citations
-We’re excited to continue the strong legacy of open source software and have been inspired over the years by 
-Caffe, Theano, Keras, PyTorch, torchbearer, ignite, sklearn and fast.ai. When/if a paper is written about this, 
+We’re excited to continue the strong legacy of open source software and have been inspired over the years by
+Caffe, Theano, Keras, PyTorch, torchbearer, ignite, sklearn and fast.ai. When/if a paper is written about this,
 we’ll be happy to cite these frameworks and the corresponding authors.
 
 ## License

docs/.build_docs.sh

@@ -1,2 +1,2 @@
 make clean
-make html --debug --jobs $(nproc)
+make html --debug --jobs $(nproc)

docs/Makefile

@@ -16,4 +16,4 @@ help:
 # Catch-all target: route all unknown targets to Sphinx using the new
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
 %: Makefile
-	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/source/conf.py

@@ -28,15 +28,10 @@
 FOLDER_GENERATED = 'generated'
 SPHINX_MOCK_REQUIREMENTS = int(os.environ.get("SPHINX_MOCK_REQUIREMENTS", True))
 
-try:
-    from torchmetrics import __about__ as about
-except ImportError:
-    # alternative https://stackoverflow.com/a/67692/4521646
-    spec = spec_from_file_location(
-        "torchmetrics/__about__.py", os.path.join(_PATH_ROOT, "torchmetrics", "__about__.py")
-    )
-    about = module_from_spec(spec)
-    spec.loader.exec_module(about)
+# alternative https://stackoverflow.com/a/67692/4521646
+spec = spec_from_file_location("torchmetrics/__about__.py", os.path.join(_PATH_ROOT, "torchmetrics", "__about__.py"))
+about = module_from_spec(spec)
+spec.loader.exec_module(about)
 
 html_favicon = '_static/images/icon.svg'