From 15f31e26cc3604901a1e8df9555ed373926e988f Mon Sep 17 00:00:00 2001 From: Jirka Date: Mon, 25 Oct 2021 14:10:28 +0200 Subject: [PATCH 1/5] prepare 0.6 RC --- CHANGELOG.md | 64 ++++--------------- .../classification/test_average_precision.py | 2 +- torchmetrics/__about__.py | 2 +- torchmetrics/classification/__init__.py | 2 +- ...{average_precision.py => avg_precision.py} | 0 5 files changed, 17 insertions(+), 53 deletions(-) rename torchmetrics/classification/{average_precision.py => avg_precision.py} (100%) diff --git a/CHANGELOG.md b/CHANGELOG.md index 49ebaf56417..14bf326b303 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,67 +6,38 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 **Note: we move fast, but still we preserve 0.1 version (one feature release) back compatibility.** -## [unReleased] - 2021-MM-DD +## [0.6.0] - 2021-10-DD ### Added -- Added Learned Perceptual Image Patch Similarity (LPIPS) ([#431](https://github.com/PyTorchLightning/metrics/issues/431)) - - -- Added Tweedie Deviance Score ([#499](https://github.com/PyTorchLightning/metrics/pull/499)) - - +- Added audio metrics: + - Perceptual Evaluation of Speech Quality (PESQ) ([#353](https://github.com/PyTorchLightning/metrics/issues/353)) + - Short Term Objective Intelligibility (STOI) ([#353](https://github.com/PyTorchLightning/metrics/issues/353)) +- Added Information retrieval metrics: + - `RetrievalRPrecision` ([#577](https://github.com/PyTorchLightning/metrics/pull/577/)) + - `RetrievalHitRate` ([#576](https://github.com/PyTorchLightning/metrics/pull/576)) +- Added NLP metrics: + - `SacreBLEUScore` ([#546](https://github.com/PyTorchLightning/metrics/pull/546)) + - `CharErrorRate` ([#575](https://github.com/PyTorchLightning/metrics/pull/575)) +- Added other metrics: + - Tweedie Deviance Score ([#499](https://github.com/PyTorchLightning/metrics/pull/499)) + - Learned Perceptual Image Patch Similarity (LPIPS) ([#431](https://github.com/PyTorchLightning/metrics/issues/431)) - Added support for float targets in `nDCG` metric ([#437](https://github.com/PyTorchLightning/metrics/pull/437)) - - -- Added `average` argument to `AveragePrecision` metric for reducing multilabel and multiclass problems ([#477](https://github.com/PyTorchLightning/metrics/pull/477)) - - -- Added Perceptual Evaluation of Speech Quality (PESQ) ([#353](https://github.com/PyTorchLightning/metrics/issues/353)) - - +- Added `average` argument to `AveragePrecision` metric for reducing multi-label and multi-class problems ([#477](https://github.com/PyTorchLightning/metrics/pull/477)) - Added `MultioutputWrapper` ([#510](https://github.com/PyTorchLightning/metrics/pull/510)) - - - Added metric sweeping `higher_is_better` as constant attribute ([#544](https://github.com/PyTorchLightning/metrics/pull/544)) - - -- Added `SacreBLEUScore` metric to text package ([#546](https://github.com/PyTorchLightning/metrics/pull/546)) - - - Added simple aggregation metrics: `SumMetric`, `MeanMetric`, `CatMetric`, `MinMetric`, `MaxMetric` ([#506](https://github.com/PyTorchLightning/metrics/pull/506)) - - - Added pairwise submodule with metrics ([#553](https://github.com/PyTorchLightning/metrics/pull/553)) - `pairwise_cosine_similarity` - `pairwise_euclidean_distance` - `pairwise_linear_similarity` - `pairwise_manhatten_distance` - -- Added Short Term Objective Intelligibility (`STOI`) ([#353](https://github.com/PyTorchLightning/metrics/issues/353)) - - -- Added `RetrievalRPrecision` metric to retrieval package ([#577](https://github.com/PyTorchLightning/metrics/pull/577/)) - - -- Added `RetrievalHitRate` metric to retrieval package ([#576](https://github.com/PyTorchLightning/metrics/pull/576)) - - -- Added `CharErrorRate` metric to text package ([#575](https://github.com/PyTorchLightning/metrics/pull/575)) - - ### Changed - `AveragePrecision` will now as default output the `macro` average for multilabel and multiclass problems ([#477](https://github.com/PyTorchLightning/metrics/pull/477)) - - - `half`, `double`, `float` will no longer change the dtype of the metric states. Use `metric.set_dtype` instead ([#493](https://github.com/PyTorchLightning/metrics/pull/493)) - - - Renamed `AverageMeter` to `MeanMetric` ([#506](https://github.com/PyTorchLightning/metrics/pull/506)) - - - Changed `is_differentiable` from property to a constant attribute ([#551](https://github.com/PyTorchLightning/metrics/pull/551)) ### Deprecated @@ -77,18 +48,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Removed `dtype` property ([#493](https://github.com/PyTorchLightning/metrics/pull/493)) - ### Fixed - Fixed bug in `F1` with `average='macro'` and `ignore_index!=None` ([#495](https://github.com/PyTorchLightning/metrics/pull/495)) - - - Fixed bug in `pit` by using the returned first result to initialize device and type ([#533](https://github.com/PyTorchLightning/metrics/pull/533)) - - - Fixed `SSIM` metric using too much memory ([#539](https://github.com/PyTorchLightning/metrics/pull/539)) - - - Fixed bug where `device` property was not properly update when metric was a child of a module ([#542](https://github.com/PyTorchLightning/metrics/pull/542)) ## [0.5.1] - 2021-08-30 diff --git a/tests/classification/test_average_precision.py b/tests/classification/test_average_precision.py index 5c65a2256cf..557fb92b154 100644 --- a/tests/classification/test_average_precision.py +++ b/tests/classification/test_average_precision.py @@ -24,7 +24,7 @@ from tests.classification.inputs import _input_multilabel from tests.helpers import seed_all from tests.helpers.testers import NUM_CLASSES, MetricTester -from torchmetrics.classification.average_precision import AveragePrecision +from torchmetrics.classification.avg_precision import AveragePrecision from torchmetrics.functional import average_precision seed_all(42) diff --git a/torchmetrics/__about__.py b/torchmetrics/__about__.py index ea1d5dcd3c0..334914962d1 100644 --- a/torchmetrics/__about__.py +++ b/torchmetrics/__about__.py @@ -1,4 +1,4 @@ -__version__ = "0.6.0dev" +__version__ = "0.6.0rc0" __author__ = "PyTorchLightning et al." __author_email__ = "name@pytorchlightning.ai" __license__ = "Apache-2.0" diff --git a/torchmetrics/classification/__init__.py b/torchmetrics/classification/__init__.py index 35476172b06..0ed2d3d8d8b 100644 --- a/torchmetrics/classification/__init__.py +++ b/torchmetrics/classification/__init__.py @@ -14,7 +14,7 @@ from torchmetrics.classification.accuracy import Accuracy # noqa: F401 from torchmetrics.classification.auc import AUC # noqa: F401 from torchmetrics.classification.auroc import AUROC # noqa: F401 -from torchmetrics.classification.average_precision import AveragePrecision # noqa: F401 +from torchmetrics.classification.avg_precision import AveragePrecision # noqa: F401 from torchmetrics.classification.binned_precision_recall import BinnedAveragePrecision # noqa: F401 from torchmetrics.classification.binned_precision_recall import BinnedPrecisionRecallCurve # noqa: F401 from torchmetrics.classification.binned_precision_recall import BinnedRecallAtFixedPrecision # noqa: F401 diff --git a/torchmetrics/classification/average_precision.py b/torchmetrics/classification/avg_precision.py similarity index 100% rename from torchmetrics/classification/average_precision.py rename to torchmetrics/classification/avg_precision.py From ea0fc45720432ab1f550855a10f6a8cee6218b0d Mon Sep 17 00:00:00 2001 From: SkafteNicki Date: Wed, 27 Oct 2021 08:52:54 +0200 Subject: [PATCH 2/5] update --- torchmetrics/audio/snr.py | 1 + torchmetrics/classification/accuracy.py | 1 + torchmetrics/classification/auroc.py | 1 + torchmetrics/classification/calibration_error.py | 1 + torchmetrics/classification/cohen_kappa.py | 1 + torchmetrics/classification/f_beta.py | 1 + torchmetrics/classification/hamming_distance.py | 1 + torchmetrics/classification/hinge.py | 1 + torchmetrics/classification/iou.py | 1 + torchmetrics/classification/kl_divergence.py | 1 + torchmetrics/classification/matthews_corrcoef.py | 1 + torchmetrics/classification/precision_recall.py | 2 ++ torchmetrics/classification/specificity.py | 1 + torchmetrics/image/fid.py | 1 + torchmetrics/image/inception.py | 1 + torchmetrics/image/kid.py | 1 + torchmetrics/image/lpip_similarity.py | 1 + torchmetrics/image/psnr.py | 1 + torchmetrics/image/ssim.py | 1 + torchmetrics/regression/cosine_similarity.py | 1 + torchmetrics/regression/explained_variance.py | 1 + torchmetrics/regression/mean_absolute_error.py | 1 + torchmetrics/regression/mean_absolute_percentage_error.py | 1 + torchmetrics/regression/mean_squared_error.py | 1 + torchmetrics/regression/mean_squared_log_error.py | 1 + torchmetrics/regression/pearson.py | 1 + torchmetrics/regression/r2.py | 1 + torchmetrics/regression/spearman.py | 1 + .../regression/symmetric_mean_absolute_percentage_error.py | 1 + torchmetrics/regression/tweedie_deviance.py | 1 + 30 files changed, 31 insertions(+) diff --git a/torchmetrics/audio/snr.py b/torchmetrics/audio/snr.py index 3afdb93b884..46b1f217bfb 100644 --- a/torchmetrics/audio/snr.py +++ b/torchmetrics/audio/snr.py @@ -71,6 +71,7 @@ class SNR(Metric): """ is_differentiable = True + higher_is_better = True sum_snr: Tensor total: Tensor diff --git a/torchmetrics/classification/accuracy.py b/torchmetrics/classification/accuracy.py index 9eb30721f13..b6dec6f8036 100644 --- a/torchmetrics/classification/accuracy.py +++ b/torchmetrics/classification/accuracy.py @@ -165,6 +165,7 @@ class Accuracy(StatScores): """ is_differentiable = False + higher_is_better = True correct: Tensor total: Tensor diff --git a/torchmetrics/classification/auroc.py b/torchmetrics/classification/auroc.py index 37903542a5a..ffa98e33715 100644 --- a/torchmetrics/classification/auroc.py +++ b/torchmetrics/classification/auroc.py @@ -100,6 +100,7 @@ class AUROC(Metric): """ is_differentiable = False + higher_is_better = True preds: List[Tensor] target: List[Tensor] diff --git a/torchmetrics/classification/calibration_error.py b/torchmetrics/classification/calibration_error.py index c9c8f9062c5..6ece374a690 100644 --- a/torchmetrics/classification/calibration_error.py +++ b/torchmetrics/classification/calibration_error.py @@ -60,6 +60,7 @@ class CalibrationError(Metric): default: None (which selects the entire world) """ DISTANCES = {"l1", "l2", "max"} + higher_is_better = False confidences: List[Tensor] accuracies: List[Tensor] diff --git a/torchmetrics/classification/cohen_kappa.py b/torchmetrics/classification/cohen_kappa.py index 213497a9734..6ceb391bf3a 100644 --- a/torchmetrics/classification/cohen_kappa.py +++ b/torchmetrics/classification/cohen_kappa.py @@ -78,6 +78,7 @@ class labels. """ is_differentiable = False + higher_is_better = True confmat: Tensor def __init__( diff --git a/torchmetrics/classification/f_beta.py b/torchmetrics/classification/f_beta.py index 687eb3226e3..ca9e78c48b6 100644 --- a/torchmetrics/classification/f_beta.py +++ b/torchmetrics/classification/f_beta.py @@ -270,6 +270,7 @@ class F1(FBeta): """ is_differentiable = False + higher_is_better = True def __init__( self, diff --git a/torchmetrics/classification/hamming_distance.py b/torchmetrics/classification/hamming_distance.py index b60e98c5f6a..4da5c601220 100644 --- a/torchmetrics/classification/hamming_distance.py +++ b/torchmetrics/classification/hamming_distance.py @@ -68,6 +68,7 @@ class HammingDistance(Metric): """ is_differentiable = False + higher_is_better = False correct: Tensor total: Tensor diff --git a/torchmetrics/classification/hinge.py b/torchmetrics/classification/hinge.py index 5ea86287ffb..a1e7dca84a3 100644 --- a/torchmetrics/classification/hinge.py +++ b/torchmetrics/classification/hinge.py @@ -85,6 +85,7 @@ class Hinge(Metric): """ is_differentiable = True + higher_is_better = False measure: Tensor total: Tensor diff --git a/torchmetrics/classification/iou.py b/torchmetrics/classification/iou.py index 20db9a42600..789d57e8523 100644 --- a/torchmetrics/classification/iou.py +++ b/torchmetrics/classification/iou.py @@ -78,6 +78,7 @@ class IoU(ConfusionMatrix): """ is_differentiable = False + higher_is_better = True def __init__( self, diff --git a/torchmetrics/classification/kl_divergence.py b/torchmetrics/classification/kl_divergence.py index 40aa9f8b6b7..cf85ee79533 100644 --- a/torchmetrics/classification/kl_divergence.py +++ b/torchmetrics/classification/kl_divergence.py @@ -62,6 +62,7 @@ class KLDivergence(Metric): """ is_differentiable = True + higher_is_better = False # TODO: canot be used because if scripting # measures: Union[List[Tensor], Tensor] total: Tensor diff --git a/torchmetrics/classification/matthews_corrcoef.py b/torchmetrics/classification/matthews_corrcoef.py index 096ce46a2f0..c01ac0f1f3c 100644 --- a/torchmetrics/classification/matthews_corrcoef.py +++ b/torchmetrics/classification/matthews_corrcoef.py @@ -74,6 +74,7 @@ class MatthewsCorrcoef(Metric): """ is_differentiable = False + higher_is_better = True confmat: Tensor def __init__( diff --git a/torchmetrics/classification/precision_recall.py b/torchmetrics/classification/precision_recall.py index 206ce77da9e..627fde14a4b 100644 --- a/torchmetrics/classification/precision_recall.py +++ b/torchmetrics/classification/precision_recall.py @@ -121,6 +121,7 @@ class Precision(StatScores): """ is_differentiable = False + higher_is_better = True def __init__( self, @@ -271,6 +272,7 @@ class Recall(StatScores): """ is_differentiable = False + higher_is_better = True def __init__( self, diff --git a/torchmetrics/classification/specificity.py b/torchmetrics/classification/specificity.py index ee0e6c88332..c21ccb8211a 100644 --- a/torchmetrics/classification/specificity.py +++ b/torchmetrics/classification/specificity.py @@ -122,6 +122,7 @@ class Specificity(StatScores): """ is_differentiable = False + higher_is_better = True def __init__( self, diff --git a/torchmetrics/image/fid.py b/torchmetrics/image/fid.py index a6dadba6fe1..3b891ec3145 100644 --- a/torchmetrics/image/fid.py +++ b/torchmetrics/image/fid.py @@ -204,6 +204,7 @@ class FID(Metric): """ real_features: List[Tensor] fake_features: List[Tensor] + higher_is_better = False def __init__( self, diff --git a/torchmetrics/image/inception.py b/torchmetrics/image/inception.py index 3d70d90939c..d2c4504c1d7 100644 --- a/torchmetrics/image/inception.py +++ b/torchmetrics/image/inception.py @@ -101,6 +101,7 @@ class IS(Metric): """ features: List + higher_is_better = True def __init__( self, diff --git a/torchmetrics/image/kid.py b/torchmetrics/image/kid.py index ccdc362e5b2..d860a2706b8 100644 --- a/torchmetrics/image/kid.py +++ b/torchmetrics/image/kid.py @@ -164,6 +164,7 @@ class KID(Metric): """ real_features: List[Tensor] fake_features: List[Tensor] + higher_is_better = False def __init__( self, diff --git a/torchmetrics/image/lpip_similarity.py b/torchmetrics/image/lpip_similarity.py index 7f26e358d26..2b23b8570d0 100644 --- a/torchmetrics/image/lpip_similarity.py +++ b/torchmetrics/image/lpip_similarity.py @@ -88,6 +88,7 @@ class LPIPS(Metric): """ is_differentiable = True + higher_is_better = False real_features: List[Tensor] fake_features: List[Tensor] diff --git a/torchmetrics/image/psnr.py b/torchmetrics/image/psnr.py index 7dff5214d72..948675980a3 100644 --- a/torchmetrics/image/psnr.py +++ b/torchmetrics/image/psnr.py @@ -69,6 +69,7 @@ class PSNR(Metric): """ min_target: Tensor max_target: Tensor + higher_is_better = False def __init__( self, diff --git a/torchmetrics/image/ssim.py b/torchmetrics/image/ssim.py index 17f05e2e9b5..e2d26855d65 100644 --- a/torchmetrics/image/ssim.py +++ b/torchmetrics/image/ssim.py @@ -52,6 +52,7 @@ class SSIM(Metric): preds: List[Tensor] target: List[Tensor] + higher_is_better = True def __init__( self, diff --git a/torchmetrics/regression/cosine_similarity.py b/torchmetrics/regression/cosine_similarity.py index 7721b55013b..42c1e4d9095 100644 --- a/torchmetrics/regression/cosine_similarity.py +++ b/torchmetrics/regression/cosine_similarity.py @@ -62,6 +62,7 @@ class CosineSimilarity(Metric): """ is_differentiable = True + higher_is_better = True preds: List[Tensor] target: List[Tensor] diff --git a/torchmetrics/regression/explained_variance.py b/torchmetrics/regression/explained_variance.py index e11b637dcfa..939a6884734 100644 --- a/torchmetrics/regression/explained_variance.py +++ b/torchmetrics/regression/explained_variance.py @@ -78,6 +78,7 @@ class ExplainedVariance(Metric): """ is_differentiable = True + higher_is_better = True n_obs: Tensor sum_error: Tensor sum_squared_error: Tensor diff --git a/torchmetrics/regression/mean_absolute_error.py b/torchmetrics/regression/mean_absolute_error.py index 7468db589d9..0f84270c9ad 100644 --- a/torchmetrics/regression/mean_absolute_error.py +++ b/torchmetrics/regression/mean_absolute_error.py @@ -49,6 +49,7 @@ class MeanAbsoluteError(Metric): tensor(0.5000) """ is_differentiable = True + higher_is_better = False sum_abs_error: Tensor total: Tensor diff --git a/torchmetrics/regression/mean_absolute_percentage_error.py b/torchmetrics/regression/mean_absolute_percentage_error.py index cab503ab493..98512c4d578 100644 --- a/torchmetrics/regression/mean_absolute_percentage_error.py +++ b/torchmetrics/regression/mean_absolute_percentage_error.py @@ -58,6 +58,7 @@ class MeanAbsolutePercentageError(Metric): """ is_differentiable = True + higher_is_better = False sum_abs_per_error: Tensor total: Tensor diff --git a/torchmetrics/regression/mean_squared_error.py b/torchmetrics/regression/mean_squared_error.py index 791f736f3d9..34b7d0274b0 100644 --- a/torchmetrics/regression/mean_squared_error.py +++ b/torchmetrics/regression/mean_squared_error.py @@ -52,6 +52,7 @@ class MeanSquaredError(Metric): """ is_differentiable = True + higher_is_better = False sum_squared_error: Tensor total: Tensor diff --git a/torchmetrics/regression/mean_squared_log_error.py b/torchmetrics/regression/mean_squared_log_error.py index c42f542e677..eefbfc71b75 100644 --- a/torchmetrics/regression/mean_squared_log_error.py +++ b/torchmetrics/regression/mean_squared_log_error.py @@ -53,6 +53,7 @@ class MeanSquaredLogError(Metric): """ is_differentiable = True + higher_is_better = False sum_squared_log_error: Tensor total: Tensor diff --git a/torchmetrics/regression/pearson.py b/torchmetrics/regression/pearson.py index 830ee6a3c1e..60f66be2562 100644 --- a/torchmetrics/regression/pearson.py +++ b/torchmetrics/regression/pearson.py @@ -86,6 +86,7 @@ class PearsonCorrcoef(Metric): """ is_differentiable = True + higher_is_better = None # both -1 and 1 are optimal preds: List[Tensor] target: List[Tensor] mean_x: Tensor diff --git a/torchmetrics/regression/r2.py b/torchmetrics/regression/r2.py index 0453d35e28d..5d67889b71e 100644 --- a/torchmetrics/regression/r2.py +++ b/torchmetrics/regression/r2.py @@ -88,6 +88,7 @@ class R2Score(Metric): """ is_differentiable = True + higher_is_better = True sum_squared_error: Tensor sum_error: Tensor residual: Tensor diff --git a/torchmetrics/regression/spearman.py b/torchmetrics/regression/spearman.py index 75325ef0e4a..0d0c5a6ba6b 100644 --- a/torchmetrics/regression/spearman.py +++ b/torchmetrics/regression/spearman.py @@ -54,6 +54,7 @@ class SpearmanCorrcoef(Metric): """ is_differentiable = False + higher_is_better = True preds: List[Tensor] target: List[Tensor] diff --git a/torchmetrics/regression/symmetric_mean_absolute_percentage_error.py b/torchmetrics/regression/symmetric_mean_absolute_percentage_error.py index dc26df15a05..05cc46dad39 100644 --- a/torchmetrics/regression/symmetric_mean_absolute_percentage_error.py +++ b/torchmetrics/regression/symmetric_mean_absolute_percentage_error.py @@ -55,6 +55,7 @@ class SymmetricMeanAbsolutePercentageError(Metric): tensor(0.2290) """ is_differentiable = True + higher_is_better = False sum_abs_per_error: Tensor total: Tensor diff --git a/torchmetrics/regression/tweedie_deviance.py b/torchmetrics/regression/tweedie_deviance.py index ed1cf4429d9..4e5558b1338 100644 --- a/torchmetrics/regression/tweedie_deviance.py +++ b/torchmetrics/regression/tweedie_deviance.py @@ -75,6 +75,7 @@ class TweedieDevianceScore(Metric): """ is_differentiable = True + higher_is_better = None # both -1 and 1 are optimal sum_deviance_score: Tensor num_observations: Tensor From 4db31811131d453b79b9a12272e8a52ebcedcb70 Mon Sep 17 00:00:00 2001 From: SkafteNicki Date: Wed, 27 Oct 2021 09:03:46 +0200 Subject: [PATCH 3/5] changelog --- CHANGELOG.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 14bf326b303..66968f7d88b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,11 +21,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - `CharErrorRate` ([#575](https://github.com/PyTorchLightning/metrics/pull/575)) - Added other metrics: - Tweedie Deviance Score ([#499](https://github.com/PyTorchLightning/metrics/pull/499)) - - Learned Perceptual Image Patch Similarity (LPIPS) ([#431](https://github.com/PyTorchLightning/metrics/issues/431)) + - Learned Perceptual Image Patch Similarity (LPIPS) ([#431](https://github.com/PyTorchLightning/metrics/pull/431)) - Added support for float targets in `nDCG` metric ([#437](https://github.com/PyTorchLightning/metrics/pull/437)) - Added `average` argument to `AveragePrecision` metric for reducing multi-label and multi-class problems ([#477](https://github.com/PyTorchLightning/metrics/pull/477)) - Added `MultioutputWrapper` ([#510](https://github.com/PyTorchLightning/metrics/pull/510)) -- Added metric sweeping `higher_is_better` as constant attribute ([#544](https://github.com/PyTorchLightning/metrics/pull/544)) +- Added metric sweeping: + - `higher_is_better` as constant attribute ([#544](https://github.com/PyTorchLightning/metrics/pull/544)) + - `higher_is_better` to rest of codebase ([#584](https://github.com/PyTorchLightning/metrics/pull/584)) - Added simple aggregation metrics: `SumMetric`, `MeanMetric`, `CatMetric`, `MinMetric`, `MaxMetric` ([#506](https://github.com/PyTorchLightning/metrics/pull/506)) - Added pairwise submodule with metrics ([#553](https://github.com/PyTorchLightning/metrics/pull/553)) - `pairwise_cosine_similarity` From 49ebb6aa34581e1ba9e330020798d41ae52e12d0 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 27 Oct 2021 07:04:52 +0000 Subject: [PATCH 4/5] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 66968f7d88b..b3807d38f3a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,7 +25,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Added support for float targets in `nDCG` metric ([#437](https://github.com/PyTorchLightning/metrics/pull/437)) - Added `average` argument to `AveragePrecision` metric for reducing multi-label and multi-class problems ([#477](https://github.com/PyTorchLightning/metrics/pull/477)) - Added `MultioutputWrapper` ([#510](https://github.com/PyTorchLightning/metrics/pull/510)) -- Added metric sweeping: +- Added metric sweeping: - `higher_is_better` as constant attribute ([#544](https://github.com/PyTorchLightning/metrics/pull/544)) - `higher_is_better` to rest of codebase ([#584](https://github.com/PyTorchLightning/metrics/pull/584)) - Added simple aggregation metrics: `SumMetric`, `MeanMetric`, `CatMetric`, `MinMetric`, `MaxMetric` ([#506](https://github.com/PyTorchLightning/metrics/pull/506)) From e55413f66abcd686763c7f5fb3e0577ccd1fd063 Mon Sep 17 00:00:00 2001 From: Jirka Borovec Date: Wed, 27 Oct 2021 10:59:11 +0200 Subject: [PATCH 5/5] todo note Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com> --- torchmetrics/regression/tweedie_deviance.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/torchmetrics/regression/tweedie_deviance.py b/torchmetrics/regression/tweedie_deviance.py index 4e5558b1338..bf1d37c5f90 100644 --- a/torchmetrics/regression/tweedie_deviance.py +++ b/torchmetrics/regression/tweedie_deviance.py @@ -75,7 +75,7 @@ class TweedieDevianceScore(Metric): """ is_differentiable = True - higher_is_better = None # both -1 and 1 are optimal + higher_is_better = None # TODO: both -1 and 1 are optimal sum_deviance_score: Tensor num_observations: Tensor