Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[skip ci] add doctest for regression metrics #2324

Merged
merged 3 commits into from
Nov 13, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 15 additions & 9 deletions ignite/contrib/metrics/regression/canberra_metric.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,18 +37,24 @@ class CanberraMetric(_BaseRegression):
.. _`Botchkarev 2018`:
https://arxiv.org/ftp/arxiv/papers/1809/1809.03006.pdf

.. testcode::
Examples:
To use with ``Engine`` and ``process_function``, simply attach the metric instance to the engine.
The output of the engine's ``process_function`` needs to be in format of
``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y, ...}``.

metric = CanberraMetric()
metric.attach(default_evaluator, 'canberra')
y_pred = torch.Tensor([[3.8], [9.9], [-5.4], [2.1]])
y_true = y_pred * 1.5
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['canberra'])
.. testcode::

.. testoutput::
metric = CanberraMetric()
metric.attach(default_evaluator, 'canberra')
y_pred = torch.Tensor([[3.8], [9.9], [-5.4], [2.1]])
y_true = y_pred * 1.5
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['canberra'])

.. testoutput::

0.8000...

0.8000...
.. versionchanged:: 0.4.3

- Fixed implementation: ``abs`` in denominator.
Expand Down
24 changes: 15 additions & 9 deletions ignite/contrib/metrics/regression/fractional_absolute_error.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,18 +34,24 @@ class FractionalAbsoluteError(_BaseRegression):
metric's device to be the same as your ``update`` arguments ensures the ``update`` method is
non-blocking. By default, CPU.

.. testcode::
Examples:
To use with ``Engine`` and ``process_function``, simply attach the metric instance to the engine.
The output of the engine's ``process_function`` needs to be in format of
``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y, ...}``.

metric = FractionalAbsoluteError()
metric.attach(default_evaluator, 'fractional_abs_error')
y_pred = torch.Tensor([[3.8], [9.9], [-5.4], [2.1]])
y_true = y_pred * 0.8
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['fractional_abs_error'])
.. testcode::

.. testoutput::
metric = FractionalAbsoluteError()
metric.attach(default_evaluator, 'fractional_abs_error')
y_pred = torch.Tensor([[3.8], [9.9], [-5.4], [2.1]])
y_true = y_pred * 0.8
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['fractional_abs_error'])

.. testoutput::

0.2222...

0.2222...
.. versionchanged:: 0.4.5
- Works with DDP.
"""
Expand Down
23 changes: 14 additions & 9 deletions ignite/contrib/metrics/regression/fractional_bias.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,18 +34,23 @@ class FractionalBias(_BaseRegression):
metric's device to be the same as your ``update`` arguments ensures the ``update`` method is
non-blocking. By default, CPU.

.. testcode::
Examples:
To use with ``Engine`` and ``process_function``, simply attach the metric instance to the engine.
The output of the engine's ``process_function`` needs to be in format of
``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y, ...}``.

metric = FractionalBias()
metric.attach(default_evaluator, 'fractional_bias')
y_pred = torch.Tensor([[3.8], [9.9], [5.4], [2.1]])
y_true = y_pred * 1.5
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['fractional_bias'])
.. testcode::

.. testoutput::
metric = FractionalBias()
metric.attach(default_evaluator, 'fractional_bias')
y_pred = torch.Tensor([[3.8], [9.9], [5.4], [2.1]])
y_true = y_pred * 1.5
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['fractional_bias'])

0.4000...
.. testoutput::

0.4000...

.. versionchanged:: 0.4.5
- Works with DDP.
Expand Down
24 changes: 15 additions & 9 deletions ignite/contrib/metrics/regression/geometric_mean_absolute_error.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,18 +34,24 @@ class GeometricMeanAbsoluteError(_BaseRegression):
metric's device to be the same as your ``update`` arguments ensures the ``update`` method is
non-blocking. By default, CPU.

.. testcode::
Examples:
To use with ``Engine`` and ``process_function``, simply attach the metric instance to the engine.
The output of the engine's ``process_function`` needs to be in format of
``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y, ...}``.

metric = GeometricMeanAbsoluteError()
metric.attach(default_evaluator, 'gmae')
y_pred = torch.Tensor([[3.8], [9.9], [-5.4], [2.1]])
y_true = y_pred * 1.5
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['gmae'])
.. testcode::

.. testoutput::
metric = GeometricMeanAbsoluteError()
metric.attach(default_evaluator, 'gmae')
y_pred = torch.Tensor([[3.8], [9.9], [-5.4], [2.1]])
y_true = y_pred * 1.5
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['gmae'])

.. testoutput::

2.2723...

2.2723...
.. versionchanged:: 0.4.5
- Works with DDP.
"""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,24 @@ class GeometricMeanRelativeAbsoluteError(_BaseRegression):
device: specifies which device updates are accumulated on. Setting the
metric's device to be the same as your ``update`` arguments ensures the ``update`` method is
non-blocking. By default, CPU.

Examples:
To use with ``Engine`` and ``process_function``, simply attach the metric instance to the engine.
The output of the engine's ``process_function`` needs to be in format of
``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y, ...}``.

.. testcode::

metric = GeometricMeanRelativeAbsoluteError()
metric.attach(default_evaluator, 'gmare')
y_true = torch.Tensor([0, 1, 2, 3, 4, 5])
y_pred = y_true * 0.75
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['gmare'])

.. testoutput::

0.0...
"""

@reinit__is_reduced
Expand Down
18 changes: 18 additions & 0 deletions ignite/contrib/metrics/regression/manhattan_distance.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,24 @@ class ManhattanDistance(_BaseRegression):
metric's device to be the same as your ``update`` arguments ensures the ``update`` method is
non-blocking. By default, CPU.

Examples:
To use with ``Engine`` and ``process_function``, simply attach the metric instance to the engine.
The output of the engine's ``process_function`` needs to be in format of
``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y, ...}``.

.. testcode::

metric = ManhattanDistance()
metric.attach(default_evaluator, 'manhattan')
y_true = torch.Tensor([0, 1, 2, 3, 4, 5])
y_pred = y_true * 0.75
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['manhattan'])

.. testoutput::

3.75...

.. versionchanged:: 0.4.3

- Fixed sklearn compatibility.
Expand Down
18 changes: 18 additions & 0 deletions ignite/contrib/metrics/regression/maximum_absolute_error.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,24 @@ class MaximumAbsoluteError(_BaseRegression):
metric's device to be the same as your ``update`` arguments ensures the ``update`` method is
non-blocking. By default, CPU.

Examples:
To use with ``Engine`` and ``process_function``, simply attach the metric instance to the engine.
The output of the engine's ``process_function`` needs to be in format of
``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y, ...}``.

.. testcode::

metric = MaximumAbsoluteError()
metric.attach(default_evaluator, 'mae')
y_true = torch.Tensor([0, 1, 2, 3, 4, 5])
y_pred = y_true * 0.75
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['mae'])

.. testoutput::

1.25...

.. versionchanged:: 0.4.5
- Works with DDP.
"""
Expand Down
18 changes: 18 additions & 0 deletions ignite/contrib/metrics/regression/mean_absolute_relative_error.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,24 @@ class MeanAbsoluteRelativeError(_BaseRegression):
metric's device to be the same as your ``update`` arguments ensures the ``update`` method is
non-blocking. By default, CPU.

Examples:
To use with ``Engine`` and ``process_function``, simply attach the metric instance to the engine.
The output of the engine's ``process_function`` needs to be in format of
``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y, ...}``.

.. testcode::

metric = MeanAbsoluteRelativeError()
metric.attach(default_evaluator, 'mare')
y_true = torch.Tensor([1, 2, 3, 4, 5])
y_pred = y_true * 0.75
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['mare'])

.. testoutput::

0.25...

.. versionchanged:: 0.4.5
- Works with DDP.
"""
Expand Down
18 changes: 18 additions & 0 deletions ignite/contrib/metrics/regression/mean_error.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,24 @@ class MeanError(_BaseRegression):
device: specifies which device updates are accumulated on. Setting the
metric's device to be the same as your ``update`` arguments ensures the ``update`` method is
non-blocking. By default, CPU.

Examples:
To use with ``Engine`` and ``process_function``, simply attach the metric instance to the engine.
The output of the engine's ``process_function`` needs to be in format of
``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y, ...}``.

.. testcode::

metric = MeanError()
metric.attach(default_evaluator, 'me')
y_true = torch.Tensor([0, 1, 2, 3, 4, 5])
y_pred = y_true * 0.75
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['me'])

.. testoutput::

0.625...
"""

@reinit__is_reduced
Expand Down
18 changes: 18 additions & 0 deletions ignite/contrib/metrics/regression/mean_normalized_bias.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,24 @@ class MeanNormalizedBias(_BaseRegression):
metric's device to be the same as your ``update`` arguments ensures the ``update`` method is
non-blocking. By default, CPU.

Examples:
To use with ``Engine`` and ``process_function``, simply attach the metric instance to the engine.
The output of the engine's ``process_function`` needs to be in format of
``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y, ...}``.

.. testcode::

metric = MeanNormalizedBias()
metric.attach(default_evaluator, 'mnb')
y_true = torch.Tensor([1, 2, 3, 4, 5])
y_pred = y_true * 0.75
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['mnb'])

.. testoutput::

0.25...

.. versionchanged:: 0.4.5
- Works with DDP.
"""
Expand Down
19 changes: 19 additions & 0 deletions ignite/contrib/metrics/regression/median_absolute_error.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,25 @@ class MedianAbsoluteError(EpochMetric):
you want to compute the metric with respect to one of the outputs.
By default, metrics require the output as ``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y}``.
device: optional device specification for internal storage.


Examples:
To use with ``Engine`` and ``process_function``, simply attach the metric instance to the engine.
The output of the engine's ``process_function`` needs to be in format of
``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y, ...}``.

.. testcode::

metric = MedianAbsoluteError()
metric.attach(default_evaluator, 'mae')
y_true = torch.Tensor([0, 1, 2, 3, 4, 5])
y_pred = y_true * 0.75
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['mae'])

.. testoutput::

0.5...
"""

def __init__(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ class MedianAbsolutePercentageError(EpochMetric):
Current implementation stores all input data (output and target) in as tensors before computing a metric.
This can potentially lead to a memory error if the input data is larger than available RAM.


__ https://arxiv.org/abs/1809.03006

Args:
Expand All @@ -38,6 +37,24 @@ class MedianAbsolutePercentageError(EpochMetric):
you want to compute the metric with respect to one of the outputs.
By default, metrics require the output as ``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y}``.
device: optional device specification for internal storage.

Examples:
To use with ``Engine`` and ``process_function``, simply attach the metric instance to the engine.
The output of the engine's ``process_function`` needs to be in format of
``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y, ...}``.

.. testcode::

metric = MedianAbsolutePercentageError()
metric.attach(default_evaluator, 'mape')
y_true = torch.Tensor([1, 2, 3, 4, 5])
y_pred = y_true * 0.75
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['mape'])

.. testoutput::

25.0...
"""

def __init__(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ class MedianRelativeAbsoluteError(EpochMetric):
Current implementation stores all input data (output and target) in as tensors before computing a metric.
This can potentially lead to a memory error if the input data is larger than available RAM.


__ https://arxiv.org/abs/1809.03006

Args:
Expand All @@ -38,6 +37,24 @@ class MedianRelativeAbsoluteError(EpochMetric):
you want to compute the metric with respect to one of the outputs.
By default, metrics require the output as ``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y}``.
device: optional device specification for internal storage.

Examples:
To use with ``Engine`` and ``process_function``, simply attach the metric instance to the engine.
The output of the engine's ``process_function`` needs to be in format of
``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y, ...}``.

.. testcode::

metric = MedianRelativeAbsoluteError()
metric.attach(default_evaluator, 'mrae')
y_true = torch.Tensor([0, 1, 2, 3, 4, 5])
y_pred = y_true * 0.75
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics['mrae'])

.. testoutput::

0.5...
"""

def __init__(
Expand Down
Loading