Some metrics save all targets and predictions in the buffer. For large datasets, this may lead to a large memory footprint

[Enmartz]

🐛 Bug

Hi everyone! I am currently working with your libraries to manage my deep-learning projects. However, I have run into some metrics-related challenges, especially when dealing with large data sets. The problem arises because the used metrics are storing the entire data set in memory during training, validation, or testing, which can cause memory overflow issues. Specifically, I've been experiencing this problem with the SpectralAngleMapper metric.

To Reproduce

Just add the metric:

self.sam = torchmetrics.image.SpectralAngleMapper()

in your model lightning.LightningModule and use it in training_step, validation_step, or test_step functions.

The exact warning I obtain in console is this:

/home/myusername/anaconda3/envs/myenv/lib/python3.10/site-packages/torchmetrics/utilities/prints.py:43: UserWarning: Metric SpectralAngleMapper will save all targets and predictions in the buffer. For large datasets, this may lead to a large memory footprint.

Environment

• TorchMetrics 1.1.0
• Python 3.10.12
• torch 2.0.1+cu118

[github-actions]

Hi! thanks for your contribution!, great first issue!