Feature/log figure abstract #4975

Haydnspass · 2020-12-04T15:14:17Z

What does this PR do?

Make abstraction layer for figure logging, i.e. add default for logging figures for all implemented loggers.

My suggestion to #4973

ToDo:

Before submitting

Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests?
Did you verify new and existing tests pass locally with your changes?
If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self-explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified; Bugfixes should be including in bug-fix release milestones (m.f.X) and features should be included in (m.X.b) releases.

Did you have fun?

Make sure you had fun coding 🙃

pep8speaks · 2020-12-04T15:14:24Z

Hello @Haydnspass! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-02-26 14:46:43 UTC

# Conflicts: # environment.yml # requirements.txt # tests/loggers/test_tensorboard.py # tests/loggers/test_wandb.py

codecov · 2020-12-14T09:52:44Z

Codecov Report

Merging #4975 (e32c1e7) into master (e7298b5) will decrease coverage by 47%.
The diff coverage is 52%.

@@           Coverage Diff            @@
##           master   #4975     +/-   ##
========================================
- Coverage      93%     46%    -47%     
========================================
  Files         159     159             
  Lines       11381   11332     -49     
========================================
- Hits        10626    5244   -5382     
- Misses        755    6088   +5333

* Add colab badges to notebook Add colab badges to notebook to notebooks 4 & 5 * Add colab badges Co-authored-by: chaton <thomas@grid.ai>

* Prune CHANGELOG.md * Update CHANGELOG.md Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com> * Update CHANGELOG.md Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com> * Update CHANGELOG.md Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com> * Update CHANGELOG.md Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com> * Update CHANGELOG.md Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com> Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

…-AI#4861) * disable version if not required * disable version if not required * pep * chlog * improve test * improve test * parametrize test and update del_list * Update pytorch_lightning/callbacks/model_checkpoint.py Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> * try appending version to already saved ckpt_file * Revert "try appending version to already saved ckpt_file" This reverts commit 710e05e. * add more assertions * use BoringModel Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> Co-authored-by: chaton <thomas@grid.ai> Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>

* Update isort config * Apply isort with new config * Fix typo in isort config * fix rebase Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Fix reset TensorRunningAccum * add test for TensorRunningAccum's reset method * fix CI failed due to PEP8 Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Fix hang in DDP HPC accelerators init_device was never called * Update CHANGELOG.md

* support number * add two tests * wip * add ddp in special test * remove a test * move device to bottom * simplify test * update test * Update pytorch_lightning/core/step_result.py Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> * resolve sync_ddp Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* resolve bug * clean code * resolve comments * Update tests/trainer/optimization/test_multiple_optimizers.py Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com> * resolve another bug * add comments * use abs to find diff * update * resolve flake8 Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* remove nan loss whe missing * Update pytorch_lightning/core/lightning.py Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> * Apply suggestions from code review Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* add test * resolve bug * udpate test * wrongly copy / paste * update test * resolve a second bug Co-authored-by: Ubuntu <ubuntu@ip-172-31-62-109.ec2.internal>

# Conflicts: # .drone.yml # .github/CODEOWNERS # .github/ISSUE_TEMPLATE/how-to-question.md # .github/PULL_REQUEST_TEMPLATE.md # .github/mergify.yml # .github/prepare-nightly_version.py # .github/workflows/ci_dockers.yml # .github/workflows/ci_test-conda.yml # .github/workflows/ci_test-full.yml # .github/workflows/ci_test-mnodes.yml # .github/workflows/events-nightly.yml # .github/workflows/release-pypi.yml # CHANGELOG.md # Makefile # README.md # benchmarks/generate_comparison.py # benchmarks/test_basic_parity.py # benchmarks/test_sharded_parity.py # dockers/base-conda/Dockerfile # dockers/base-cuda/Dockerfile # dockers/tpu-tests/Dockerfile # docs/.build_docs.sh # docs/source/_static/images/benchmarks/figure-parity-times.png # docs/source/advanced/multi_gpu.rst # docs/source/advanced/transfer_learning.rst # docs/source/common/optimizers.rst # docs/source/common/test_set.rst # docs/source/common/trainer.rst # docs/source/extensions/metrics.rst # docs/source/governance.rst # docs/source/index.rst # docs/source/starter/introduction_guide.rst # docs/source/starter/new-project.rst # environment.yml # legacy/zero_training.py # notebooks/06-mnist-tpu-training.ipynb # pl_examples/__init__.py # pl_examples/basic_examples/autoencoder.py # pl_examples/basic_examples/backbone_image_classifier.py # pl_examples/basic_examples/conv_sequential_example.py # pl_examples/basic_examples/dali_image_classifier.py # pl_examples/basic_examples/mnist_datamodule.py # pl_examples/basic_examples/simple_image_classifier.py # pl_examples/bug_report_model.py # pl_examples/domain_templates/computer_vision_fine_tuning.py # pl_examples/domain_templates/generative_adversarial_net.py # pl_examples/domain_templates/imagenet.py # pl_examples/domain_templates/reinforce_learn_Qnet.py # pl_examples/domain_templates/semantic_segmentation.py # pl_examples/domain_templates/unet.py # pyproject.toml # pytorch_lightning/__init__.py # pytorch_lightning/accelerators/accelerator.py # pytorch_lightning/accelerators/accelerator_connector.py # pytorch_lightning/accelerators/cpu_accelerator.py # pytorch_lightning/accelerators/ddp2_accelerator.py # pytorch_lightning/accelerators/ddp_accelerator.py # pytorch_lightning/accelerators/ddp_cpu_hpc_accelerator.py # pytorch_lightning/accelerators/ddp_cpu_spawn_accelerator.py # pytorch_lightning/accelerators/ddp_hpc_accelerator.py # pytorch_lightning/accelerators/ddp_spawn_accelerator.py # pytorch_lightning/accelerators/dp_accelerator.py # pytorch_lightning/accelerators/gpu_accelerator.py # pytorch_lightning/accelerators/horovod_accelerator.py # pytorch_lightning/accelerators/tpu_accelerator.py # pytorch_lightning/callbacks/early_stopping.py # pytorch_lightning/callbacks/lr_monitor.py # pytorch_lightning/callbacks/model_checkpoint.py # pytorch_lightning/core/hooks.py # pytorch_lightning/core/lightning.py # pytorch_lightning/core/optimizer.py # pytorch_lightning/core/saving.py # pytorch_lightning/core/step_result.py # pytorch_lightning/loggers/__init__.py # pytorch_lightning/loggers/comet.py # pytorch_lightning/loggers/mlflow.py # pytorch_lightning/loggers/neptune.py # pytorch_lightning/loggers/tensorboard.py # pytorch_lightning/loggers/wandb.py # pytorch_lightning/metrics/classification/precision_recall.py # pytorch_lightning/metrics/metric.py # pytorch_lightning/overrides/data_parallel.py # pytorch_lightning/plugins/ddp_plugin.py # pytorch_lightning/plugins/environments/slurm_environment.py # pytorch_lightning/plugins/environments/torchelastic_environment.py # pytorch_lightning/plugins/native_amp.py # pytorch_lightning/plugins/rpc_plugin.py # pytorch_lightning/plugins/sharded_plugin.py # pytorch_lightning/plugins/training_type/rpc_sequential.py # pytorch_lightning/profiler/profilers.py # pytorch_lightning/setup_tools.py # pytorch_lightning/trainer/connectors/checkpoint_connector.py # pytorch_lightning/trainer/connectors/debugging_connector.py # pytorch_lightning/trainer/connectors/logger_connector/epoch_result_store.py # pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py # pytorch_lightning/trainer/connectors/optimizer_connector.py # pytorch_lightning/trainer/connectors/precision_connector.py # pytorch_lightning/trainer/connectors/slurm_connector.py # pytorch_lightning/trainer/evaluation_loop.py # pytorch_lightning/trainer/optimizers.py # pytorch_lightning/trainer/properties.py # pytorch_lightning/trainer/trainer.py # pytorch_lightning/trainer/training_loop.py # pytorch_lightning/trainer/training_tricks.py # pytorch_lightning/tuner/batch_size_scaling.py # pytorch_lightning/utilities/__init__.py # pytorch_lightning/utilities/apply_func.py # pytorch_lightning/utilities/distributed.py # pytorch_lightning/utilities/parsing.py # pytorch_lightning/utilities/seed.py # pytorch_lightning/utilities/xla_device_utils.py # requirements.txt # requirements/extra.txt # setup.py # tests/__init__.py # tests/accelerators/test_multi_nodes_gpu.py # tests/callbacks/test_callbacks.py # tests/callbacks/test_early_stopping.py # tests/callbacks/test_lr_monitor.py # tests/checkpointing/test_checkpoint_callback_frequency.py # tests/checkpointing/test_legacy_checkpoints.py # tests/checkpointing/test_model_checkpoint.py # tests/checkpointing/test_trainer_checkpoint.py # tests/conftest.py # tests/core/test_datamodules.py # tests/core/test_lightning_module.py # tests/core/test_lightning_optimizer.py # tests/core/test_memory.py # tests/deprecated_api/test_remove_1-3.py # tests/helpers/boring_model.py # tests/helpers/datasets.py # tests/helpers/utils.py # tests/loggers/test_all.py # tests/loggers/test_base.py # tests/loggers/test_comet.py # tests/loggers/test_mlflow.py # tests/loggers/test_neptune.py # tests/loggers/test_tensorboard.py # tests/loggers/test_wandb.py # tests/metrics/classification/test_f_beta.py # tests/metrics/test_metric.py # tests/models/conf/config.yaml # tests/models/test_amp.py # tests/models/test_cpu.py # tests/models/test_hooks.py # tests/models/test_horovod.py # tests/models/test_hparams.py # tests/models/test_restore.py # tests/models/test_sync_batchnorm.py # tests/models/test_torchscript.py # tests/plugins/test_rpc_sequential_plugin.py # tests/special_tests.sh # tests/test_profiler.py # tests/trainer/dynamic_args/test_multiple_optimizers.py # tests/trainer/flags/test_fast_dev_run.py # tests/trainer/legacy_deprecate_flow_log/test_eval_loop_dict_return.py # tests/trainer/logging_/test_eval_loop_logging_1_0.py # tests/trainer/logging_/test_progress_bar_logging.py # tests/trainer/logging_/test_train_loop_logging_1_0.py # tests/trainer/optimization/test_manual_optimization.py # tests/trainer/optimization/test_multiple_optimizers.py # tests/trainer/optimization/test_optimizers.py # tests/trainer/test_supporters.py # tests/trainer/test_trainer.py # tests/utilities/test_parsing.py # tests/utilities/test_seed.py # tests/utilities/test_xla_device_utils.py

Haydnspass · 2021-02-26T12:15:21Z

I am going to mark this PR now ready for review although some of the checks do not pass. However these affect only files that I have not touched, so I guess they are failing in master as well? Or am I missing something?

Otherwise, input/help appreciated.

SkafteNicki · 2021-02-26T14:45:50Z

hi @Haydnspass,
Seems the PR contains quite a lot unrelated changes. Your base branch was changed from release/1.2-dev to master since the PR was opened, so please rebase on master.

Haydnspass · 2021-02-26T18:00:01Z

Closed in favor of PR on a clean branch.

Haydnspass added 4 commits October 23, 2020 15:35

Add logger for tensorboard

be2ccde

Add close argument

79135a7

Add loggers and tests

d0d952c

Add more informative error

e3bfd53

edenlightning added the logging Related to the `LoggerConnector` and `log()` label Dec 4, 2020

edenlightning linked an issue Dec 4, 2020 that may be closed by this pull request

Abstract matplotlib figure logging from individual logger API #4973

Closed

edenlightning added the feature Is an improvement or enhancement label Dec 4, 2020

edenlightning added this to the 1.2 milestone Dec 4, 2020

Haydnspass added 4 commits December 5, 2020 13:27

Merge branch 'master' into feature/log_figure_abstract

ccb56dd

# Conflicts: # environment.yml # requirements.txt # tests/loggers/test_tensorboard.py # tests/loggers/test_wandb.py

PEP8ify

9721837

Rm whitespace, PEP8ify

28d26fa

Make matplotlib 'base' and not extra requirement.

b046bbe

Add Google Colab badges (Lightning-AI#5111)

eb9cb3c

* Add colab badges to notebook Add colab badges to notebook to notebooks 4 & 5 * Add colab badges Co-authored-by: chaton <thomas@grid.ai>

Borda changed the base branch from master to release/1.2-dev December 14, 2020 17:33

tadejsv and others added 14 commits December 14, 2020 20:13

Fix hanging metrics tests (Lightning-AI#5134)

69123af

simplify changelog (Lightning-AI#5135)

84bb9db

add copyright to tests (Lightning-AI#5143)

fde972f

Update changelog, increment version (Lightning-AI#5148)

fe75c73

Update isort config (Lightning-AI#5142)

afe5da7

* Update isort config * Apply isort with new config * Fix typo in isort config * fix rebase Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

Fix reset TensorRunningAccum (Lightning-AI#5106)

b4d926b

* Fix reset TensorRunningAccum * add test for TensorRunningAccum's reset method * fix CI failed due to PEP8 Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

Fix hang in DDP HPC accelerators (Lightning-AI#5157)

94838d3

* Fix hang in DDP HPC accelerators init_device was never called * Update CHANGELOG.md

merge 1.1.x branch (SN)

bc4008b

[bug-fix] Metric reduction with Logging (Lightning-AI#5150)

81fd33b

* add test * resolve bug * udpate test * wrongly copy / paste * update test * resolve a second bug Co-authored-by: Ubuntu <ubuntu@ip-172-31-62-109.ec2.internal>

Merge current master

d9c8415

edenlightning modified the milestones: 1.2, 1.3 Feb 8, 2021

Base automatically changed from release/1.2-dev to master February 11, 2021 14:31

Lucas-Raphael Müller added 4 commits February 12, 2021 15:26

Centralize plt.close test

92a5961

Update changelog and docs

c1f376d

Add file extension option to mlflow.

fe37359

Update tests to fileextension.

69ac37b

edenlightning removed this from the 1.3 milestone Feb 22, 2021

Haydnspass added 4 commits February 26, 2021 12:09

Post-Merge fixes

a0b99e5

Satisfy code style requirements

dc213f8

Make matplotlib base requirement

a18ef6b

Haydnspass marked this pull request as ready for review February 26, 2021 12:15

Haydnspass requested review from awaelchli, Borda, carmocca, edenlightning, justusschock, SeanNaren, SkafteNicki, tchaton and williamFalcon as code owners February 26, 2021 12:15

mergify bot removed the has conflicts label Feb 26, 2021

Removed tensorboard log_figure during merge by accident.

a822bc1

Haydnspass mentioned this pull request Feb 26, 2021

Abstract log_figure API that implements reasonable default for all lo… #6227

Closed

18 tasks

Haydnspass closed this Feb 26, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/log figure abstract #4975

Feature/log figure abstract #4975

Haydnspass commented Dec 4, 2020 •

edited

Loading

pep8speaks commented Dec 4, 2020 •

edited

Loading

codecov bot commented Dec 14, 2020 •

edited

Loading

Haydnspass commented Feb 26, 2021

SkafteNicki commented Feb 26, 2021

Haydnspass commented Feb 26, 2021

Feature/log figure abstract #4975

Feature/log figure abstract #4975

Conversation

Haydnspass commented Dec 4, 2020 • edited Loading

What does this PR do?

Before submitting

PR review

Did you have fun?

pep8speaks commented Dec 4, 2020 • edited Loading

Comment last updated at 2021-02-26 14:46:43 UTC

codecov bot commented Dec 14, 2020 • edited Loading

Codecov Report

Haydnspass commented Feb 26, 2021

SkafteNicki commented Feb 26, 2021

Haydnspass commented Feb 26, 2021

Haydnspass commented Dec 4, 2020 •

edited

Loading

pep8speaks commented Dec 4, 2020 •

edited

Loading

codecov bot commented Dec 14, 2020 •

edited

Loading