Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move contrib.handlers #3204

Merged
merged 5 commits into from
Mar 22, 2024
Merged

Conversation

leej3
Copy link
Collaborator

@leej3 leej3 commented Mar 15, 2024

Moves ignite.contrib.handers to ignite.handlers maintaining backwards compatible. Proposed removal is 0.8.0 0.6.0.

Full test suite was run locally (though with 356skips, mostly due to missing (multi-node, horovod, or xla dependencies). Unrelated failures were observed before and after these changes in ignite/metrics/nlp/test_rouge.py and ignite/handlers/test_checkpoint.py.

@leej3 leej3 changed the title Move contrib.handler files [WIP] Move contrib.handlers, add FBResearchLogger, and move contrib.metrics files Mar 15, 2024
@vfdev-5
Copy link
Collaborator

vfdev-5 commented Mar 17, 2024

@leej3 thanks for the PR!
I think it would be better to split it into multiple ones such that we could review and land them quickier

@leej3 leej3 force-pushed the move-contrib-handler-files branch from 40552c6 to 8d77d87 Compare March 20, 2024 14:40
@leej3 leej3 changed the title [WIP] Move contrib.handlers, add FBResearchLogger, and move contrib.metrics files Move contrib.handlers Mar 20, 2024
@github-actions github-actions bot added module: handlers Core Handlers module module: contrib Contrib module labels Mar 20, 2024
@leej3 leej3 changed the title Move contrib.handlers [WIP] Move contrib.handlers Mar 20, 2024
@leej3
Copy link
Collaborator Author

leej3 commented Mar 20, 2024

@vfdev-5 two remaining details that I can spot, and will work through now, are corresponding updates to handlers.rst files, and altering the imports across the codebase (notebooks etc.) to try to guide usage to the new import location of this functionality.

One potential issue is the dependencies that this introduces (I spotted in the guidelines for contributing that the contrib module has "handlers classes that may require additional dependencies". I am assuming that this will not cause an issue but let me know if there is a way of addressing this.

@vfdev-5
Copy link
Collaborator

vfdev-5 commented Mar 20, 2024

two remaining details that I can spot, and will work through now, are corresponding updates to handlers.rst files, and altering the imports across the codebase (notebooks etc.) to try to guide usage to the new import location of this functionality.

thanks, sounds good!

One potential issue is the dependencies that this introduces (I spotted in the guidelines for contributing that the contrib module has "handlers classes that may require additional dependencies". I am assuming that this will not cause an issue but let me know if there is a way of addressing this.

I think we can remove that sentence where it is written. Overall global idea is to get rid of contrib module (keeping it just for BC).

@leej3
Copy link
Collaborator Author

leej3 commented Mar 21, 2024

I'm encountering a circular import with "global_step_from_engine". In ignite.contrib.handlers it is imported in both the top level init and in each logger module e.g. ignite.contrib.handlers.clearml_logger.
As we migrate this functionality we get a circular import because "global_step_from_engine" is defined in the ignite.handlers init.

I think the solution is to remove global_step_from_engine from the namespace of each logger module and change everywhere in the codebase that currently imports it from those modules to

from ignite.handlers import global_step_from_engine

Shall I include a deprecated "global_step_from_engine" in the migrated API for now to help users with the migration? Something like:

# Inside each logger module where global_step_from_engine was available

import warnings

def global_step_from_engine(*args, **kwargs):
    warnings.warn(
        "Importing 'global_step_from_engine' from this module is deprecated and will be removed in a future version. "
        "Please import it from 'ignite.handlers' instead.",
        DeprecationWarning,
        stacklevel=2,
    )
    from ignite.handlers import global_step_from_engine as gsf_engine
    return gsf_engine(*args, **kwargs)

Also, let know if you want/think of another solution.

@github-actions github-actions bot added docs module: engine Engine module examples Examples labels Mar 21, 2024
@leej3 leej3 force-pushed the move-contrib-handler-files branch from 79d9d0f to 959abc2 Compare March 21, 2024 17:59
move loggers and other class from contrib to ignite.handlers
make changes in a backwards compatible manner
fix circular imports that happened during the migration in ignite/__init__.py
and ignite/handlers/__init__.py
move tests to ignite.handlers along with fixtures and changing the
imports

add a test to ensure deprecation warnings for ignite.contrib.handlers
@leej3 leej3 force-pushed the move-contrib-handler-files branch from b41b65a to b2c0ab2 Compare March 21, 2024 18:56
@leej3 leej3 changed the title [WIP] Move contrib.handlers Move contrib.handlers Mar 21, 2024
@leej3
Copy link
Collaborator Author

leej3 commented Mar 21, 2024

As discussed, this implementation makes sure that global_step_from_engine is still importable from the logger modules and ignite.handlers (the circular import is solved by defining the function in a utils.py). Another circular import was resolved in ignite/__init__.py. Tests pass for me in ignite.contrib.handlers and ignite.handlers for both commits.

mark as deprecated and provide link to migrated functionality
ignite/__init__.py Outdated Show resolved Hide resolved
previously this occured as a side effect, adding it explicitly now.
@leej3 leej3 force-pushed the move-contrib-handler-files branch from a827eb7 to ed7db21 Compare March 22, 2024 13:34
Copy link
Collaborator

@vfdev-5 vfdev-5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks John, looks good!

@vfdev-5 vfdev-5 merged commit 1e7d336 into pytorch:master Mar 22, 2024
16 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs examples Examples module: contrib Contrib module module: engine Engine module module: handlers Core Handlers module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants