make_current/clear_current are deprecated in Tornado #6784

QuLogic · 2022-07-26T05:33:20Z

What happened:
make_current and clear_current appear to be deprecated in Tornado 6.2, which causes errors in tests, since warnings are treated as errors. Though it's my mistake that I left that on, it's probably something that should be fixed soonish.

_______________________ test_loop_started_in_constructor _______________________

cleanup = None

    def test_loop_started_in_constructor(cleanup):
        # test that SpecCluster.__init__ starts a loop in another thread
>       cluster = SpecCluster(worker_spec, scheduler=scheduler, loop=None)

../../BUILDROOT/python-distributed-2022.7.1-1.fc37.x86_64/usr/lib/python3.11/site-packages/distributed/deploy/tests/test_spec_cluster.py:87: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../BUILDROOT/python-distributed-2022.7.1-1.fc37.x86_64/usr/lib/python3.11/site-packages/distributed/deploy/spec.py:255: in __init__
    super().__init__(
../../BUILDROOT/python-distributed-2022.7.1-1.fc37.x86_64/usr/lib/python3.11/site-packages/distributed/deploy/cluster.py:67: in __init__
    self._loop_runner = LoopRunner(loop=loop, asynchronous=asynchronous)
../../BUILDROOT/python-distributed-2022.7.1-1.fc37.x86_64/usr/lib/python3.11/site-packages/distributed/utils.py:440: in __init__
    self._loop = IOLoop()
/usr/lib64/python3.11/site-packages/tornado/util.py:276: in __new__
    instance.initialize(*args, **init_kwargs)
/usr/lib64/python3.11/site-packages/tornado/platform/asyncio.py:339: in initialize
    super().initialize(**kwargs)
/usr/lib64/python3.11/site-packages/tornado/platform/asyncio.py:140: in initialize
    super().initialize(**kwargs)
/usr/lib64/python3.11/site-packages/tornado/ioloop.py:350: in initialize
    self.make_current()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tornado.platform.asyncio.AsyncIOLoop object at 0x7fb8eff84690>

    def make_current(self) -> None:
>       warnings.warn(
            "make_current is deprecated; start the event loop first",
            DeprecationWarning,
            stacklevel=2,
        )
E       DeprecationWarning: make_current is deprecated; start the event loop first

/usr/lib64/python3.11/site-packages/tornado/platform/asyncio.py:353: DeprecationWarning

_____________________ test_client_async_before_loop_starts _____________________

cleanup = None

    def test_client_async_before_loop_starts(cleanup):
        async def close():
            async with client:
                pass
    
>       with pristine_loop() as loop:

../../BUILDROOT/python-distributed-2022.7.1-1.fc37.x86_64/usr/lib/python3.11/site-packages/distributed/tests/test_client.py:5540: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib64/python3.11/contextlib.py:137: in __enter__
    return next(self.gen)
../../BUILDROOT/python-distributed-2022.7.1-1.fc37.x86_64/usr/lib/python3.11/site-packages/distributed/utils_test.py:222: in pristine_loop
    IOLoop.clear_instance()
/usr/lib64/python3.11/site-packages/tornado/ioloop.py:227: in clear_instance
    IOLoop.clear_current()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    @staticmethod
    def clear_current() -> None:
        """Clears the `IOLoop` for the current thread.
    
        Intended primarily for use by test frameworks in between tests.
    
        .. versionchanged:: 5.0
           This method also clears the current `asyncio` event loop.
        .. deprecated:: 6.2
        """
>       warnings.warn(
            "clear_current is deprecated",
            DeprecationWarning,
            stacklevel=2,
        )
E       DeprecationWarning: clear_current is deprecated

/usr/lib64/python3.11/site-packages/tornado/ioloop.py:317: DeprecationWarning

What you expected to happen:
Tests pass.

Environment:

Dask version: 2022.7.1
Python version: 3.11.0.b4
Operating System: Fedora Rawhide
Install method (conda, pip, source): source

The text was updated successfully, but these errors were encountered:

jrbourbeau · 2022-07-26T14:56:44Z

Thanks @QuLogic! We're currently setting tornado <6.2 to avoid these deprecation errors / other possible unknown issues (xref #6669)

distributed/requirements.txt

Line 12 in a53858a

tornado >= 6.0.3, <6.2

I'm curious where this is showing up. Is while while packaging distributed?

QuLogic · 2022-07-27T06:08:27Z

Yes, I am trying to prepare packages for review in Fedora again, though there's lots of trouble in Rawhide at the moment due to new Tornado and Python 3.11 beta. I am a bit closer to passing on Fedora 36.

hendrikmakait · 2022-07-28T14:55:02Z

xref: #6669

rikardn · 2022-12-06T08:20:40Z

Using distributed==2022.12.0 and tornado==6.2 I still get the deprecation warning (and potentially some other issues).

_____________________________________________________________________________ test_allometry _____________________________________________________________________________
tests/integration/test_allometry.py:19: in test_allometry
    res = run_tool(
.tox/py310-integration/lib/python3.10/site-packages/pharmpy/tools/run.py:178: in run_tool
    return run_tool_with_name(name, tool, *args, **kwargs)
.tox/py310-integration/lib/python3.10/site-packages/pharmpy/tools/run.py:206: in run_tool_with_name
    res = execute_workflow(wf, dispatcher=dispatcher, database=database)
.tox/py310-integration/lib/python3.10/site-packages/pharmpy/workflows/execute.py:71: in execute_workflow
    res: T = dispatcher.run(workflow)
.tox/py310-integration/lib/python3.10/site-packages/pharmpy/workflows/dispatchers/local_dask.py:77: in run
    with LocalCluster(
.tox/py310-integration/lib/python3.10/site-packages/distributed/deploy/local.py:253: in __init__
    super().__init__(
.tox/py310-integration/lib/python3.10/site-packages/distributed/deploy/spec.py:259: in __init__
    super().__init__(
.tox/py310-integration/lib/python3.10/site-packages/distributed/deploy/cluster.py:70: in __init__
    self._loop_runner = LoopRunner(loop=loop, asynchronous=asynchronous)
.tox/py310-integration/lib/python3.10/site-packages/distributed/utils.py:449: in __init__
    self._loop = IOLoop()
.tox/py310-integration/lib/python3.10/site-packages/tornado/util.py:276: in __new__
    instance.initialize(*args, **init_kwargs)
.tox/py310-integration/lib/python3.10/site-packages/tornado/platform/asyncio.py:339: in initialize
    super().initialize(**kwargs)
.tox/py310-integration/lib/python3.10/site-packages/tornado/platform/asyncio.py:140: in initialize
    super().initialize(**kwargs)
.tox/py310-integration/lib/python3.10/site-packages/tornado/ioloop.py:350: in initialize
    self.make_current()
.tox/py310-integration/lib/python3.10/site-packages/tornado/platform/asyncio.py:353: in make_current
    warnings.warn(
E   DeprecationWarning: make_current is deprecated; start the event loop first
======================================================================== short test summary info =========================================================================
FAILED tests/integration/test_allometry.py::test_allometry - DeprecationWarning: make_current is deprecated; start the event loop first
=========================================================================== 1 failed in 1.35s ============================================================================
Exception ignored in atexit callback: <function close_clusters at 0x7ff1c3b120e0>
Traceback (most recent call last):
  File "/home/devel/pharmpy/.tox/py310-integration/lib/python3.10/site-packages/distributed/deploy/spec.py", line 677, in close_clusters
    cluster.close(timeout=10)
  File "/home/devel/pharmpy/.tox/py310-integration/lib/python3.10/site-packages/distributed/deploy/cluster.py", line 217, in close
    return self.sync(self._close, callback_timeout=timeout)
  File "/home/devel/pharmpy/.tox/py310-integration/lib/python3.10/site-packages/distributed/utils.py", line 332, in sync
    asynchronous = self.asynchronous
  File "/home/devel/pharmpy/.tox/py310-integration/lib/python3.10/site-packages/distributed/utils.py", line 325, in asynchronous
    return in_async_call(self.loop, default=getattr(self, "_asynchronous", False))
  File "/home/devel/pharmpy/.tox/py310-integration/lib/python3.10/site-packages/distributed/deploy/cluster.py", line 102, in loop
    self.__loop = loop = self._loop_runner.loop
AttributeError: 'LocalCluster' object has no attribute '_loop_runner'
Exception ignored in: <function Cluster.__del__ at 0x7ff1c3c1ecb0>
Traceback (most recent call last):
  File "/home/devel/pharmpy/.tox/py310-integration/lib/python3.10/site-packages/distributed/deploy/cluster.py", line 225, in __del__
ResourceWarning: unclosed cluster with a broken __repr__ <distributed.deploy.local.LocalCluster object at 0x7ff1c3b0ec80>
ERROR: InvocationError for command /home/devel/pharmpy/.tox/py310-integration/bin/pytest -vv tests/integration/test_allometry.py (exited with code 1)
________________________________________________________________________________ summary _________________________________________________________________________________
ERROR:   py310-integration: commands failed

fjetter · 2022-12-06T10:51:54Z

cc @graingert

graingert · 2022-12-07T15:42:33Z

@rikardn for your usecase could you grab the distributed.utils_test.loop_in_thread fixture and use that until we fix the LoopRunner?

…n LoopRunner Closes dask#6784

graingert mentioned this issue Nov 7, 2022

remove deprecated code calls to IOLoop.make_current() #7240

Merged

2 tasks

rikardn mentioned this issue Dec 6, 2022

Cannot upgrade tornado and jupyter-client pharmpy/pharmpy#1113

Closed

hayesgb assigned graingert Dec 7, 2022

graingert added a commit to graingert/distributed that referenced this issue Jan 11, 2023

avoid calls to make_current() and make_clear() by using asyncio.run i…

c8da8a3

…n LoopRunner Closes dask#6784

graingert mentioned this issue Jan 11, 2023

avoid calls to make_current() and make_clear() by using asyncio.run in LoopRunner #7467

Merged

2 tasks

graingert added a commit to graingert/distributed that referenced this issue Jan 11, 2023

avoid calls to make_current() and make_clear() by using asyncio.run i…

dba048b

…n LoopRunner Closes dask#6784

graingert added a commit to graingert/distributed that referenced this issue Jan 11, 2023

avoid calls to make_current() and make_clear() by using asyncio.run i…

7c46b17

…n LoopRunner Closes dask#6784

graingert added a commit to graingert/distributed that referenced this issue Jan 12, 2023

avoid calls to make_current() and make_clear() by using asyncio.run i…

a7b96ca

…n LoopRunner Closes dask#6784

graingert added a commit to graingert/distributed that referenced this issue Jan 12, 2023

avoid calls to make_current() and make_clear() by using asyncio.run i…

be56b65

…n LoopRunner Closes dask#6784

graingert added a commit to graingert/distributed that referenced this issue Jan 12, 2023

avoid calls to make_current() and make_clear() by using asyncio.run i…

86de36a

…n LoopRunner Closes dask#6784

graingert added a commit to graingert/distributed that referenced this issue Feb 1, 2023

avoid calls to make_current() and make_clear() by using asyncio.run i…

f121d04

…n LoopRunner Closes dask#6784

graingert added a commit to graingert/distributed that referenced this issue Feb 15, 2023

avoid calls to make_current() and make_clear() by using asyncio.run i…

21c7e40

…n LoopRunner Closes dask#6784

crusaderky closed this as completed in #7467 Jun 16, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

make_current/clear_current are deprecated in Tornado #6784

make_current/clear_current are deprecated in Tornado #6784

QuLogic commented Jul 26, 2022 •

edited

Loading

jrbourbeau commented Jul 26, 2022

QuLogic commented Jul 27, 2022

hendrikmakait commented Jul 28, 2022

rikardn commented Dec 6, 2022

fjetter commented Dec 6, 2022

graingert commented Dec 7, 2022

make_current/clear_current are deprecated in Tornado #6784

make_current/clear_current are deprecated in Tornado #6784

Comments

QuLogic commented Jul 26, 2022 • edited Loading

jrbourbeau commented Jul 26, 2022

QuLogic commented Jul 27, 2022

hendrikmakait commented Jul 28, 2022

rikardn commented Dec 6, 2022

fjetter commented Dec 6, 2022

graingert commented Dec 7, 2022

QuLogic commented Jul 26, 2022 •

edited

Loading