Failing to import from wakepy when running test with tox (KeyError: 'DBUS_SESSION_BUS_ADDRESS')

[fohrloop]

Setup:

• System: Ubuntu (with dbus installed)
• Running with tox, which runs pytest

What happens?

When calling

from wakepy import set_keepawake, unset_keepawake

user will get:

KeyError: 'DBUS_SESSION_BUS_ADDRESS'

Steps to reproduce

1. Create some tox config file, e.g. tox.ini with following contents

[tox]
env_list = py{38,39,310,311}
minversion = 4.4.11

[testenv]
description = run the tests with pytest
package = wheel
wheel_build_env = .pkg
deps =
    pytest>=6
commands =
    pytest {tty:--color=yes} {posargs}

2. Create simple test which imports set_keepawake:

# tests/test_wakepy.py
def test_import_set_and_unset_keeepawake():
    from wakepy import set_keepawake, unset_keepawake

3. Run tox:

niko@niko-ubuntu-home:~/tmp/repos/wakepy$ tox -e py310
.pkg: _optional_hooks> python /home/niko/venvs/wakepy/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_wheel> python /home/niko/venvs/wakepy/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: build_wheel> python /home/niko/venvs/wakepy/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
py310: install_package> python -I -m pip install --force-reinstall --no-deps /home/niko/tmp/repos/wakepy/.tox/.tmp/package/10/wakepy-0.6.0-py3-none-any.whl
py310: commands[0]> pytest --color=yes
=================================================================== test session starts ====================================================================
platform linux -- Python 3.10.6, pytest-7.2.2, pluggy-1.0.0
cachedir: .tox/py310/.pytest_cache
rootdir: /home/niko/tmp/repos/wakepy
collected 1 item                                                                                                                                           

tests/test_wakepy.py F                                                                                                                               [100%]

========================================================================= FAILURES =========================================================================
___________________________________________________________ test_import_set_and_unset_keeepawake ___________________________________________________________

    def test_import_set_and_unset_keeepawake():
>       from wakepy import set_keepawake, unset_keepawake

tests/test_wakepy.py:2: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.tox/py310/lib/python3.10/site-packages/wakepy/__init__.py:35: in <module>
    from ._linux import set_keepawake, unset_keepawake
.tox/py310/lib/python3.10/site-packages/wakepy/_linux/__init__.py:17: in <module>
    my_module = import_module(f".{module}", f"wakepy._linux")
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
.tox/py310/lib/python3.10/site-packages/wakepy/_linux/_jeepney_dbus.py:26: in <module>
    connection = open_dbus_connection(bus="SESSION")
.tox/py310/lib/python3.10/site-packages/jeepney/io/blocking.py:341: in open_dbus_connection
    bus_addr = get_bus(bus)
.tox/py310/lib/python3.10/site-packages/jeepney/bus.py:53: in get_bus
    return find_session_bus()
.tox/py310/lib/python3.10/site-packages/jeepney/bus.py:42: in find_session_bus
    addr = os.environ['DBUS_SESSION_BUS_ADDRESS']
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = environ({'HOME': '/home/niko', 'LANG': 'en_US.UTF-8', 'TERM': 'xterm-256color', 'PATH': '/home/niko/tmp/repos/wakepy/....NS': '156', 'LINES': '26', 'PYTEST_CURRENT_TEST': 'tests/test_wakepy.py::test_import_set_and_unset_keeepawake (call)'})
key = 'DBUS_SESSION_BUS_ADDRESS'

    def __getitem__(self, key):
        try:
            value = self._data[self.encodekey(key)]
        except KeyError:
            # raise KeyError with the original key value
>           raise KeyError(key) from None
E           KeyError: 'DBUS_SESSION_BUS_ADDRESS'

/usr/lib/python3.10/os.py:679: KeyError
================================================================= short test summary info ==================================================================
FAILED tests/test_wakepy.py::test_import_set_and_unset_keeepawake - KeyError: 'DBUS_SESSION_BUS_ADDRESS'
==================================================================== 1 failed in 0.08s =====================================================================
py310: exit 1 (0.33 seconds) /home/niko/tmp/repos/wakepy> pytest --color=yes pid=8462
.pkg: _exit> python /home/niko/venvs/wakepy/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
  py310: FAIL code 1 (1.18=setup[0.84]+cmd[0.33] seconds)
  evaluation failed :( (1.22 seconds)

[fohrloop]

I checked the environment variables with my current terminal (myenv) and inside the tox pytest process (testenv). These are the common environment variables:

key          my                                                  test
-----------  --------------------------------------------------  --------------------------------------------------
LANG         en_US.UTF-8                                         en_US.UTF-8
TERM         xterm-256color                                      xterm-256color
PATH         /home/niko/.local/bin:/home/niko/venvs/wakepy/bin:  /home/niko/tmp/repos/wakepy/.tox/py310/bin:/home/n
VIRTUAL_ENV  /home/niko/venvs/wakepy                             /home/niko/tmp/repos/wakepy/.tox/py310
HOME         /home/niko                                          /home/niko

These are the environment variables specific to the test environment:

key                            test
-----------------------------  -----------------------------------------------------------------------------
LINES                          26
TOX_PACKAGE                    /home/niko/tmp/repos/wakepy/.tox/.tmp/package/5/wakepy-0.6.0-py3-none-any.whl
TOX_ENV_NAME                   py310
PYTHONHASHSEED                 424929341
PYTHONIOENCODING               utf-8
TOX_ENV_DIR                    /home/niko/tmp/repos/wakepy/.tox/py310
PYTEST_CURRENT_TEST            tests/test_wakepy.py::test_import_set_and_unset_keeepawake (call)
COLUMNS                        156
PIP_DISABLE_PIP_VERSION_CHECK  1
TOX_WORK_DIR                   /home/niko/tmp/repos/wakepy/.tox

The test environment is really missing the environment variable DBUS_SESSION_BUS_ADDRESS. It is set on my current terminal to:

$ echo $DBUS_SESSION_BUS_ADDRESS
unix:path=/run/user/1000/bus

[fohrloop]

Wakepy 0.6.0 uses by default the Inhibit/Uninhibit methods of the org.freedesktop.Screensaver interface of the session D-Bus.

Reading at Well-known Message Bus Instances from D-Bus Specification:

The address of the login session message bus is given in the DBUS_SESSION_BUS_ADDRESS environment variable.

When using a breakpoint inside the test (tox+pytest), I can see that the dbus is running:

(Pdb) os.system('ps -x | grep dbus')
   1107 ?        Ss     0:03 /usr/bin/dbus-daemon --session --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
   1246 ?        S      0:00 /usr/bin/dbus-daemon --config-file=/usr/share/defaults/at-spi2/accessibility.conf --nofork --print-address 11 --address=un
  14110 pts/0    S+     0:00 sh -c ps -x | grep dbus
  14113 pts/0    S+     0:00 grep dbus
0

but there is no DBUS_SESSION_BUS_ADDRESS environment variable:

(Pdb) pprint.pprint(dict(os.environ))
{'COLUMNS': '151',
 'HOME': '/home/niko',
 'LANG': 'en_US.UTF-8',
 'LINES': '15',
 'PATH': '/home/niko/tmp/repos/wakepy/.tox/py310/bin:/home/niko/.local/bin:/home/niko/venvs/wakepy/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin',
 'PIP_DISABLE_PIP_VERSION_CHECK': '1',
 'PYTEST_CURRENT_TEST': 'tests/test_wakepy.py::test_import_set_and_unset_keeepawake '
                        '(call)',
 'PYTHONHASHSEED': '1404011170',
 'PYTHONIOENCODING': 'utf-8',
 'TERM': 'xterm-256color',
 'TOX_ENV_DIR': '/home/niko/tmp/repos/wakepy/.tox/py310',
 'TOX_ENV_NAME': 'py310',
 'TOX_PACKAGE': '/home/niko/tmp/repos/wakepy/.tox/.tmp/package/11/wakepy-0.6.0-py3-none-any.whl',
 'TOX_WORK_DIR': '/home/niko/tmp/repos/wakepy/.tox',
 'VIRTUAL_ENV': '/home/niko/tmp/repos/wakepy/.tox/py310'}

(Pdb) os.environ['DBUS_SESSION_BUS_ADDRESS']
*** KeyError: 'DBUS_SESSION_BUS_ADDRESS'

[fohrloop]

This can be fixed with adding DBUS_SESSION_BUS_ADDRESS to pass_env of the corresponding tox.ini section. For example:

[testenv]
description = run the tests with pytest
package = wheel
wheel_build_env = .pkg
pass_env = 
    DBUS_SESSION_BUS_ADDRESS

This way tox will pass the environment variable DBUS_SESSION_BUS_ADDRESS which contains the session D-Bus address, to the process running the test.

[fohrloop]

The reason for the exception is now clear. User should always pass the DBUS_SESSION_BUS_ADDRESS environment variable to subprocess if using wakepy with dbus method in subprocesses. I'll close this one since there are these new tickets that should make usage of wakepy more convenient:

• Make it possible for the user to select the used method #28
• Make it possible to select what happens with errors #29

I'm also considering if there should be some automation trying to get the DBUS_SESSION_BUS_ADDRESS from the parent python process or the dbus-daemon process. but that seems a bit hacky.