Section substitution doesn't work in setenv

[barthisrael]

Issue

setenv variables are not available when running commands from a tox test environment, if the variables were defined using Substitution for values from other sections.

That seems some kind of caching issue, as during some debugging I found that removing this IF statement makes tox work as I would expect.

I'm providing a small reproducible below.
That reproducible defines 3 environment variables through setenv: one directly, and 2 through substitution.
Only the variable defined directly is available when running bash from commands, while the ones defined through substitution are not available.

If we check the output of tox config, we can see all of the 3 variables correctly defined. And as I mentioned above, if I remove that IF statement, making it always update the variables on environment_variables property call, then all the 3 variables are available when running bash.

Environment

Provide at least:

• OS: AlmaLinux 8.8

Output of pip list of the host Python, where tox is installed

$ pip list
Package       Version
------------- -------
cachetools    5.3.3
chardet       5.2.0
colorama      0.4.6
distlib       0.3.8
filelock      3.13.3
packaging     24.0
pip           24.0
platformdirs  4.2.0
pluggy        1.4.0
pyproject-api 1.6.1
setuptools    58.1.0
tomli         2.0.1
tox           4.14.2
virtualenv    20.25.1

Output of running tox

Output of tox -rvv

$ tox -rvv
py: 194 I find interpreter for spec PythonSpec(path=/var/lib/pgsql/.pyenv/versions/3.9.18/envs/tox_issue/bin/python3.9) [virtualenv/discovery/builtin.py:58]
py: 194 D discover exe for PythonInfo(spec=CPython3.9.18.final.0-64, exe=/var/lib/pgsql/.pyenv/versions/3.9.18/envs/tox_issue/bin/python3.9, platform=linux, version='3.9.18 (main, Feb 26 2024, 16:24:53) \n[GCC 8.5.0 20210514 (Red Hat 8.5.0-20)]', encoding_fs_io=utf-8-utf-8) in /var/lib/pgsql/.pyenv/versions/3.9.18 [virtualenv/discovery/py_info.py:438]
py: 195 D filesystem is case-sensitive [virtualenv/info.py:25]
py: 199 D got python info of %s from (PosixPath('/var/lib/pgsql/.pyenv/versions/3.9.18/bin/python3.9'), PosixPath('/var/lib/pgsql/.local/share/virtualenv/py_info/1/7a41df77756a29a17ad174a1eadcfcb87b684555bf31d80d33de969f26bf0026.json')) [virtualenv/app_data/via_disk_folder.py:131]
py: 200 I proposed PythonInfo(spec=CPython3.9.18.final.0-64, system=/var/lib/pgsql/.pyenv/versions/3.9.18/bin/python3.9, exe=/var/lib/pgsql/.pyenv/versions/3.9.18/envs/tox_issue/bin/python3.9, platform=linux, version='3.9.18 (main, Feb 26 2024, 16:24:53) \n[GCC 8.5.0 20210514 (Red Hat 8.5.0-20)]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:65]
py: 201 D accepted PythonInfo(spec=CPython3.9.18.final.0-64, system=/var/lib/pgsql/.pyenv/versions/3.9.18/bin/python3.9, exe=/var/lib/pgsql/.pyenv/versions/3.9.18/envs/tox_issue/bin/python3.9, platform=linux, version='3.9.18 (main, Feb 26 2024, 16:24:53) \n[GCC 8.5.0 20210514 (Red Hat 8.5.0-20)]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:67]
py: 232 I create virtual environment via CPython3Posix(dest=/var/lib/pgsql/tox_issue/.tox/py, clear=False, no_vcs_ignore=False, global=False) [virtualenv/run/session.py:50]
py: 232 D create folder /var/lib/pgsql/tox_issue/.tox/py/bin [virtualenv/util/path/_sync.py:12]
py: 233 D create folder /var/lib/pgsql/tox_issue/.tox/py/lib/python3.9/site-packages [virtualenv/util/path/_sync.py:12]
py: 233 D write /var/lib/pgsql/tox_issue/.tox/py/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:33]
py: 234 D 	home = /var/lib/pgsql/.pyenv/versions/3.9.18/bin [virtualenv/create/pyenv_cfg.py:38]
py: 234 D 	implementation = CPython [virtualenv/create/pyenv_cfg.py:38]
py: 234 D 	version_info = 3.9.18.final.0 [virtualenv/create/pyenv_cfg.py:38]
py: 234 D 	virtualenv = 20.25.1 [virtualenv/create/pyenv_cfg.py:38]
py: 234 D 	include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:38]
py: 234 D 	base-prefix = /var/lib/pgsql/.pyenv/versions/3.9.18 [virtualenv/create/pyenv_cfg.py:38]
py: 235 D 	base-exec-prefix = /var/lib/pgsql/.pyenv/versions/3.9.18 [virtualenv/create/pyenv_cfg.py:38]
py: 235 D 	base-executable = /var/lib/pgsql/.pyenv/versions/3.9.18/bin/python3.9 [virtualenv/create/pyenv_cfg.py:38]
py: 236 D symlink /var/lib/pgsql/.pyenv/versions/3.9.18/bin/python3.9 to /var/lib/pgsql/tox_issue/.tox/py/bin/python [virtualenv/util/path/_sync.py:32]
py: 236 D create virtualenv import hook file /var/lib/pgsql/tox_issue/.tox/py/lib/python3.9/site-packages/_virtualenv.pth [virtualenv/create/via_global_ref/api.py:91]
py: 237 D create /var/lib/pgsql/tox_issue/.tox/py/lib/python3.9/site-packages/_virtualenv.py [virtualenv/create/via_global_ref/api.py:94]
py: 237 D ============================== target debug ============================== [virtualenv/run/session.py:52]
py: 237 D debug via /var/lib/pgsql/tox_issue/.tox/py/bin/python /var/lib/pgsql/.pyenv/versions/3.9.18/envs/tox_issue/lib/python3.9/site-packages/virtualenv/create/debug.py [virtualenv/create/creator.py:200]
py: 237 D {
  "sys": {
    "executable": "/var/lib/pgsql/tox_issue/.tox/py/bin/python",
    "_base_executable": "/var/lib/pgsql/tox_issue/.tox/py/bin/python",
    "prefix": "/var/lib/pgsql/tox_issue/.tox/py",
    "base_prefix": "/var/lib/pgsql/.pyenv/versions/3.9.18",
    "real_prefix": null,
    "exec_prefix": "/var/lib/pgsql/tox_issue/.tox/py",
    "base_exec_prefix": "/var/lib/pgsql/.pyenv/versions/3.9.18",
    "path": [
      "/var/lib/pgsql/.pyenv/versions/3.9.18/lib/python39.zip",
      "/var/lib/pgsql/.pyenv/versions/3.9.18/lib/python3.9",
      "/var/lib/pgsql/.pyenv/versions/3.9.18/lib/python3.9/lib-dynload",
      "/var/lib/pgsql/tox_issue/.tox/py/lib/python3.9/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.9.18 (main, Feb 26 2024, 16:24:53) \n[GCC 8.5.0 20210514 (Red Hat 8.5.0-20)]",
  "makefile_filename": "/var/lib/pgsql/.pyenv/versions/3.9.18/lib/python3.9/config-3.9-x86_64-linux-gnu/Makefile",
  "os": "<module 'os' from '/var/lib/pgsql/.pyenv/versions/3.9.18/lib/python3.9/os.py'>",
  "site": "<module 'site' from '/var/lib/pgsql/.pyenv/versions/3.9.18/lib/python3.9/site.py'>",
  "datetime": "<module 'datetime' from '/var/lib/pgsql/.pyenv/versions/3.9.18/lib/python3.9/datetime.py'>",
  "math": "<module 'math' from '/var/lib/pgsql/.pyenv/versions/3.9.18/lib/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so'>",
  "json": "<module 'json' from '/var/lib/pgsql/.pyenv/versions/3.9.18/lib/python3.9/json/__init__.py'>"
} [virtualenv/run/session.py:53]
py: 272 I add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/var/lib/pgsql/.local/share/virtualenv) [virtualenv/run/session.py:57]
py: 280 D install pip from wheel /var/lib/pgsql/.pyenv/versions/3.9.18/envs/tox_issue/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/pip-24.0-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:49]
py: 282 D install setuptools from wheel /var/lib/pgsql/.pyenv/versions/3.9.18/envs/tox_issue/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/setuptools-69.1.0-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:49]
py: 283 D install wheel from wheel /var/lib/pgsql/.pyenv/versions/3.9.18/envs/tox_issue/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/wheel-0.42.0-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:49]
py: 287 D copy directory /var/lib/pgsql/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-24.0-py3-none-any/pip to /var/lib/pgsql/tox_issue/.tox/py/lib/python3.9/site-packages/pip [virtualenv/util/path/_sync.py:40]
py: 289 D copy directory /var/lib/pgsql/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.42.0-py3-none-any/wheel to /var/lib/pgsql/tox_issue/.tox/py/lib/python3.9/site-packages/wheel [virtualenv/util/path/_sync.py:40]
py: 292 D copy /var/lib/pgsql/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-69.1.0-py3-none-any/distutils-precedence.pth to /var/lib/pgsql/tox_issue/.tox/py/lib/python3.9/site-packages/distutils-precedence.pth [virtualenv/util/path/_sync.py:40]
py: 295 D copy directory /var/lib/pgsql/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-69.1.0-py3-none-any/_distutils_hack to /var/lib/pgsql/tox_issue/.tox/py/lib/python3.9/site-packages/_distutils_hack [virtualenv/util/path/_sync.py:40]
py: 301 D copy directory /var/lib/pgsql/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-69.1.0-py3-none-any/pkg_resources to /var/lib/pgsql/tox_issue/.tox/py/lib/python3.9/site-packages/pkg_resources [virtualenv/util/path/_sync.py:40]
py: 336 D copy directory /var/lib/pgsql/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.42.0-py3-none-any/wheel-0.42.0.dist-info to /var/lib/pgsql/tox_issue/.tox/py/lib/python3.9/site-packages/wheel-0.42.0.dist-info [virtualenv/util/path/_sync.py:40]
py: 348 D copy /var/lib/pgsql/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.42.0-py3-none-any/wheel-0.42.0.virtualenv to /var/lib/pgsql/tox_issue/.tox/py/lib/python3.9/site-packages/wheel-0.42.0.virtualenv [virtualenv/util/path/_sync.py:40]
py: 354 D generated console scripts wheel3 wheel3.9 wheel-3.9 wheel [virtualenv/seed/embed/via_app_data/pip_install/base.py:43]
py: 389 D copy directory /var/lib/pgsql/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-69.1.0-py3-none-any/setuptools to /var/lib/pgsql/tox_issue/.tox/py/lib/python3.9/site-packages/setuptools [virtualenv/util/path/_sync.py:40]
py: 569 D copy directory /var/lib/pgsql/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-69.1.0-py3-none-any/setuptools-69.1.0.dist-info to /var/lib/pgsql/tox_issue/.tox/py/lib/python3.9/site-packages/setuptools-69.1.0.dist-info [virtualenv/util/path/_sync.py:40]
py: 579 D copy /var/lib/pgsql/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-69.1.0-py3-none-any/setuptools-69.1.0.virtualenv to /var/lib/pgsql/tox_issue/.tox/py/lib/python3.9/site-packages/setuptools-69.1.0.virtualenv [virtualenv/util/path/_sync.py:40]
py: 580 D generated console scripts  [virtualenv/seed/embed/via_app_data/pip_install/base.py:43]
py: 725 D copy directory /var/lib/pgsql/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-24.0-py3-none-any/pip-24.0.dist-info to /var/lib/pgsql/tox_issue/.tox/py/lib/python3.9/site-packages/pip-24.0.dist-info [virtualenv/util/path/_sync.py:40]
py: 731 D copy /var/lib/pgsql/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-24.0-py3-none-any/pip-24.0.virtualenv to /var/lib/pgsql/tox_issue/.tox/py/lib/python3.9/site-packages/pip-24.0.virtualenv [virtualenv/util/path/_sync.py:40]
py: 733 D generated console scripts pip3.9 pip pip-3.9 pip3 [virtualenv/seed/embed/via_app_data/pip_install/base.py:43]
py: 734 I add activators for Bash, CShell, Fish, Nushell, PowerShell, Python [virtualenv/run/session.py:63]
py: 737 D write /var/lib/pgsql/tox_issue/.tox/py/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:33]
py: 737 D 	home = /var/lib/pgsql/.pyenv/versions/3.9.18/bin [virtualenv/create/pyenv_cfg.py:38]
py: 737 D 	implementation = CPython [virtualenv/create/pyenv_cfg.py:38]
py: 737 D 	version_info = 3.9.18.final.0 [virtualenv/create/pyenv_cfg.py:38]
py: 737 D 	virtualenv = 20.25.1 [virtualenv/create/pyenv_cfg.py:38]
py: 737 D 	include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:38]
py: 737 D 	base-prefix = /var/lib/pgsql/.pyenv/versions/3.9.18 [virtualenv/create/pyenv_cfg.py:38]
py: 738 D 	base-exec-prefix = /var/lib/pgsql/.pyenv/versions/3.9.18 [virtualenv/create/pyenv_cfg.py:38]
py: 738 D 	base-executable = /var/lib/pgsql/.pyenv/versions/3.9.18/bin/python3.9 [virtualenv/create/pyenv_cfg.py:38]
  py: OK (0.56 seconds)
  congratulations :) (0.64 seconds)

Minimal example

• tox.ini content:

[tox]

[variables]
postgres_variant =
    pg: PG_VARIANT = PostgreSQL
    rds: PG_VARIANT = RDS

postgres_version =
    12: PG_MAJOR = 12
    13: PG_MAJOR = 13
    14: PG_MAJOR = 14
    15: PG_MAJOR = 15
    16: PG_MAJOR = 16

[testenv:{pg,rds}-{12,13,14,15,16}-test]
description = Issue reproducer

setenv =
    {[variables]postgres_variant}
    {[variables]postgres_version}
    CONTAINER_IMAGE = test:{env:PG_VARIANT}-{env:PG_MAJOR}

commands =
    bash -c "env | grep -P 'PG_(VARIANT|MAJOR)|CONTAINER_IMAGE'"

allowlist_externals =
    bash

• Output of tox run:

$ tox run -e rds-13-test
rds-13-test: commands[0]> bash -c 'env | grep -P '"'"'PG_(VARIANT|MAJOR)|CONTAINER_IMAGE'"'"''
CONTAINER_IMAGE=test:RDS-13
  rds-13-test: OK (0.04=setup[0.03]+cmd[0.01] seconds)
  congratulations :) (0.11 seconds)

• Output of tox config:

$ tox config -e rds-13-test
[testenv:rds-13-test]
type = VirtualEnvRunner
set_env =
  CONTAINER_IMAGE=test:RDS-13
  PG_MAJOR=13
  PG_VARIANT=RDS
  PIP_DISABLE_PIP_VERSION_CHECK=1
  PYTHONHASHSEED=3633014905
  PYTHONIOENCODING=utf-8
base = testenv
runner = virtualenv
description = Issue reproducer
depends =
env_name = rds-13-test
labels =
env_dir = /var/lib/pgsql/tox_issue/.tox/rds-13-test
env_tmp_dir = /var/lib/pgsql/tox_issue/.tox/rds-13-test/tmp
env_log_dir = /var/lib/pgsql/tox_issue/.tox/rds-13-test/log
suicide_timeout = 0.0
interrupt_timeout = 0.3
terminate_timeout = 0.2
platform =
pass_env =
  CC
  CCSHARED
  CFLAGS
  CPPFLAGS
  CURL_CA_BUNDLE
  CXX
  FORCE_COLOR
  HOME
  LANG
  LANGUAGE
  LDFLAGS
  LD_LIBRARY_PATH
  NO_COLOR
  PIP_*
  PKG_CONFIG
  PKG_CONFIG_PATH
  PKG_CONFIG_SYSROOT_DIR
  REQUESTS_CA_BUNDLE
  SSL_CERT_FILE
  TERM
  TMPDIR
  VIRTUALENV_*
  http_proxy
  https_proxy
  no_proxy
parallel_show_output = False
recreate = False
allowlist_externals = bash
pip_pre = False
install_command = python -I -m pip install '{packages}'
constrain_package_deps = False
use_frozen_constraints = False
list_dependencies_command = python -m pip freeze --all
commands_pre =
commands = bash -c 'env | grep -P '"'"'PG_(VARIANT|MAJOR)|CONTAINER_IMAGE'"'"''
commands_post =
change_dir = /var/lib/pgsql/tox_issue
args_are_paths = True
ignore_errors = False
ignore_outcome = False
base_python = /var/lib/pgsql/.pyenv/versions/3.9.18/envs/tox_issue/bin/python3.9
env_site_packages_dir = /var/lib/pgsql/tox_issue/.tox/rds-13-test/lib/python3.9/site-packages
env_bin_dir = /var/lib/pgsql/tox_issue/.tox/rds-13-test/bin
env_python = /var/lib/pgsql/tox_issue/.tox/rds-13-test/bin/python
py_dot_ver = 3.9
py_impl = cpython
deps =
system_site_packages = False
always_copy = False
download = False
skip_install = False
use_develop = False
package = skip

[jugmac00]

Thanks for the bug report and the reproducer.

Both maintainers are currently very limited on engineering time, and also in general we welcome contributions from the user base. This means we won't be able to work on that issue in the near future, but we would be happy to help anybody to land a PR to fix it.

[barthisrael]

You are welcome, and thanks for the heads up!

FWIW we worked around that by not using the substitution feature in that case for now.

I was willing to dig into the issue when I bumped into it, but I was also missing some cycles to work on it, unfortunately :/
I decided to bring it to your (and other people's) attention, just in case someone happened to be available to work on it -- and to avoid losing track of it.