Fix Checkbox configuration value resolution and add Metabox scenarios to test it

[pieqq]

Description

This PR provides several changes:

1. Refactor [environment] section testing using Metabox scenarios
2. Add Metabox scenarios to test [test selection] section of Checkbox config
3. Fix Checkbox config resolution order, namely: handle config from launchers last, so it takes precedence over anything else.
4. Update documentation based on resolution order source code.

Please check individual commit messages for more information.

Resolved issues

CHECKBOX-436

Tests

# Running the config-related scenarios using Checkbox local
$ metabox local-source-22.04.py --do-not-dispose --tag config --log INFO
13:51:09 | INFO     Config: No 'uri' element defined.
13:51:09 | INFO     Config: Setting 'uri' to '/home/pieq/dev/work/checkbox'.
13:51:09 | INFO     Including scenario tag(s): config
13:51:22 | INFO     Starting scenario: config.environment.CheckboxConfXDG
13:51:23 | SUCCESS  [local][('jammy',)] config.environment.CheckboxConfXDG scenario has passed.
13:51:32 | INFO     Starting scenario: config.environment.CheckboxConfLocalHome
13:51:33 | SUCCESS  [local][('jammy',)] config.environment.CheckboxConfLocalHome scenario has passed.
13:51:44 | INFO     Starting scenario: config.environment.CheckboxConfLauncher
13:51:45 | SUCCESS  [local][('jammy',)] config.environment.CheckboxConfLauncher scenario has passed.
13:52:00 | INFO     Starting scenario: config.environment.CheckboxConfLocalHomePrecedence
13:52:01 | SUCCESS  [local][('jammy',)] config.environment.CheckboxConfLocalHomePrecedence scenario has passed.
13:52:11 | INFO     Starting scenario: config.environment.CheckboxConfLocalResolutionOrder
13:52:12 | SUCCESS  [local][('jammy',)] config.environment.CheckboxConfLocalResolutionOrder scenario has passed.
13:52:22 | INFO     Starting scenario: config.test_selection.TestSelectionDefault
13:52:22 | SUCCESS  [local][('jammy',)] config.test_selection.TestSelectionDefault scenario has passed.
13:52:38 | INFO     Starting scenario: config.test_selection.TestSelectionForced
13:52:38 | SUCCESS  [local][('jammy',)] config.test_selection.TestSelectionForced scenario has passed.
13:52:52 | INFO     Starting scenario: config.test_selection.TestSelectionExcludedJob
13:52:53 | SUCCESS  [local][('jammy',)] config.test_selection.TestSelectionExcludedJob scenario has passed.
13:53:05 | INFO     Starting scenario: config.test_selection.LocalTestSelectionResolution
13:53:06 | SUCCESS  [local][('jammy',)] config.test_selection.LocalTestSelectionResolution scenario has passed.
--------------------------------------------------------------------------------
13:53:08 | SUCCESS  Ran 9 scenarios in 114.430s

# Running the config-related scenarios using Checkbox remote
$ metabox remote-source-22.04.py --do-not-dispose --tag config --log INFO
13:53:21 | INFO     Config: No 'uri' element defined.
13:53:21 | INFO     Config: Setting 'uri' to '/home/pieq/dev/work/checkbox'.
13:53:21 | INFO     Config: No 'uri' element defined.
13:53:21 | INFO     Config: Setting 'uri' to '/home/pieq/dev/work/checkbox'.
13:53:22 | INFO     Including scenario tag(s): config
13:53:39 | INFO     Starting scenario: config.environment.CheckboxConfXDG
13:53:42 | SUCCESS  [remote][('jammy', 'jammy')] config.environment.CheckboxConfXDG scenario has passed.
13:54:07 | INFO     Starting scenario: config.environment.CheckboxConfRemoteHome
13:54:09 | SUCCESS  [remote][('jammy', 'jammy')] config.environment.CheckboxConfRemoteHome scenario has passed.
13:54:34 | INFO     Starting scenario: config.environment.CheckboxConfLauncher
13:54:36 | SUCCESS  [remote][('jammy', 'jammy')] config.environment.CheckboxConfLauncher scenario has passed.
13:54:56 | INFO     Starting scenario: config.environment.CheckboxConfLauncherPrecedence
13:54:58 | SUCCESS  [remote][('jammy', 'jammy')] config.environment.CheckboxConfLauncherPrecedence scenario has passed.
13:55:20 | INFO     Starting scenario: config.environment.CheckboxConfRemoteServiceResolutionOrder
13:55:22 | SUCCESS  [remote][('jammy', 'jammy')] config.environment.CheckboxConfRemoteServiceResolutionOrder scenario has passed.
13:55:45 | INFO     Starting scenario: config.test_selection.TestSelectionDefault
13:55:46 | SUCCESS  [remote][('jammy', 'jammy')] config.test_selection.TestSelectionDefault scenario has passed.
13:56:09 | INFO     Starting scenario: config.test_selection.TestSelectionForced
13:56:10 | SUCCESS  [remote][('jammy', 'jammy')] config.test_selection.TestSelectionForced scenario has passed.
13:56:32 | INFO     Starting scenario: config.test_selection.TestSelectionExcludedJob
13:56:33 | SUCCESS  [remote][('jammy', 'jammy')] config.test_selection.TestSelectionExcludedJob scenario has passed.
13:57:00 | INFO     Starting scenario: config.test_selection.RemoteTestSelectionResolution
13:57:02 | SUCCESS  [remote][('jammy', 'jammy')] config.test_selection.RemoteTestSelectionResolution scenario has passed.
--------------------------------------------------------------------------------
13:57:13 | SUCCESS  Ran 9 scenarios in 226.775s

[Hook25]

I have found a corner case where this (seems) to not work.

Creating checkbox.conf in /etc/xdg with the following:

[test selection]
forced = yes

Checkbox remote seems to ignore the configuration. Is this related to what was changed here?

[yphus]

For the record, the ConfigParser read() works this way:

It is possible to read several configurations into a single ConfigParser, where the most recently added configuration has the highest priority. Any conflicting keys are taken from the more recent configuration while the previously existing keys are retained.

[yphus]

LGTM, thanks for the config test directory reorganization, it helps a lot to separate the various files involved to validate each section.