Config created by aiida_profile fixture does not pass validation

[greschd]

Describe the bug

The aiida_profile pytest fixture automatically creates a configuration file. However, that file is malformed w.r.t. the schema validation. Specifically, the broker_* keys are set to null.

Steps to reproduce

Running the following test produces the error:

from aiida.manage.configuration import get_config
from aiida.manage.configuration.config import Config


def test_profile(aiida_profile):
    config = Config.from_file(get_config().filepath)

Error message:

E           aiida.manage.configuration.config.ConfigValidationError: Validation Error: /tmp/tmpi8csz00a/.aiida/config.json:/profiles/test_profile/broker_protocol: None is not of type 'string'
E             schema:
E             {'description': 'Protocol for connecting to the RabbitMQ server', 'type': 'string', 'enum': ['amqp', 'amqps'], 'default': 'amqp'}

Full configuration file:

{
    "CONFIG_VERSION": {
        "CURRENT": 5,
        "OLDEST_COMPATIBLE": 5
    },
    "profiles": {
        "test_profile": {
            "AIIDADB_ENGINE": "postgresql_psycopg2",
            "AIIDADB_BACKEND": "django",
            "AIIDADB_PORT": 50883,
            "AIIDADB_HOST": "localhost",
            "AIIDADB_NAME": "aiida_db",
            "AIIDADB_USER": "aiida",
            "AIIDADB_PASS": "aiida_pw",
            "broker_protocol": null,
            "broker_username": null,
            "broker_password": null,
            "broker_host": null,
            "broker_port": null,
            "broker_virtual_host": null,
            "AIIDADB_REPOSITORY_URI": "file:///tmp/tmp3g0uw68j/test_repo",
            "PROFILE_UUID": "6e41151e0b34449a97315a46e37ec39e",
            "options": {},
            "default_user_email": "tests@aiida.mail"
        }
    },
    "default_profile": "test_profile"
}

expand for full pytest log

============================= test session starts ==============================
platform linux -- Python 3.7.10, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /home/a-dogres/programming/test/aiida_pytest_test
collected 1 item

test_bug.py F                                                            [100%]

=================================== FAILURES ===================================
_________________________________ test_profile _________________________________

config = {'CONFIG_VERSION': {'CURRENT': 5, 'OLDEST_COMPATIBLE': 5}, 'default_profile': 'test_profile', 'profiles': {'test_profi...ND': 'django', 'AIIDADB_ENGINE': 'postgresql_psycopg2', 'AIIDADB_HOST': 'localhost', 'AIIDADB_NAME': 'aiida_db', ...}}}
filepath = '/tmp/tmp6s7povlb/.aiida/config.json'

    @staticmethod
    def validate(config: dict, filepath: Optional[str] = None):
        """Validate a configuration dictionary."""
        try:
>           jsonschema.validate(instance=config, schema=config_schema())

../../../.virtualenvs/aiida-pytest-check/lib/python3.7/site-packages/aiida/manage/configuration/config.py:128: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

instance = {'CONFIG_VERSION': {'CURRENT': 5, 'OLDEST_COMPATIBLE': 5}, 'default_profile': 'test_profile', 'profiles': {'test_profi...ND': 'django', 'AIIDADB_ENGINE': 'postgresql_psycopg2', 'AIIDADB_HOST': 'localhost', 'AIIDADB_NAME': 'aiida_db', ...}}}
schema = {'$schema': 'http://json-schema.org/draft-07/schema', 'definitions': {'options': {'properties': {'autofill.user.email'...ption': 'Configured profiles', 'patternProperties': {'.+': {'$ref': '#/definitions/profile'}}, 'type': 'object'}}, ...}
cls = <class 'jsonschema.validators.create.<locals>.Validator'>, args = ()
kwargs = {}
validator = <jsonschema.validators.create.<locals>.Validator object at 0x7f5237ef8950>
error = <ValidationError: "None is not of type 'string'">

    def validate(instance, schema, cls=None, *args, **kwargs):
        """
        Validate an instance under the given schema.
    
            >>> validate([2, 3, 4], {"maxItems": 2})
            Traceback (most recent call last):
                ...
            ValidationError: [2, 3, 4] is too long
    
        :func:`validate` will first verify that the provided schema is
        itself valid, since not doing so can lead to less obvious error
        messages and fail in less obvious or consistent ways.
    
        If you know you have a valid schema already, especially if you
        intend to validate multiple instances with the same schema, you
        likely would prefer using the `IValidator.validate` method directly
        on a specific validator (e.g. ``Draft7Validator.validate``).
    
    
        Arguments:
    
            instance:
    
                The instance to validate
    
            schema:
    
                The schema to validate with
    
            cls (IValidator):
    
                The class that will be used to validate the instance.
    
        If the ``cls`` argument is not provided, two things will happen
        in accordance with the specification. First, if the schema has a
        :validator:`$schema` property containing a known meta-schema [#]_
        then the proper validator will be used. The specification recommends
        that all schemas contain :validator:`$schema` properties for this
        reason. If no :validator:`$schema` property is found, the default
        validator class is the latest released draft.
    
        Any other provided positional and keyword arguments will be passed
        on when instantiating the ``cls``.
    
        Raises:
    
            `jsonschema.exceptions.ValidationError` if the instance
                is invalid
    
            `jsonschema.exceptions.SchemaError` if the schema itself
                is invalid
    
        .. rubric:: Footnotes
        .. [#] known by a validator registered with
            `jsonschema.validators.validates`
        """
        if cls is None:
            cls = validator_for(schema)
    
        cls.check_schema(schema)
        validator = cls(schema, *args, **kwargs)
        error = exceptions.best_match(validator.iter_errors(instance))
        if error is not None:
>           raise error
E           jsonschema.exceptions.ValidationError: None is not of type 'string'
E           
E           Failed validating 'type' in schema['properties']['profiles']['patternProperties']['.+']['properties']['broker_protocol']:
E               {'default': 'amqp',
E                'description': 'Protocol for connecting to the RabbitMQ server',
E                'enum': ['amqp', 'amqps'],
E                'type': 'string'}
E           
E           On instance['profiles']['test_profile']['broker_protocol']:
E               None

../../../.virtualenvs/aiida-pytest-check/lib/python3.7/site-packages/jsonschema/validators.py:934: ValidationError

During handling of the above exception, another exception occurred:

aiida_profile = <aiida.manage.tests.TestManager object at 0x7f5255765f50>

    def test_profile(aiida_profile):
>       config = Config.from_file(get_config().filepath)

test_bug.py:6: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../.virtualenvs/aiida-pytest-check/lib/python3.7/site-packages/aiida/manage/configuration/config.py:98: in from_file
    config = Config(filepath, check_and_migrate_config(config))
../../../.virtualenvs/aiida-pytest-check/lib/python3.7/site-packages/aiida/manage/configuration/config.py:146: in __init__
    self.validate(config, filepath)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

config = {'CONFIG_VERSION': {'CURRENT': 5, 'OLDEST_COMPATIBLE': 5}, 'default_profile': 'test_profile', 'profiles': {'test_profi...ND': 'django', 'AIIDADB_ENGINE': 'postgresql_psycopg2', 'AIIDADB_HOST': 'localhost', 'AIIDADB_NAME': 'aiida_db', ...}}}
filepath = '/tmp/tmp6s7povlb/.aiida/config.json'

    @staticmethod
    def validate(config: dict, filepath: Optional[str] = None):
        """Validate a configuration dictionary."""
        try:
            jsonschema.validate(instance=config, schema=config_schema())
        except jsonschema.ValidationError as error:
            raise ConfigValidationError(
>               message=error.message, keypath=error.path, schema=error.schema, filepath=filepath
            )
E           aiida.manage.configuration.config.ConfigValidationError: Validation Error: /tmp/tmp6s7povlb/.aiida/config.json:/profiles/test_profile/broker_protocol: None is not of type 'string'
E             schema:
E             {'description': 'Protocol for connecting to the RabbitMQ server', 'type': 'string', 'enum': ['amqp', 'amqps'], 'default': 'amqp'}

../../../.virtualenvs/aiida-pytest-check/lib/python3.7/site-packages/aiida/manage/configuration/config.py:131: ConfigValidationError
------------------------------ Captured log setup ------------------------------
DEBUG    pgsu:__init__.py:141 Trying to connect via "psycopg2"...
=========================== short test summary info ============================
FAILED test_bug.py::test_profile - aiida.manage.configuration.config.ConfigVa...
============================== 1 failed in 5.34s ===============================

Expected behavior

The created config file should conform to the schema, or the schema should allow null and fall back to the default value.

Your environment

• Operating system: Ubuntu 18.04 on WSL2
• Python version: 3.7.10
• aiida-core version: 1.6.0

Additional context

Encountered while running tests with aiida-pytest, which executes verdi commands while the test profile is active. These failed due to the config validation error.

Tagging @chrisjsewell @ltalirz for input on what the expected behavior should be here.

[chrisjsewell]

Ah annoying that didn't get picked up in any of the aiida-core tests. But anyhow I would say that null should not be allowed, and they should just be omitted to signal use of default values.