Skip to content

Commit

Permalink
Added clear_resolver method to OmegaConf (#770)
Browse files Browse the repository at this point in the history
  • Loading branch information
sugatoray authored Aug 5, 2021
1 parent aa62b48 commit c861d00
Show file tree
Hide file tree
Showing 4 changed files with 133 additions and 12 deletions.
13 changes: 7 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ pre-commit will verify your code lints cleanly when you commit. You can use `git
OmegaConf is compatible with Python 3.6.4 and newer. Unfortunately Mac comes with older versions.

One way to install multiple Python versions on Mac to to use pyenv.
The instructions [here](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/MAC_SETUP.md)
will provide full details. It shows how to use pyenv on mac to install multiple versions of Python and have
The instructions [here](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/MAC_SETUP.md)
will provide full details. It shows how to use pyenv on mac to install multiple versions of Python and have
pyenv make specific versions available in specific directories automatically.
This plays well with Conda, which supports a single Python version. Pyenv will provide the versions not installed by Conda (which are used when running nox).

Expand Down Expand Up @@ -49,23 +49,24 @@ Sessions defined in /home/omry/dev/omegaconf/noxfile.py:
* test_jupyter_notebook-3.8
* test_jupyter_notebook-3.9
```

To run a specific session use `-s`, for example `nox -s lint` will run linting


OmegaConf is formatted with black, to format your code automatically use `black .`

Imports are sorted using isort, use `isort .` to sort all imports prior to pushing.
Imports are sorted using isort, use `isort .` to sort all imports prior to pushing.

To build the docs execute `nox -s docs` or `make`(inside docs folder). Make gives you different options, for example, you can build the docs as html files with `make html`. Once the docs are built you can open `index.html` in the build directory to view the generated docs with your browser.

### Modifying Jupyter notebook

In order to change the Jupyter notebook you first need to open it with `jupyter notebook`.
Change the cell you want and then, execute it so the expected output is shown.
Note that the output after you execute the cell is saved as expected ouput for further
Change the cell you want and then, execute it so the expected output is shown.
Note that the output after you execute the cell is saved as expected ouput for further
testing.

In case that the in[number] of cells aren't in order you should go to the
In case that the in[number] of cells aren't in order you should go to the
kernel in the toolbar and restart it.


Expand Down
76 changes: 70 additions & 6 deletions docs/source/custom_resolvers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ Built-in resolvers

.. _oc.env:

oc.env
oc.env
^^^^^^

Access to environment variables is supported using ``oc.env``:
Expand All @@ -176,7 +176,7 @@ Input YAML file:

You can specify a default value to use in case the environment variable is not set.
In such a case, the default value is converted to a string using ``str(default)``,
unless it is ``null`` (representing Python ``None``) - in which case ``None`` is returned.
unless it is ``null`` (representing Python ``None``) - in which case ``None`` is returned.

The following example falls back to default passwords when ``DB_PASSWORD`` is not defined:

Expand Down Expand Up @@ -221,7 +221,7 @@ oc.create
... "dict_config_env": "${oc.create:${oc.env:YAML_ENV}}",
... }
... )
>>> os.environ["YAML_ENV"] = "A: 10\nb: 20\nC: ${.A}"
>>> os.environ["YAML_ENV"] = "A: 10\nb: 20\nC: ${.A}"
>>> show(cfg.plain_dict) # `make_dict` returns a Python dict
type: dict, value: {'a': 10}
>>> show(cfg.dict_config) # `oc.create` converts it to DictConfig
Expand All @@ -246,11 +246,11 @@ It takes two parameters:
.. doctest::

>>> conf = OmegaConf.create({
... "rusty_key": "${oc.deprecated:shiny_key}",
... "rusty_key": "${oc.deprecated:shiny_key}",
... "custom_msg": "${oc.deprecated:shiny_key, 'Use $NEW_KEY'}",
... "shiny_key": 10
... })
>>> # Accessing rusty_key will issue a deprecation warning
>>> # Accessing rusty_key will issue a deprecation warning
>>> # and return the new value automatically
>>> warning = "'rusty_key' is deprecated. Change your" \
... " code and config to use 'shiny_key'"
Expand Down Expand Up @@ -348,7 +348,7 @@ Another scenario where ``oc.select`` can be useful is if you want to select a mi
... "with_default": "${oc.select:missing,default value}",
... }
... )
...
...
>>> print(cfg.interpolation)
Traceback (most recent call last):
...
Expand Down Expand Up @@ -391,3 +391,67 @@ as interpolations), and they return a ``ListConfig`` that contains keys or value
>>> show(cfg.ips)
type: ListConfig, value: ['${workers.node3}', '${workers.node7}']
>>> assert cfg.ips == ["10.0.0.2", "10.0.0.9"]

.. _clearing_resolvers:

Clearing/removing resolvers
---------------------------

.. _clear_resolvers:

clear_resolvers
^^^^^^^^^^^^^^^

Use ``OmegaConf.clear_resolvers()`` to remove all resolvers except the built-in resolvers (like ``oc.env`` etc).

.. code-block:: python
def clear_resolvers() -> None
In the following example, first we register a new custom resolver ``str.lower``, and then clear all
custom resolvers.
.. doctest::
>>> # register a new resolver: str.lower
>>> OmegaConf.register_new_resolver(
... name='str.lower',
... resolver=lambda x: str(x).lower(),
... )
>>> # check if resolver exists (after adding, before removal)
>>> OmegaConf.has_resolver("str.lower")
True
>>> # clear all custom-resolvers
>>> OmegaConf.clear_resolvers()
>>> # check if resolver exists (after removal)
>>> OmegaConf.has_resolver("str.lower")
False
>>> # default resolvers are not affected
>>> OmegaConf.has_resolver("oc.env")
True
.. _clear_resolver:

clear_resolver
^^^^^^^^^^^^^^

Use ``OmegaConf.clear_resolver()`` to remove a single resolver (including built-in resolvers).

.. code-block:: python
def clear_resolver(name: str) -> bool
``OmegaConf.clear_resolver()`` returns True if the resolver was found and removed, and False otherwise.
Here is an example.
.. doctest::
>>> OmegaConf.has_resolver("oc.env")
True
>>> # This will remove the default resolver: oc.env
>>> OmegaConf.clear_resolver("oc.env")
True
>>> OmegaConf.has_resolver("oc.env")
False
17 changes: 17 additions & 0 deletions omegaconf/omegaconf.py
Original file line number Diff line number Diff line change
Expand Up @@ -467,6 +467,23 @@ def clear_resolvers() -> None:
BaseContainer._resolvers = {}
register_default_resolvers()

@classmethod
def clear_resolver(cls, name: str) -> bool:
"""Clear(remove) any resolver only if it exists. Returns a bool: True if resolver is removed and False if not removed.
.. warning:
This method can remove deafult resolvers as well.
:param name: Name of the resolver.
:return: A bool (``True`` if resolver is removed, ``False`` if not found before removing).
"""
if cls.has_resolver(name):
BaseContainer._resolvers.pop(name)
return True
else:
# return False if resolver does not exist
return False

@staticmethod
def get_cache(conf: BaseContainer) -> Dict[str, Any]:
return conf._metadata.resolver_cache
Expand Down
39 changes: 39 additions & 0 deletions tests/test_omegaconf.py
Original file line number Diff line number Diff line change
Expand Up @@ -518,3 +518,42 @@ def test_resolve(cfg: Any, expected: Any) -> None:
def test_resolve_invalid_input() -> None:
with raises(ValueError):
OmegaConf.resolve("aaa") # type: ignore


@mark.parametrize(
("register_resolver_params", "name", "expected"),
[
param(
dict(
name="iamnew",
resolver=lambda x: str(x).lower(),
use_cache=False,
replace=False,
),
"iamnew",
dict(pre_clear=True, result=True),
id="remove-new-custom-resolver",
),
param(
dict(),
"oc.env",
dict(pre_clear=True, result=True),
id="remove-default-resolver",
),
param(
dict(),
"idonotexist",
dict(pre_clear=False, result=False),
id="remove-nonexistent-resolver",
),
],
)
def test_clear_resolver(
restore_resolvers: Any, register_resolver_params: Any, name: str, expected: Any
) -> None:
if register_resolver_params:
OmegaConf.register_new_resolver(**register_resolver_params)
assert expected["pre_clear"] == OmegaConf.has_resolver(name)

assert OmegaConf.clear_resolver(name) == expected["result"]
assert not OmegaConf.has_resolver(name)

0 comments on commit c861d00

Please sign in to comment.