Generating a list of multiple configs from inside a config group

[DirkKuhn]

Hi,

I really like hydra and hydra_zen. Unfortunately, I have a hard time applying them to my specific problem.
I am currently trying to write a config in a config group which generates a list of configs stored in the same group.
Basically, what I want is similar to facebookresearch/hydra#1939 (comment) with the difference that the config should be written one level lower inside the config group.
I was able to solve this problem with yaml files and a package directive in pure hydra, but I am unable to translate it to hydra_zen.

Let me illustrate the problem with a small example. I have a class Item

class Item:
  def __init__(self, msg: str):
    self.msg = msg

  def __repr__(self) -> str:
    return f'Item(msg={self.msg})'

and a class Repository

class Repository:
    def __init__(self, items: Item | Sequence[Item]):
        self.items = items
    
    def __repr__(self) -> str:
        return f'Repository(items={self.items})'

The Repository can either be configured with a single Item or with a Sequence of Items.
I would like the config to be of the form:

repository:
  _target_: Repository
  items:
    - _target_: Item
      msg: 'a'
    - _target_: Item
      msg: 'b'

First, I create a config group repository/items, to store the configs of multiple single Items

items_store = store(group='repository/items')
items_store(Item, msg='a', name='item_a')
items_store(Item, msg='b', name='item_b')

Then comes the hard part. I try to write a config in the group repository/items which generates the list

- _target_: Item
  msg: 'a'
- _target_: Item
  msg: 'b'

using the configs item_a and item_b.

items_store(
    make_config(
        items='${oc.dict.values:repository.items._items_dict}',
        hydra_defaults=[
            'item_a@_items_dict.item1',
            'item_b@_items_dict.item2',
            '_self_'
        ],
        zen_meta={'_items_dict': MISSING}
    ),
    package='_global_.repository',
    name='default'
)

The idea is to relocate item_a and item_b in a dict repository.items._items_dict and then generate a list using oc.dict.values which should be placed under repository.items using the package directive. In pure hydra the following worked:

# @package _global_.repository
defaults:
  - item_a@_items_dict.item1
  - item_b@_items_dict.item2
  - _self_

items: ${oc.dict.values:._items_dict}

I should maybe also explain why I want this. This way I could easily select either a single Item or the list of all Items and I could add other lists which might include only some items and everything without duplication of the config of a single Item. (In reality I want to configure different lists of callbacks and loggers for the pytorch-lightning Trainer).

Remaining part for completion:

repository_store = store(group='repository')
repository_store(
    Repository,
    hydra_defaults=[
        '_self_',
        {'items': 'default'}
    ],
    name='default'
)
    
store(
    app,
    hydra_defaults=[
        '_self_',
        {'repository': 'default'}
    ]
)

Unfortunately, this results in the error:

In 'repository/items/item_a': ConfigKeyError raised while composing config:
Key '_items_dict' not in 'Builds_Repository'
    full_key: repository._items_dict
    object_type=Builds_Repository

I have also included the code in a singe file (converted to .txt).
Is there a better way to do this? Should I build a list with builds? Should I do this in a config group one level high (i.e. in the repository)? Or should I not use the defaults list altogether? What about the store do I not understand?
I am unfamiliar with writing these questions. I apologize for my bad explanation and type setting.

I think I should add that the following seems to work. However, I don't like it and I am not able to extend the list with another config.

items_store(
    builds(
        list,
        '${oc.dict.values:repository.items._items_dict}',
        hydra_defaults=[
            'item_a@_items_dict.item1',
            'item_b@_items_dict.item2',
            '_self_'
        ],
        zen_meta={'_items_dict': MISSING}
    ),
    name='default'
)

Thanks a lot!
app_hydra_zen.txt

[DirkKuhn]

I think I found the answer:

ItemAConf = builds(Item, msg='a')
ItemBConf = builds(Item, msg='b')
ItemCConf = builds(Item, msg='c')

default_items = [ItemAConf, ItemBConf]

ItemListConf = just(default_items)
FullItemListConf = just(default_items + [ItemCConf])

items_store = store(group='repository/items')
items_store(ItemAConf, name='item_a')
items_store(ItemBConf, name='item_b')
items_store(ItemCConf, name='item_c')
items_store(ItemListConf, name='default')
items_store(FullItemListConf, name='full')

Can someone please confirm whether this is the right way?

[rsokl]

Looks good to me!

[DirkKuhn]

I have another question along the same lines. Adding meta fields when using just doesn't work.

ItemAConf = builds(Item, msg='a_${repository.items.mode}')
items_store = store(group='repository/items')
items_store(just([ItemAConf]), name='default', zen_meta={'mode': 'test'})

results in the error

omegaconf.errors.InterpolationResolutionError: TypeError raised while resolving interpolation: Index 'mode' (str) is not an int

Is there a way around this? Or do I have to specify the meta field in a config at a higher level?
Thanks so much! And I am sorry for the long question!

[rsokl]

This is a case where store's design to "just do what you want, automatically" makes it overly permissive. just([ItemAConf]) returns a config that can be stored straight away. You can see the code that store calls when you pass it an object here: https://mit-ll-responsible-ai.github.io/hydra-zen/_modules/hydra_zen/wrapper/_implementations.html#default_to_config

Because builds is never actually being called, zen_meta is silently ignored.... which is no good. I should raise here.

[DirkKuhn]

A problem with my current solution is that I can't change or add a list entry with the command line. I was wondering whether I can improve that. Something like https://hydra.cc/docs/patterns/select_multiple_configs_from_config_group/ or https://github.com/ashleve/lightning-hydra-template/tree/main/configs/callbacks is what I have in mind.
My new idea is to configure a dict and use a wrapper to convert the dict into a list upon instantiation. Here is the code.

from collections.abc import Sequence
from hydra_zen import store, launch, to_yaml, instantiate, builds

class Item:
    def __init__(self, number: int):
        self.number = number

    def __repr__(self) -> str:
        return f'Item(number={self.number})'

class Repository:
    def __init__(self, items: Item | Sequence[Item]):
        self.items = items

    def __repr__(self) -> str:
        return f'Repository(items={self.items})'

def app(cfg) -> None:
    obj = instantiate(cfg)
    print(obj, end='\n\n')
    print(to_yaml(cfg))

def dict2list(dict_cls):
    def wrapper(*args, **kwargs):
        d = dict_cls(*args, **kwargs)
        return [v for v in d.values() if v is not None]

    return wrapper

a_store = store(group='items/a')
a_store(Item, number=0, name='itemA')

b_store = store(group='items/b')
b_store(Item, number=1, name='itemB')

c_store = store(group='items/c')
c_store(Item, number=2, name='itemC')

items_store = store(group='items')
items_store(
    builds(
        dict, hydra_defaults=['_self_', {'a': 'itemA'}, {'b': 'itemB'}],
        a=None, b=None, c=None, d=None, e=None, zen_wrappers=dict2list
    ),
    name='list'
)

Config = builds(
    Repository,
    populate_full_signature=True,
    hydra_defaults=['_self_', {'items': 'list'}]
)

if __name__ == '__main__':
    store.add_to_hydra_store()
    launch(Config, app, version_base='1.3', overrides=['items.a.number=100', '+items/c=itemC'])

A problem is that the keys have to be known in advance, i.e. I have to specify a=MISSING, b=MISSING, c=MISSING when building the dict.
Can this be improved? Does someone have a better solution?
Thanks a lot!