How to merge ethernet parent interface data from connected endpoints and network services

[gusmb]

Consider the following workflow:

1. Define a L2 Ethernet interface in the connected endpoints data model, with the intention to create L3 subinterfaces for each tenant
2. Define subinterfaces for each tenant, adding the data for the L3 subinterface

Before moving to list of dicts structure, the yaml_templates_to_facts plugin was merging the data correctly, creating a L2 parent interface with some data and afterwards turning it into a L3 interface as soon as one tenant subinterface data is created, but keeping the correct description and settings set for the connected endpoint.

After moving to list of dicts, this is no longer the case. yaml_templates_to_facts will now append a list element or replace, therefore losing the initial configuration, interface description etc provided for the connected endpoint, and just keeping the parent interface defined by each tenant, with minimal data.

My purpose is to define a parent interface that is tenant independent, and just define subinterfaces for each tenant, however with list of dicts I am no longer able to do so. An idea could be adding an option to yaml_templates_to_facts to merge lists by key, in a similar way it is done in the lists_mergeby filter from the community.general collection. However this would apply to all lists when merging rendered facts.

Any idea about how I could have the same merging behavior for list of dicts?

[ClausHolbechArista]

We are actually merging list of dicts on the "primary_key" found in the schema - if available. It only works if you don't have replace as your list-merge strategy, so maybe that is what causes this. We know we need to improve the flexibility in merging of lists, but the scenario you describe should actually be covered today (except if you have replace, which is why we changed the default to append_rp). Also notice this merge requires the latest requirements to be up to date (especially deepmerge and jsonschema).
Please check the list merge strategy and requirements. If it still fails I can try to recreate.

BTW: Please share your version, as these things are undergoing changes these days.

[gusmb]

Interesting... I haven't changed the default settings for yaml_templates_to_facts, and my list items (port channel interfaces in this case) don't get merged by name. A new item with the same name and less attributes gets added instead. I am using version 3.8.3

[ClausHolbechArista]

In 3.8.3 we should still be returning dict of dicts in our code, because we lacked exactly this feature. In 4.0 we have implemented conversion of the result from each template and are merging on schemas.
If you run with your own templates, you either need to return the dict-of-dict format and let eos_cli_config_gen convert them, or you can test with a 4.0 dev release to see if it solves the problem.