Houdini: Optimize collect inputs by caching scene containers once

[BigRoy]

Changelog Description

Optimize collect inputs by caching scene containers once

Additional info

Behavior should be the same - it just doesn't list all scene containers and involved nodes per instance, but does it only once in the publish session, making it much faster when running over many instances.

Testing notes:

1. Publish a few different Houdini products, preferably with loaded content used in the inputs - should work without errors.

[moonyuet]

just a small question, is the cache_key always named as __cache_containers? It seems quite hard-coded to me but if it is always named like this, it is still okay.

[BigRoy]

Good question. In the past I had sometimes prefixed it also with the class name to ensure better uniqueness. But yes, in a way it is hardcoded. Feel too unsafe? Should I prefix it more?

[BigRoy]

@moonyuet let me know what you'd like me to do here to make this ready for merging

[moonyuet]

@moonyuet let me know what you'd like me to do here to make this ready for merging

we can merge this first as I guess there would be enhancement when building the universal node for publishing.

[moonyuet]

Tested with houdini, looks good and please review comment above.

[MustafaJafar]

I think it works.
I tried many cases :

• point directly to the sop node in loaded products.
• point to the obj node that's parent of the loaded sop nodes.
• point to an object merge node that combines different loaded nodes.

I'm not quite familiar with this plugin.
So, we trace of any loaded nodes that contribute to the output data and save their ids in "inputRepresentations" ?
if my description is correct, then I'm wondering how could we benefit from that piece of information ?

[BigRoy]

I'm not quite familiar with this plugin.
So, we trace of any loaded nodes that contribute to the output data and save their ids in "inputRepresentations" ?
if my description is correct, then I'm wondering how could we benefit from that piece of information ?

That's correct - it detects what input loaded content was used in your output so they could turn into "Links" in your publish. I believe there's a IntegrateInputLinks plug-in that uses this data.

The data gets converted here:

ayon-core/client/ayon_core/plugins/publish/collect_input_representations_to_versions.py

Line 5 in 70306a1

class CollectInputRepresentationsToVersions(pyblish.api.ContextPlugin):

Then those turn into "generative links":

ayon-core/client/ayon_core/plugins/publish/integrate_inputlinks.py

Line 115 in c4f74a6

input_versions = instance.data.get("inputVersions")

It's so that from a particular publish you can see which input versions were used. So that with e.g. the (upcoming) links UI in the web frontend you could trace the inputs upstream and outputs downstream of each publish.

In OpenPype the loader also had a "Dependencies" tab bottom right hand side. This hasn't been ported to AYON loader (yet?) but could basically display the same info.