Core: Implement USD workflow with global asset/shot contributions plug-in

[BigRoy]

Changelog Description

• Implement USD publishing with asset and shot contributions through the publisher - for Houdini.
• Implement a USD Render logic which will submit to Deadline with Husk Standalone (for which you'll need a Husk Standalone Deadline Plug-in)
• Also implements a USD look publishing where textures are being copied along as resource files and are anchored relative to the USD file (similar to maya look publishing)

Additional info

Based on ynput/OpenPype#5925 but excluding the Maya logic (that will be a separate PR) - the description and notes in that PR still apply. That PR description also has some "early WIP demo videos" which may help you along.

This is currently a draft refactor from our USD Project's OpenPype logic to AYON:

• Published usd layer from Houdini with asset contribution as variant
• Published usd layer from Houdini with asset contribution without variant set
• Published usd layer from Houdini as shot contribution
• Submitted to Deadline Husk Standalone renders (with split export jobs)
• Publish a look with textures

Note that you currently need to make sure you set your default primitive path on the USD ROPs for correctly outputting the USD files and of course making sure your prims adhere to that hierarchy, e.g. char_hero/geo/my_cube.

Tutorial

Click these sections to expand/collapse

---

Background details + example videos

• In the original OpenPype PR description it contained quite some 'test' videos
• At FMX 2024 an OpenUSD presentation was given that showed usage of what is basically the functionality of this PR.

---

Your first USD publish with AYON

Creating an asset

The producer has his 🔫 pointing at you and needs the asset now!

When we are referring to the USD asset, we mean the asset with both the model's geometry and the look's materials and textures. The USD structure (simplified) then becomes:

asset.usd
  - look.usd
  - model.usd

Where the look is applied over the model inside the asset.

Creating the model

From a Maya, Houdini or Blender perspective

We will want to create our geometry's contribution to the asset.
For the asset structure it's critical we work within the asset's hierarchy so that USD referencing works. (consider that USD mumbo jumbo for now)
As such, we'll always work within a root group with the current folder (asset)'s name.
Say our character's folder path is assets/char_hero then the root group is char_hero

We can make a hierarchy like:

char_hero/
    geo/
        body
        pants

And yes, cool heroes wear no shirts. 👕 Only pants 👖 .

When done. We publish that char_hero as USD contribution to the model layer.
The publishing logic will make the layering of the model.usd into the asset.usd for you automatically.

So no we have:

asset.usd
    model.usd

When loading that in, we'll have our hero wearing pants.
But it needs some 🖌️🎨 paint.

Creating the look

From a Houdini perspective

1. Reference the asset so we have the model.

Note the word "reference" here. We are usually not "sublayering" the asset USD file

2. Make sure to add a "layer break". This special Houdini node breaks off the USD scene above it and from that moment on any changes on the existing hierarchy are just opinions that are overlaid on top of it. This means that when writing out a USD file now we're not re-exporting the geometry itself, but only any changes we make after the layer break.

3. Apply your materials to the char_hero/geo meshes and make sure your materials are within char_hero group as well. So usually you end up with:

char_hero/
    geo/
       body   <- material 'char_hero/mtl/body_shader' applied
       pants  <- material 'char_hero/mtl/pants_shader' applied
    mtl/
        body_shader
        pants_shader

4. Publish as the look product type (which is a USD look in Houdini).

---

The producer is now happy!

Creating a shot

The producer bought another gun, and is dual wielding 🔫 🔫 now and looks fierce. They need a shot ASAP.

Houdini perspective

• Quickly, reference the asset.
• Throw some lights into the scene.
• Set up the USD Render Settings.
• Publish your USD render submission to Husk Standalone

Now the tables have turned! All the producer points at you now is cash dollar bills 💵 and a medal 🥇

You are now the USD champion - until they ask you about asset variants, LODs, purposes and a layered shot structure, and more. Easy, but not for today.

Ready to jump to the next level. 3...2...1 - continue <here> (next chapter still to write)

---

Some technical details explained by @MustafaJafar

USD Publish Settings explained

I'll just focus on 4 interesting settings.

1. target-product which is a user editable text.
2. target-product_department-name which is a selection from a drop down list. and users/admins can't change the items in the list.
3. variant-set-name which is by default is set to {layer} which evaluates to the selected item in the drop down list in Num2
4. variant-name which is by default is set to {variant} as the arrow in the screenshot points.

Asset Structure

The asset structure is based on:

• ASWF Guidelines for Structuring USD Assets
• Nvidia/Omniverse's Principles of Scalable Asset Structure in OpenUSD

But, let me summarize the results.

When enabling Asset Contribution + Add as Variant

[!NOTE]
I love how the asset definition is readable!
Any data in the asset defintion are static. They are computed on publishing.
Paths can change when using AYON resolver.
List of departments, layer-ids and order are hardcoded and not exposed yet to settings.

Product: target-product

# USD Asset
{target-product}.usda
  └── Xform {folder[name]}
        ├── inherits __class__/{folder[name]}
        └── payload ./payload.usda  # Relative path

payload.usda
  └── mata data
        └── sublayers
              └── {target-product_department-name}.usda:ARGS:{layer-id}, {order}

Product: target-product_department-name

# USD Asset Layer
{target-product}.usda
  └── Xform {folder[name]}
        ├── Variant Sets -> ["{variant-set-name}"]
        └── Variant Set "{variant-set-name}"
              └── Variant {variant-name}
                    ├── reference -> Published AYON usd product variant file path
                    └── custom data
                          ├── ayon_order
                          └── ayon_uri -> AYON URI for the published AYON USD product variant

Product: AYON-product-variant

# USD product
{AYON-product-variant}.usd
# It can be any hierarchy.

When enabling Asset Contribution without Add as Variant

It still add a variant! I think there might be a bug or I did something wrong...
Also, I'll update this section once it works on my side.

When disabling Asset Contribution

[!NOTE]
It doesn't affect the version in the latest published target-product or target-product_department-name.

Product: AYON-product-variant

{AYON-product-variant}.usd
# It can be any usd hierarchy.

---

Getting an error about "Unresolved reference path" on load?"

See the comment here

---

Testing notes:

1. Publish and load USD files
2. Submit USD render to deadline
3. Do the testing notes suck? Check the "Tutorial" section above.

[moonyuet]

Have done some simple test on usd publishing. The look product type and usd product type are published successfully.

[BigRoy]

The default prim validation is great.

You're quick. The validation should be improved (no odd logs that were for debugging only + description right hand side)

[BigRoy]

You currently can't publish into a different folder ✅

Because the runtime instances are being check and they have no 'instance data' of their own so can't be disabled.

Fixed with f45012e

[BigRoy]

Did another test run:

• model publish worked,
• look published work with textures being copied along to relative texture paths on publish.
• publish layers to target usdShot with shot structure worked and could be sublayered back in.
  • strength order of anim over assembly worked

There's much room for improvement for documentation and "ready-to-go" templates. In particular configuring the contributions for shots versus assets is error prone and requires some fiddling but at the core it all works.