Asset system rework and GLTF scene loading

[cart]

I've been working with "resident asset-expert" @kabergstrom for awhile now to figure out the future of the Bevy asset system. @kabergstrom built atelier-assets, which has an impressive set of features and has basically everything a pro-grade engine needs from an asset system. While learning atelier, I decided to explore the space for a bit by experimenting with directly expanding the current bevy_asset with the features we were interested in.

After much discussion about adopting atelier-assets vs expanding the current system, we agreed that atelier was the best long term bet. @kabergstrom is alreading building the features Bevy requires into atelier (such as WASM support and loading assets as paths) and we expect to move forward relatively quickly on that. I will be lending a hand where I can. However the exact delivery date is undetermined.

We have a number of scenarios that are blocked on the current asset system (GLTF loading is a big one). Rather than rush the atelier migration, I decided it was worth it to merge a subset of the bevy_asset improvements I made to unblock certain scenarios, and to ultimately make the transition to atelier easier.

This adds a number of major changes to the asset system:

• Asset Handle Ref-Counting: assets are now automatically freed when there are no more "strong" references to them
• Asset Dependencies: assets can now depend on other assets. dependencies are loaded when an asset is loaded
• AssetLoaders can produce multiple assets of any type: this enables complex asset loading scenarios, such as full GLTF loading.
• AssetIo: An abstract "virtual filesystem" interface for loading/saving asset sources
• Assets now require TypeUuid: this makes the handle ref-counting impl as little nicer, but atelier also requires TypeUuids, so we're also including them here to make the transition smoother.
• Removed load_sync: This api wasn't WASM friendly, encouraged users to block game execution for the sake of convenience, and was incompatible with the new AssetLoader api.

It also adds the following:

• A new GLTF scene loader: GLTF files are loaded as scenes, which include strong handles to the materials, textures, and meshes in the file.
• Preserve Entity references in Spawned Scenes via the new EntityMap trait: up until now, spawning components like Parent would result in broken references. Components can now implement the EntityMap trait, which allows the references to be remapped during spawn
• Fixed hot-asset reloading: we stopped properly listening for changes awhile ago. now its fixed!
• Better In-Memory Scene Representation: A new "Scene as World" representation for efficient in-memory scene storage/access. It also makes scene composition at runtime much more straightforward. These can be converted to and from DynamicScenes.
• New GLTF Example: This example loads the Khronos flight helmet GLTF sample
• Spawn Scene Command: simpler way to spawn scenes
• Consolidated WorldWriter and ResourceWriter: This is simpler to use, simpler to implement, and allows combined World+Resource writer scenarios
• Added TypeUuid to bevy_type_registry: A slighty modified version of the type_uuid crate. We don't pull it in directly because: (1) it lets bevy users use the derive without needing to explicitly depend on the type_uuid crate (2) we can avoid the rust "orphan rule" (3) we were able to make the TypeUuid interface slightly nicer by returning a const Uuid directly instead of Bytes

[cart]

I'm going to try to re-add WASM support by making the AssetIo interface async.

[anarelion]

I am trying to implement an archive, not exactly a GTLF, but imagine those world of warcraft fat files that are 2gb long and contain many assets, but you don't really need to load all of them.

This system enables the load an asset that depends on more assets on a single file, so the archive needs to know the formats and type of all the sub-assets.

Maybe I should use a AssetIo system that exposes all the files inside, but doesn't seem to be an easy way to register new AssetIos.

I am not sure I understood this PR completely, and I am sure this PR definitely improves things, and maybe in the near future it will be easier to support my use-case.

I am saying maybe because I am not really sure I understand everything. Sorry for the noise :)

[cart]

@anarelion You could probably do something like that with this pr (by implementing a loader capable of loading all of the asset types you need from a single file), but its worth pointing out that @kabergstrom has plans to add "pack file" support to atelier, which would allow you to "pack" all of your assets into a single file, then load from that in your released game. It might be worth waiting for the atelier migration so you can just take advantage of that.

[anarelion]

@cart thanks for your answer. I will try to pass sub-slices to other loaders

[masonk]

0.3 removed load_sync: What's the right way to load assets without load_sync?

At some point we need to block the game until the assets needed for the next gamestate are available. Are we supposed to register systems that emit events when our assets load? I think I'm missing a few pieces of this puzzle.

[cart]

@masonk we removed explicit blocking on asset io because it (1) causes hitching and is generally an anti-pattern (2) explicitly can't work in WASM due to how the runtime works.

load_sync is "bad" because it blocks the event loop. People like it because the blocking makes it easy to ensure "game logic" happens after "loading logic". But we don't actually need to block the event loop to accomplish that outcome. Instead we just need a way to enforce the order of operations.

We added States in 0.4 as a solution to this problem (and other problems). They enable you to have a "loading" state and a "game" state. Check out this example to see how you can use States to handle "loading" scenarios without blocking.

[masonk]

Thanks, that states example with check_textures explains everything.