Component Lifecycle Hooks and a Deferred World

[james-j-obrien]

Objective

• Provide a reliable and performant mechanism to allows users to keep components synchronized with external sources: closing/opening sockets, updating indexes, debugging etc.
• Implement a generic mechanism to provide mutable access to the world without allowing structural changes; this will not only be used here but is a foundational piece for observers, which are key for a performant implementation of relations.

Solution

• Implement a new type DeferredWorld (naming is not important, StaticWorld is also suitable) that wraps a world pointer and prevents user code from making any structural changes to the ECS; spawning entities, creating components, initializing resources etc.
• Add component lifecycle hooks on_add, on_insert and on_remove that can be assigned callbacks in user code.

---

Changelog

• Add new DeferredWorld type.
• Add new world methods: register_component::<T> and register_component_with_descriptor. These differ from init_component in that they provide mutable access to the created ComponentInfo but will panic if the component is already in any archetypes. These restrictions serve two purposes:
  1. Prevent users from defining hooks for components that may already have associated hooks provided in another plugin. (a use case better served by observers)
  2. Ensure that when an Archetype is created it gets the appropriate flags to early-out when triggering hooks.
• Add methods to ComponentInfo: on_add, on_insert and on_remove to be used to register hooks of the form fn(DeferredWorld, Entity, ComponentId)
• Modify BundleInserter, BundleSpawner and EntityWorldMut to trigger component hooks when appropriate.
• Add bit flags to Archetype indicating whether or not any contained components have each type of hook, this can be expanded for other flags as needed.
• Add component_hooks example to illustrate usage. Try it out! It's fun to mash keys.

Safety

The changes to component insertion, removal and deletion involve a large amount of unsafe code and it's fair for that to raise some concern. I have attempted to document it as clearly as possible and have confirmed that all the hooks examples are accepted by cargo miri as not causing any undefined behavior. The largest issue is in ensuring there are no outstanding references when passing a DeferredWorld to the hooks which requires some use of raw pointers (as was already happening to some degree in those places) and I have taken some time to ensure that is the case but feel free to let me know if I've missed anything.

Performance

These changes come with a small but measurable performance cost of between 1-5% on add_remove benchmarks and between 1-3% on insert benchmarks. One consideration to be made is the existence of the current RemovedComponents which is on average more costly than the addition of on_remove hooks due to the early-out, however hooks doesn't completely remove the need for RemovedComponents as there is a chance you want to respond to the removal of a component that already has an on_remove hook defined in another plugin, so I have not removed it here. I do intend to deprecate it with the introduction of observers in a follow up PR.

Discussion Questions

• Currently DeferredWorld implements Deref to &World which makes sense conceptually, however it does cause some issues with rust-analyzer providing autocomplete for &mut World references which is annoying. There are alternative implementations that may address this but involve more code churn so I have attempted them here. The other alternative is to not implement Deref at all but that leads to a large amount of API duplication.
• DeferredWorld, StaticWorld, something else?
• In adding support for hooks to EntityWorldMut I encountered some unfortunate difficulties with my desired API. If commands are flushed after each call i.e. world.spawn() // flush commands .insert(A) // flush commands the entity may be despawned while EntityWorldMut still exists which is invalid. An alternative was then to add self.world.flush_commands() to the drop implementation for EntityWorldMut but that runs into other problems for implementing functions like into_unsafe_entity_cell. For now I have implemented a .flush() which will flush the commands and consume EntityWorldMut or users can manually run world.flush_commands() after using EntityWorldMut.
• In order to allowing querying on a deferred world we need implementations of WorldQuery to not break our guarantees of no structural changes through their UnsafeWorldCell. All our implementations do this, but there isn't currently any safety documentation specifying what is or isn't allowed for an implementation, just for the caller, (they also shouldn't be aliasing components they didn't specify access for etc.) is that something we should start doing? (see 10752)

Please check out the example component_hooks or the tests in bundle.rs for usage examples. I will continue to expand this description as I go.

See #10839 for a more ergonomic API built on top of this one that isn't subject to the same restrictions and supports SystemParam dependency injection.

[github-actions]

The generated examples/README.md is out of sync with the example metadata in Cargo.toml or the example readme template. Please run cargo run -p build-templated-pages -- update examples to update it, and commit the file change.

[github-actions]

The generated examples/README.md is out of sync with the example metadata in Cargo.toml or the example readme template. Please run cargo run -p build-templated-pages -- update examples to update it, and commit the file change.

[bushrat011899]

This looks like a fantastic change that would make a lot of change-detection things much better. I had a play with this and made a couple of changes if you'd be interested in incorporating them. I still think this would be amazing as-is, but if you could include those couple of extras, it'd be really helpful.

[bushrat011899]

If this was changed to &mut DeferredWorld you avoid needing to add mut in the hook closure definition, and can avoid repeated instantiation of DeferredWorld when triggering the hooks.

[james-j-obrien]

The presence of mut world was not something I had considered but there is no cost in constructing DeferredWorld as it is just a wrapped world pointer.

[bushrat011899]

Yeah I doubt it has any performance implications, moreso just the ergonomics. Conceptually as well, it matches how things like resource_scope work (providing a world reference). Obviously DeferredWorld is a reference, but it would help with first-glance clarity IMO.

[Diddykonga]

+1 For StaticWorld, as commands are the natural way of modifying world structure, and they are Deferred, which DeferredWorld can not do.

Edit: Oof, I should have looked at the code first.

[maniwani]

+1 For StaticWorld, as commands are the natural way of modifying world structure, and they are deferred, which DeferredWorld can not do.

It's called DeferredWorld in the sense of "you can't make immediate structural changes, you can only queue commands."

[Nilirad]

About finding a better name for DeferredWorld

The word static is synonym of immutable. I'm concerned this might mislead people into thinking FixedWorld cannot be mutated.

The main property of DeferredWorld is its peculiar mutability model. Let's see the mutability levels of World to help us find an appropriate name:

• Immutable. This is like passing &World to a function. It's functionally equivalent to systems with read-only parameters.
• Mutable with structural immutability. Non-exclusive systems can directly mutate the World, not arbitrarily, but only via system parameters, which prevent structural changes by design. However, systems can plan structural changes by using the Commands system parameter. DeferredWorld works in a functionally equal manner, where the only difference is the addition of a managed pointer that has the same structural immutability guarantees of system parameters.
• Fully mutable. Functions owning World, functions with parameter &mut World, and exclusive systems have full mutability potential, including structural mutability.

At this point we understand that we lack terminology for non-structural mutability. This kind of mutability ensures in other words that the layout of the World will remain the same despite allowing other kinds of mutability. This reminds me a lot about rigid bodies in physics who cannot be deformed (in theory), and, even a better analogy, solid state hardware, like SSDs, which have no mechanical moving parts, but they can change state internally.

Some names that came up to my mind are: SolidWorld, RigidWorld and HardWorld

Those names however don't clarify that the data structure is just a pointer to the World, not the real thing. I think we can live without that, or maybe we can add Ptr or Handle as suffix. Most importantly, they are based on arbitrary analogies that may not be immediate to others.

LLMs to the rescue!

I just had a conversation with ChatGPT (GPT-3.5 model) and I got some more inspiration:

• StructurallyStableWorldView: "view" conveys the feel of an indirection layer but IMO is too generic and doesn't stress enough the concept of mutability.
• ManagedWorldAccessor: Drops the "structurally stable" for a more generic "managed accessor" that can be clarified in API docs.
• WorldGuard: even more concise. The word "guard" implies protection from unauthorized access.
• StructuralWorldGuard: less concise than WorldGuard, but indicates the specificity of the guard.

[alice-i-cecile]

Extremely useful, well-engineered. There will be some interesting challenges around how to teach this feature (when should you use this versus other methods? What are the perf characteristics?) but I think this should exist (both for relations and other use cases), and I think this is at a level of code quality that's acceptable to merge.

I'm quite curious about how this might interact with #1481's design space in the future, but that's a separate conversation.

[james7132]

Exposing &mut ComponentInfo is my biggest concern here, as the World's metadata is safety critical, and any possibility of invalidating those metadata stores is an opportunity for unsoundness. I don't like merging this as is, but solving this is a problem bigger than the scope of this PR.