From 3f3ed51cdb6dac063908ccb93400cf119f5fdd0d Mon Sep 17 00:00:00 2001 From: james7132 Date: Tue, 22 Feb 2022 11:53:28 -0500 Subject: [PATCH 01/20] Start #51: Animation Composition --- rfcs/51-animation-composition.md | 220 +++++++++++++++++++++++++++++++ 1 file changed, 220 insertions(+) create mode 100644 rfcs/51-animation-composition.md diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md new file mode 100644 index 00000000..55c1f531 --- /dev/null +++ b/rfcs/51-animation-composition.md @@ -0,0 +1,220 @@ +# Feature Name: `animation-composition` + +## Summary +Animation is particularly complex, with many stateful and intersecting +elements. This RFC aims to detail a methodlogy for composing multiple animations +together in a generic and composable way. + +## Motivation +Animation is at the heart of modern game development. A game engine without an +animation system is generally not considered production-ready. To provide Bevy +users a complete animation suite, simply start, stop, and seek operations on +singular clips of animation is not sufficient to cover all of the animation +driven use cases in modern video games. An animation system should be able to +support composing individual animations in a generic and flexible way. This would +enable artists, animators, and game designers to create increasingly dynamic +animations without needing to resort to reauthoring multiple variants of the same +animation by hand. + +## Scope +Animation is a huge area that spans multiple problem domains: + + 1. **Storage**: this generally covers on-disk storage in the form of assets as + well as the in-memory representation of static animation data. + 2. **Sampling**: this is how we sample the animation assets over time and + transform it into what is applied to the animated entities. + 3. **Application**: this is how we applied the sampled values to animated + entities. + 4. **Composition**: this is how we compose simple clips to make more complex + animated behaviors. + 4. **Authoring**: this is how we create the animation assets used by the engine. + +This RFC only aims to resolve problems within the domain of composition. Storage +and sampling is addressed in [RFC #49][primitives]. Application can be distinctly +decoupled from these earlier two stages, treating the sampled values as a black +box output, and authoring can be built separately upon the primitives provided by +this RFC and thus are explicit non-goals here. + +## User-facing explanation +Individual animations describe the state of a hierarchy of entities at a given +moment in time. The values in a given animation clip can be blended with one or +more others to produce a composite animation that would otherwise need to be +authored by hand. These blends are usually weighted to allow certain clips to +have a stronger influence the final animation output, and these weights may +evolve over time. + +A common example of this kind of blending is smoothing out a transition from one +animation clip to another. As a character stops walking and starts running, bones +do not immediately snap into place when starting a dash. Blending can help here +by smoothing out the transition. Upon beginning the transition, the blend is +weighted only towards the walk, but smoothly transitions over time to favor the +run. This removes any potential discontinuities and provides for a visually +appealing transition from one animation to another. + +Another example allows developers to convey a range of game states to the player +via animation. For example, if a character is hurt, running may get progressively +harder and harder. To convey this to the player, developers may decrease the +overall movement speed of the character while blending the normal walk and run +animations with a limp animation. As the character gets increasingly damaged, the +blend becomes more and more weighted towards the limp animation. Unlike the +transition case, this is a persistent blend and is not driven by time but by +gameplay design. + +The `AnimationGraph` component allows for storing and blending the clips that +drive animation for an `Entity` hierarchy. The graph itself defines a public +interface for creating a directed acyclic graph of blend nodes, with the terminal +nodes in the graph being individual animation clips. + +## Implementation strategy +*Prototype implementation: https://github.com/HouraiTeahouse/bevy_prototype_animation* + +### Graph Nodes +An animation graph is comprised of a network of connected graph nodes. A public +facing API allwos developers to create, get, and mutate nodes. Nodes can be of +either two types: blend node or clip node. The graph always has one single root +node of either type. + +Blend nodes do not contain any animation of their own, but can have zero or more +input edges. Each of these edges has a `f32` weight associated with it and points +to a source node, which can be either clip node or another blend node. The weight +of parent to child edge affects the end weighting of all descendant clip nodes. + +Clip nodes cannot have inputs, but contain a reference to an animation clip. A +animation graph can have multiple clip nodes referring to same underlying clip. + +Edges can be disconnected without losing metadata on it's input. This is +functionally equivalent to setting a weight of 0, but the associated input node +and it's descendants will not be evaluated. + +Every node has it's own time value. For clip nodes, this is the time at which the +underlying clip is sampled at. For blend nodes, this value doesn't do anything. +However, an optional setting will allow a blend node to propagate the time value +to all of it's immediate children. If this option is set for it's chilren, it +will continue to propagate downwards. This is disabled by default. Another option +for advancing the animation is to advance the entire graph's time in one go. This +will increment the time value on all nodes by a provided time delta. + +### Graph Storage +Two main stores are needed internally for the graph: node storage and clip +storage. Nodes are stored in a flat `Vec`, and a newtyped `NodeId` is used +to keep a reference to a node within the graph. Due to this structure, nodes +cannot be removed from the graph. As a worksaround, a node can be effectively +removed by disconnecting all of it's incoming and outgoing edges. The root NodeId +is always 0. + +Clip storage decomposes clips based on the properties each clip is animating. +Instead of using `Handle`, each clip is instead internally +assigned a auto-imcrementing `ClipId` much like `NodeId`. However, instead of +storing a `Vec` is stored. + +`Track` stores a contiguous sparse array of pointers to all curves associated +with a given property. Here's a rough outline of how such a struct might look: + +```rust +struct Track { + curves: Vec>>>, +} +``` +The individual curves stored within are indexed by `ClipId`, and each pointer to +a curve is non-None if and only if the original clip had a curve for the +corresponding property. + +A type erased version of `Track` will be needed to store the track in a +concrete map. The generic type parameter of the sample function will be used to +downcast the type erased version into the generic version. + +As mentioned in the prior primitives RFC, there will likely need to be a separate +clip storage dedicated to storing specialized `Transform` curves for the purposes +of skeletal animation that avoids both the runtime type casting and the dynamic +dispatch of `Arc>`. + +### Graph Evaluation +To remove the need to evaluate the graph every time a property is sampled, an +influences map is constructed based on the current state of the graph. This is +effectively a `Vec` indexed on `ClipId` mapping clips and their respective +*cumulative* weights. To avoid constant reallocation, this influences map is +stored as a companion component and is cleared before every evaluation. + +During evaluation, the graph is traversed along every connected edge and weights +are multiplicatively propagated downward. For example, if we have a simple graph +`A -> B -> C`, where A is the root and C is the final clip and the edges `A -> B` +and `B -> C` have the weights 0.5 and 0.25 respectively, the final weight for +clip C is 0.125. If multiple paths to a end clip are present, the final weight +for the clip is the sum of all path's weights. + +After traversing the graph, the weights of all active inputs are normalized. + +By default, a dedicated system will run before sampling every app tick that +evaluates every changed graph, relying on change detection to ensure that any +mutation to the graph results in a change in the influences. This behavior can be +disabled via a config option, requiring users to manually evaluate the graph +instead. + +### Graph Sampling +Sampling a single value from the current state of the graph has the rough +following flow: + + - The corresponding track is located in clip storage based on the property + path. Fail if not found. + - The curves for the active clips is retrieved from the track's sparse array of + curves. + - Individual values are sampled from each active curve. Skipping curves with a + weight of 0. + - `Animatable::blend` is used to blend the sampled values together based on the + weights in the influences map. + +This approach has a number of performance related benefits: + + - It only requires one string hash lookup instead of one for every active clip. + - The state of the graph does not need to be evaluated for every sampling. + - Curves can be sampled in `ClipId` order based on the influence map. This + parallel iteration should be cache friendly. + - No intermediate storage is inherently required during sampling. + (`Animatable::blend` may require it for some types). + - This process is (mostly) branch free and can be accelerated easily with SIMD + compatible `Animatable::blend` implementations. + +## Drawbacks +TODO: Complete this section + +## Rationale and alternatives +TODO: Complete this section + +## Prior art +This proposal is largely inspired by Unity's [Playable][playable] API, which has +a similar goal of building composable time-sequenced graphs for animation, audio, +and game logic. Several other game engines have very similar APIs and features: + + - Unreal has [AnimGraph][animgraph] for creating dynamic animations in + Blueprints. + - Godot has [animation trees][animation-trees] for creating dynamic animations in + Blueprints. + +The proposed API here doesn't purport or aim to directly replicate the features +seen in these other engines, but provide the absolute bare minimum API so that +engine developers or game developers can build them if they need to. + +Currently nothing like this exists in the entire Rust ecosystem. + +[playable]: https://docs.unity3d.com/Manual/Playables.html +[animgraph]: https://docs.unrealengine.com/4.27/en-US/AnimatingObjects/SkeletalMeshAnimation/AnimBlueprints/AnimGraph/ +[animation-trees]: https://docs.godotengine.org/en/stable/tutorials/animation/animation_tree.html + +## Unresolved questions + - Is there a way to utilize change detection for graph evaluation without + having a component exposed in userspace? + +## Future possibilities +This RFC only details the lowest level interface for controlling the composition +and blending of multiple animations together and requires code to be written to +manually control the weight and time of every input going into the graph. This +provides signfigant flexibility, but isn't accessible to artists and animators +that don't have or need to interface at such a low level. One major potential +future extension is to expose an asset driven workflow for creating animation +graphs or state machines (i.e. Unity's Animation State Machine). + +Another potential extension is to allow this graph-like composition structure for +non-animation purposes. Using graphs for low level composition of audio +immediately comes to mind, for example. + +[primitives]: https://github.com/bevyengine/rfcs/pr/49 From 5a0d211296df47a0e4e8ec4f64d6ff3f54c5590f Mon Sep 17 00:00:00 2001 From: james7132 Date: Mon, 21 Nov 2022 20:10:58 -0800 Subject: [PATCH 02/20] Update with more pseudo-code --- rfcs/51-animation-composition.md | 219 ++++++++++++++++++++++++++----- 1 file changed, 184 insertions(+), 35 deletions(-) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index 55c1f531..f9c1f04e 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -36,12 +36,15 @@ box output, and authoring can be built separately upon the primitives provided b this RFC and thus are explicit non-goals here. ## User-facing explanation -Individual animations describe the state of a hierarchy of entities at a given -moment in time. The values in a given animation clip can be blended with one or -more others to produce a composite animation that would otherwise need to be -authored by hand. These blends are usually weighted to allow certain clips to -have a stronger influence the final animation output, and these weights may -evolve over time. +An `AnimationGraph` is a component that lives on the root of an entity hierarchy. +Users can add clips to it for it to play. Unlike the current `AnimationPlayer`, +it does not always exclusively play one active animation clip, allowing users to +blend any combination of the clips stored within. This produces a composite +animation that would otherwise need to be authored by hand. + +Each clip has its on assigned weight. As a clips weight grows relative to the +other clips in the graph, it's influence on the final output grows. These weights +may evolve over time to allow a smooth transition between animations. A common example of this kind of blending is smoothing out a transition from one animation clip to another. As a character stops walking and starts running, bones @@ -60,31 +63,29 @@ blend becomes more and more weighted towards the limp animation. Unlike the transition case, this is a persistent blend and is not driven by time but by gameplay design. -The `AnimationGraph` component allows for storing and blending the clips that -drive animation for an `Entity` hierarchy. The graph itself defines a public -interface for creating a directed acyclic graph of blend nodes, with the terminal -nodes in the graph being individual animation clips. - -## Implementation strategy -*Prototype implementation: https://github.com/HouraiTeahouse/bevy_prototype_animation* +Each clip has its on assigned time. Some clips can advance faster or slower than +others. This allows more fine grained control over individual animations than +using the global time scaling feature to speed up or slow down animations. This +can be tied to some gameplay element to produce more believable effects. For +example, speeding up the run animation for a character when they pick up a speed +up can be a cheap and effective way to convey the power up without needing to +author a speed up manually. ### Graph Nodes -An animation graph is comprised of a network of connected graph nodes. A public -facing API allwos developers to create, get, and mutate nodes. Nodes can be of -either two types: blend node or clip node. The graph always has one single root -node of either type. - -Blend nodes do not contain any animation of their own, but can have zero or more -input edges. Each of these edges has a `f32` weight associated with it and points -to a source node, which can be either clip node or another blend node. The weight -of parent to child edge affects the end weighting of all descendant clip nodes. +To help manage the blending of a large number of clips, an animation graph is +comprised of a network of connected graph nodes. A public facing API allows +developers to create, get, and mutate nodes. Nodes can be of either two types: +blend node or clip node. The graph always has one single root node of either +type. Clip nodes cannot have inputs, but contain a reference to an animation clip. A animation graph can have multiple clip nodes referring to same underlying clip. -Edges can be disconnected without losing metadata on it's input. This is -functionally equivalent to setting a weight of 0, but the associated input node -and it's descendants will not be evaluated. +Blend nodes mix all of the provided inputs using a weighted blend. Blend nodes do +not contain any animation of their own, but can have zero or more inputs. Each of +these edges has a `f32` weight associated with it and points to a source node, +which can be any other node, including another blend node. The weight +of parent to child edge affects the end weighting of all descendant clip nodes. Every node has it's own time value. For clip nodes, this is the time at which the underlying clip is sampled at. For blend nodes, this value doesn't do anything. @@ -92,7 +93,32 @@ However, an optional setting will allow a blend node to propagate the time value to all of it's immediate children. If this option is set for it's chilren, it will continue to propagate downwards. This is disabled by default. Another option for advancing the animation is to advance the entire graph's time in one go. This -will increment the time value on all nodes by a provided time delta. +will increment the time value on all nodes by a provided time delta multiplied by +a clip node's. + +### Graph Edges +Edges can be disconnected without losing metadata on it's input. This is +functionally equivalent to setting a weight of 0, but the associated input node +and it's descendants will not be evaluated, which can cut down CPU time spent +evaluating that subgraph. + +## Implementation strategy +*Prototype implementation: https://github.com/HouraiTeahouse/bevy_prototype_animation* + +### Overview +There are several systems that the animation system uses to drive animation from +a graph: + + 1. **Graph Evaluation**: This system evaluates the state of the graph to generate an + influences map that determines how the clips stored within the graph are going + to be blended. This can be done in parallel over every graph individually. + This is also when the internal clocks of the graph are ticked. + 2. **Binding**: This system traverses the entity hierarchy starting at the + graphs to find the corresponding entity for each bone animated by the graph. + This can be done in parallel over every graph individually. + 3. **Graph Sampling**: This system samples the graph for values from the active + clips, uses the generated influences map (from step 1) to blend them into + their final values, and applies it to the bound bones (from step 2). ### Graph Storage Two main stores are needed internally for the graph: node storage and clip @@ -115,6 +141,7 @@ struct Track { curves: Vec>>>, } ``` + The individual curves stored within are indexed by `ClipId`, and each pointer to a curve is non-None if and only if the original clip had a curve for the corresponding property. @@ -129,6 +156,17 @@ of skeletal animation that avoids both the runtime type casting and the dynamic dispatch of `Arc>`. ### Graph Evaluation +```rust +struct Influence { + weight: f32, + time: f32, +} + +struct GraphInfluences { + influence: Vec // Indexed by BoneId +} +``` + To remove the need to evaluate the graph every time a property is sampled, an influences map is constructed based on the current state of the graph. This is effectively a `Vec` indexed on `ClipId` mapping clips and their respective @@ -142,7 +180,7 @@ and `B -> C` have the weights 0.5 and 0.25 respectively, the final weight for clip C is 0.125. If multiple paths to a end clip are present, the final weight for the clip is the sum of all path's weights. -After traversing the graph, the weights of all active inputs are normalized. +After traversing the graph, the weights of all active inputs are clamped and normalized. By default, a dedicated system will run before sampling every app tick that evaluates every changed graph, relying on change detection to ensure that any @@ -150,12 +188,74 @@ mutation to the graph results in a change in the influences. This behavior can b disabled via a config option, requiring users to manually evaluate the graph instead. +### Binding + +```rust +struct BoneId(usize); + +#[derive(PartialEq, Eq, ParitalOrd, Ord)] +struct EntityPath { + path: Vec, +} + +struct GraphBindings { + paths: BTreeMap, + bones: Vec, +} + +struct Bone { + properties: BTreeMap, +} + +#[derive(Resource)] +struct BoneBindings(SparseSet>); +``` + +After graph evalution is building a global `BoneBindings` resource. This is a +newtype over a mapping between an animated entity (the key), and the +`AnimationGraph`s and corresponding `BoneId`s the entity is bound to. + +`BoneBindings` allows us to safely mutate all animated entities in parallel. This +still requires use of `RefectComponent::reflect_unchecked_mut` or +`Query::get_unchecked`, but we can be assured that there is no unsound mutable +aliasing as each entity is only accessed by one thread at any given time. + +`BTreeMap`s are used internally as they're typically more iteration friendly +than `HashMap`. Iteration on `BTreeMap` is `O(size)`, not `O(capacity)` like +HashMaps, and is array oriented which tends to be much more cache friendly than +the hash-based scan. It also provides a consistent iteration order for the +purposes of determinism. + +Rough psuedo-code for building `BoneBindings`: + +```rust +fn build_bone_bindings( + graphs: Query<(Entity, &AnimationGraph), Changed>, + queue: Local>, // Parallel queue + mut bindings: ResMut, +) { + graphs.par_for_each(|(root, graph)| { + for (path, bone_id) in graph.bindings() { + if let Ok(target) = NameLookup::find(root, path) { + queue.push((target, (root, bone_id))) + } else { + warn!("Could not find bone at {}", path); + } + } + }); + + bindings.clear(); + update_bindings(&mut bindings, parallel_queue); +} +``` + ### Graph Sampling +All animated properties on an entity are sampled at the same time. Sampling a single value from the current state of the graph has the rough following flow: - - The corresponding track is located in clip storage based on the property - path. Fail if not found. + - All field paths and tracks for a given entity are fetched for a given bone + from the associated graph. Fail if not found. - The curves for the active clips is retrieved from the track's sparse array of curves. - Individual values are sampled from each active curve. Skipping curves with a @@ -165,20 +265,69 @@ following flow: This approach has a number of performance related benefits: - - It only requires one string hash lookup instead of one for every active clip. - The state of the graph does not need to be evaluated for every sampling. - Curves can be sampled in `ClipId` order based on the influence map. This - parallel iteration should be cache friendly. - - No intermediate storage is inherently required during sampling. - (`Animatable::blend` may require it for some types). + iteration should be cache friendly. + - No allocation is inherently required during sampling. (`Animatable::blend` + may require it for some types). - This process is (mostly) branch free and can be accelerated easily with SIMD compatible `Animatable::blend` implementations. +Rough pseudo-code for sampling values with an exclusive system. + +```rust +fn sample_animators( + world: &mut World, // To ensure exclusive access + bindings: Res, +) { + let world: &World = &world; + let type_registry = world.get_resource::().clone(); + // Uses bevy_tasks::iter::ParallelIterator internally. + bindings.par_for_each_bone(move |bone, bone_id, graph| { + let graph = world.get::(); + for (field_path, track) in graph.get_bone(bone_id).tracks() { + assert!(field_path.type_id() != TypeId::of::()); + let reflect_component = reflect_component( + field_path.type_id(), + &type_registry); + let component: &mut dyn Reflect = unsafe { + reflect_component + .reflect_unchecked_mut(world, bone) + }; + track.sample( + &graph.influences(), + component.path_mut(field_path)); + } + }); +} + +impl Track { + fn sample(&self, influences: &GraphInfluences, output: &mut dyn Reflect) { + // Samples every track at the graph's targets + let tracks = self.curves + .zip(&influences) + .filter(|(curve, _)| curve.0.is_some()) + .map(|(curve, influence)| BlendInput { + value: curve.sample(influence.time), + weight: influence.weight, + ... + }); + + // Blends and uses Reflect to apply the result. + let result = T::blend(inputs); + output.apply(&mut result); + } +} +``` + ## Drawbacks -TODO: Complete this section +The animation sampling system is an exclusive system and blocks all other systems +from running. ## Rationale and alternatives -TODO: Complete this section +An alternative (easier to implement) version that doesn't use `Reflect` can be +implemented with just a `Query<&mut Transform>` instead of a `&World`, though +`Query::get_unchecked` will still need to be used. ## Prior art This proposal is largely inspired by Unity's [Playable][playable] API, which has From 8a2cf582705a8d11549646701dbcdf561ea87876 Mon Sep 17 00:00:00 2001 From: james7132 Date: Mon, 21 Nov 2022 20:32:06 -0800 Subject: [PATCH 03/20] Further document alternatives --- rfcs/51-animation-composition.md | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index f9c1f04e..639d88fc 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -325,9 +325,34 @@ The animation sampling system is an exclusive system and blocks all other system from running. ## Rationale and alternatives + +### Transforms only An alternative (easier to implement) version that doesn't use `Reflect` can be implemented with just a `Query<&mut Transform>` instead of a `&World`, though -`Query::get_unchecked` will still need to be used. +`Query::get_unchecked` will still need to be used. This could be used as an +intermediate step between the current `AnimationPlayer` and the "animate +anything" approach described by the implementation above. + +### Generic Systems instead of Reflect +An alternative to the `Reflect` based approach is to use independent generic +systems that read from `BoneBindings`, but this requires registering each +animatable component and will not generically work with non-Rust components, it +will also not avoid the userspace `unsafe` unless the parallel iteration is +removed. + +### Relatonal `BoneBinding` as a Component +Instead of using `BoneBindings` as a resource that is continually rebuilt every +frame it's changed, `BoneBinding` could be a relational component, much like +`Parent` or `Children`. This removes the need to scan the named hierarchy every +frame, and allows trivial parallel iteration via `Query::par_for_each`. However: + + - it does not avoid the need for userspace `unsafe`. + - maintaining the binding components is going to be a nightmare if there are + any hierarchy or name changes underneath an `AnimationGraph`. + - Commands require applying buffers in between maintanence and use in queries, + forces creation of a bottleneck sync point. + - Using it as a component means there will be secondary archetype moves if a + name or hierarchy changes, which increases archetype fragmentation. ## Prior art This proposal is largely inspired by Unity's [Playable][playable] API, which has From db0977c8feb04f6926fe0ebec784992c85788717 Mon Sep 17 00:00:00 2001 From: james7132 Date: Mon, 21 Nov 2022 20:48:48 -0800 Subject: [PATCH 04/20] Link potential optimizations --- rfcs/51-animation-composition.md | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index 639d88fc..db6a4579 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -35,6 +35,8 @@ decoupled from these earlier two stages, treating the sampled values as a black box output, and authoring can be built separately upon the primitives provided by this RFC and thus are explicit non-goals here. +[primitives]: https://github.com/bevyengine/rfcs/pr/49 + ## User-facing explanation An `AnimationGraph` is a component that lives on the root of an entity hierarchy. Users can add clips to it for it to play. Unlike the current `AnimationPlayer`, @@ -354,6 +356,12 @@ frame, and allows trivial parallel iteration via `Query::par_for_each`. However: - Using it as a component means there will be secondary archetype moves if a name or hierarchy changes, which increases archetype fragmentation. +### Combining Binding and Sampling Steps: Parallelizing on AnimationGraph +If we use the same `unsafe` tricks while parallelizing on `AnimationGraph`, +there's a risk of aliased mutable access if the same entity's components are +animated by two or more `AnimationGraph`s. + + ## Prior art This proposal is largely inspired by Unity's [Playable][playable] API, which has a similar goal of building composable time-sequenced graphs for animation, audio, @@ -391,4 +399,16 @@ Another potential extension is to allow this graph-like composition structure fo non-animation purposes. Using graphs for low level composition of audio immediately comes to mind, for example. -[primitives]: https://github.com/bevyengine/rfcs/pr/49 +### Potential Optimizations +[#4985](https://github.com/bevyengine/bevy/issues/4985) details one potential +optionization. `ReflectComponent::reflect_component_unchecked` calls `World::get` +internally, which requires looking up the entity's location for every component +that is animated. If an alternative that allows using a cached `Entity{Ref, Mut}` +can save that lookup from being used repeatedly. + +Using `bevy_reflect::GetPath` methods on raw strings requires string comparisons, +which can be expensvie when done repeatedly. +[#4080](https://github.com/bevyengine/bevy/issues/4080) details an option to +pre-parse the field path in to a sequence of integer comparisons instead. This +still incurs the cost of dynamic dispatch at every level, but it's signfigantly +faster than doing mutliple piecewise string comparisons for every animated field. From d6c147522ec46a5081c3a6221558af78113729c1 Mon Sep 17 00:00:00 2001 From: james7132 Date: Mon, 21 Nov 2022 20:51:27 -0800 Subject: [PATCH 05/20] Make note about caching ComponentIds --- rfcs/51-animation-composition.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index db6a4579..2b94faa0 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -404,7 +404,9 @@ immediately comes to mind, for example. optionization. `ReflectComponent::reflect_component_unchecked` calls `World::get` internally, which requires looking up the entity's location for every component that is animated. If an alternative that allows using a cached `Entity{Ref, Mut}` -can save that lookup from being used repeatedly. +can save that lookup from being used repeatedly. Likewise, caching the +`ComponentId` for a given `TypeId` when fetching the component can save time +looking up the type to component mapping from `Components`. Using `bevy_reflect::GetPath` methods on raw strings requires string comparisons, which can be expensvie when done repeatedly. From 9dfc3fe8789e738e4a113c0c549573a6ed299852 Mon Sep 17 00:00:00 2001 From: james7132 Date: Mon, 21 Nov 2022 21:06:22 -0800 Subject: [PATCH 06/20] Mention Fyrox's animation implementation --- rfcs/51-animation-composition.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index 2b94faa0..19f0ff9d 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -372,15 +372,23 @@ and game logic. Several other game engines have very similar APIs and features: - Godot has [animation trees][animation-trees] for creating dynamic animations in Blueprints. +Within the Rust ecosystem, [Fyrox][fyrox] has it's own animation system that is +signifgantly higher level than this implementation. At time of writing, it +supports blending, but does not support arbitrary property animation, only +supporting transform based skinned mesh animation. Fyrox supports a high level +state machine that can be used to transition between different states and blend +trees, much like Unity's Mecanim animation system. + The proposed API here doesn't purport or aim to directly replicate the features seen in these other engines, but provide the absolute bare minimum API so that -engine developers or game developers can build them if they need to. - -Currently nothing like this exists in the entire Rust ecosystem. +engine developers or game developers can build them if they need to. It's +entirely feasible to construct a higher level state machine animator like Unity's +Mecanim or Fyrox's animation system on top of the graph detailed here. [playable]: https://docs.unity3d.com/Manual/Playables.html [animgraph]: https://docs.unrealengine.com/4.27/en-US/AnimatingObjects/SkeletalMeshAnimation/AnimBlueprints/AnimGraph/ [animation-trees]: https://docs.godotengine.org/en/stable/tutorials/animation/animation_tree.html +[fyrox]: https://fyrox-book.github.io/fyrox/animation/animation.html ## Unresolved questions - Is there a way to utilize change detection for graph evaluation without From bdbc006a2ec0cc1acf6f3dcad40582c3038f970d Mon Sep 17 00:00:00 2001 From: james7132 Date: Mon, 21 Nov 2022 21:08:28 -0800 Subject: [PATCH 07/20] Mention the fastpath for Transform animation --- rfcs/51-animation-composition.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index 19f0ff9d..4fb37ea2 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -422,3 +422,6 @@ which can be expensvie when done repeatedly. pre-parse the field path in to a sequence of integer comparisons instead. This still incurs the cost of dynamic dispatch at every level, but it's signfigantly faster than doing mutliple piecewise string comparisons for every animated field. + +A direct path for `Transform` based animation that avoids dynamic dispatch might +also save on that very common use case for the system. From d6f4690f76c866552e57851a866d5b455c1cdc25 Mon Sep 17 00:00:00 2001 From: james7132 Date: Tue, 22 Nov 2022 14:49:22 -0800 Subject: [PATCH 08/20] Update goals --- rfcs/51-animation-composition.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index 4fb37ea2..04fa988b 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -29,11 +29,10 @@ Animation is a huge area that spans multiple problem domains: animated behaviors. 4. **Authoring**: this is how we create the animation assets used by the engine. -This RFC only aims to resolve problems within the domain of composition. Storage -and sampling is addressed in [RFC #49][primitives]. Application can be distinctly -decoupled from these earlier two stages, treating the sampled values as a black -box output, and authoring can be built separately upon the primitives provided by -this RFC and thus are explicit non-goals here. +This RFC only aims to resolve problems within the domain of composition and +application. Storage and sampling is addressed in [RFC #49][primitives], and +authoring can be built separately upon the primitives provided by this RFC and +thus are explicit non-goals here. [primitives]: https://github.com/bevyengine/rfcs/pr/49 From 7ee44a2ab7ac802999cdbee516f10d500c0e90df Mon Sep 17 00:00:00 2001 From: james7132 Date: Tue, 22 Nov 2022 14:57:49 -0800 Subject: [PATCH 09/20] Add example visual of a graph --- rfcs/51-animation-composition.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index 04fa988b..0ecb0d52 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -103,6 +103,20 @@ functionally equivalent to setting a weight of 0, but the associated input node and it's descendants will not be evaluated, which can cut down CPU time spent evaluating that subgraph. +### Example Graph + +```mermaid +flowchart LR + root(Root Node) --> mixerA, mixerB, clip05 + mixerA(Mixer A) --> clip01, clip02 + mixerB(Mixer B) --> clip02, clip03, clip04 + clip01(Walk) + clip02(Run) + clip03(Jump Start) + clip04(Jump) + clip05(Jump End) +``` + ## Implementation strategy *Prototype implementation: https://github.com/HouraiTeahouse/bevy_prototype_animation* From cbb816a6caa3799a7116b5aa833f4d3ccffb8131 Mon Sep 17 00:00:00 2001 From: james7132 Date: Tue, 22 Nov 2022 15:03:30 -0800 Subject: [PATCH 10/20] Fix visual --- rfcs/51-animation-composition.md | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index 0ecb0d52..81cec447 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -107,14 +107,23 @@ evaluating that subgraph. ```mermaid flowchart LR - root(Root Node) --> mixerA, mixerB, clip05 - mixerA(Mixer A) --> clip01, clip02 - mixerB(Mixer B) --> clip02, clip03, clip04 - clip01(Walk) - clip02(Run) - clip03(Jump Start) - clip04(Jump) - clip05(Jump End) + root([Root]) + mixerA([Mixer A]) + mixerB([Mixer B]) + clip01([Walk]) + clip02([Run]) + clip03([Jump Start]) + clip04([Jump]) + clip05([Jump End]) + clip06([Punch]) + root-->mixerA + root-->mixerB + root-->clip06 + mixerA-->clip01 + mixerA-->clip02 + mixerB-->clip02 + mixerB-->clip03 + mixerB-->clip04 ``` ## Implementation strategy From d774886b39b6bbef6878747c330c96a6156ac4cb Mon Sep 17 00:00:00 2001 From: james7132 Date: Tue, 22 Nov 2022 15:04:53 -0800 Subject: [PATCH 11/20] Add missing link --- rfcs/51-animation-composition.md | 1 + 1 file changed, 1 insertion(+) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index 81cec447..825abd5e 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -124,6 +124,7 @@ flowchart LR mixerB-->clip02 mixerB-->clip03 mixerB-->clip04 + mixerB-->clip05 ``` ## Implementation strategy From cf46cfca01e4a18e2457423eaa0d4ffc95f70a5c Mon Sep 17 00:00:00 2001 From: james7132 Date: Tue, 22 Nov 2022 15:08:28 -0800 Subject: [PATCH 12/20] Reverse graph flow to better show input/output relationship --- rfcs/51-animation-composition.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index 825abd5e..a9d04ead 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -116,15 +116,14 @@ flowchart LR clip04([Jump]) clip05([Jump End]) clip06([Punch]) - root-->mixerA - root-->mixerB - root-->clip06 - mixerA-->clip01 - mixerA-->clip02 - mixerB-->clip02 - mixerB-->clip03 - mixerB-->clip04 - mixerB-->clip05 + mixerA-->root + mixerB-->root + clip06-->root + clip01-->mixerA + clip02-->mixerB + clip03-->mixerB + clip04-->mixerB + clip05-->mixerB ``` ## Implementation strategy From 6e70b6761634c30a9d1b3bef3e14ae6c7ffd2f7a Mon Sep 17 00:00:00 2001 From: james7132 Date: Tue, 22 Nov 2022 15:09:30 -0800 Subject: [PATCH 13/20] Add missing link --- rfcs/51-animation-composition.md | 1 + 1 file changed, 1 insertion(+) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index a9d04ead..e1f69aad 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -120,6 +120,7 @@ flowchart LR mixerB-->root clip06-->root clip01-->mixerA + clip02-->mixerA clip02-->mixerB clip03-->mixerB clip04-->mixerB From d497153f05ee3c807397917ada6e99c3f47641da Mon Sep 17 00:00:00 2001 From: james7132 Date: Tue, 18 Apr 2023 11:21:28 -0700 Subject: [PATCH 14/20] Copy over the Animatable trait from the primitives RFC --- rfcs/51-animation-composition.md | 59 ++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index e1f69aad..75047767 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -274,6 +274,65 @@ fn build_bone_bindings( } ``` +### `Animatable` Trait +To define values that can be properly smoothly sampled and composed together, a +trait is needed to determine the behavior when interpolating and blending values +of the type together. The general trait may look like the following: + +```rust +struct BlendInput { + weight: f32, + value: T, +} + +trait Animatable { + fn interpolate(a: &Self, b: &Self, time: f32) -> Self; + fn blend(inputs: impl Iterator>) -> Option; + unsafe fn post_process(&mut self, world: &World) {} +} +``` + +`interpolate` implements interpolation between two values of a given type given a +time. This typically will be a [linear interpolation][lerp], and have the `time` +parameter clamped to the domain of [0, 1]. However, this may not necessarily be +strictly be a continuous interpolation for discrete types like the integral +types, `bool`, or `Handle`. This may also be implemented as [spherical linear +interpolation][slerp] for quaternions. This will typically be required to +provide smooth sampling from the variety of curve implementations. If it is +desirable to "override" the default lerp behavior, newtype'ing an underlying +`Animatable` type and implementing `Animatable` on the newtype instead. + +`blend` expands upon this and provides a way to blend a collection of weighted +inputs into one output. This can be used as the base primitive implementation for +building more complex compositional systems. For typical numerical types, this +will often come out to just be a weighted sum. For non-continuous discrete types +like `Handle`, it may select the highest weighted input. Even though a +iterator is inherently ordered in some way, the result provided by `blend` must +be order invariant for all types. If the provided iterator is empty, `None` +should be returned to signal that there were no values to blend. + +A blanket implementation could be done on types that implement `Add + +Mul`, though this might conflict with a need for specialized +implementations for the following types: + - `Vec3` - needed to take advantage of SIMD instructions via `Vec3A`. + - `Handle` - need to properly use `clone_weak`. + +An unsafe `post_process` trait function is going to be required to build values +that are dependent on the state of the World. An example of this is `Handle`, +which requires strong handles to be used properly: a `Curve` can +implement `Curve>` by postprocessing the `HandleId` by reading the +associated `Assets` resource to make a strong handle. This is applied only +after blending is applied so post processing is only applied once per sampled +value. This function is unsafe by default as it may be unsafe to read any +non-Resource or NonSend resource from the World if application is run over +multiple threads, which may cause aliasing errors if read. Other unsafe +operations that mutate the World from a read-only reference is also unsound. The +default implementation here is a no-op, as most implementations do not need this +functionality, and will be optimized out via monomorphization. + +[lerp]: https://en.wikipedia.org/wiki/Linear_interpolation +[slerp]: https://en.wikipedia.org/wiki/Slerp + ### Graph Sampling All animated properties on an entity are sampled at the same time. Sampling a single value from the current state of the graph has the rough From d06e58a5025a293e5c856c3738d05713ad73e9b9 Mon Sep 17 00:00:00 2001 From: james7132 Date: Tue, 18 Apr 2023 12:39:13 -0700 Subject: [PATCH 15/20] Switch to transform only target --- rfcs/51-animation-composition.md | 222 ++++++++++--------------------- 1 file changed, 69 insertions(+), 153 deletions(-) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index 75047767..246f603e 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -156,14 +156,23 @@ is always 0. Clip storage decomposes clips based on the properties each clip is animating. Instead of using `Handle`, each clip is instead internally assigned a auto-imcrementing `ClipId` much like `NodeId`. However, instead of -storing a `Vec` is stored. +storing a `Vec` stores a contiguous sparse array of pointers to all curves associated +`Track` stores a contiguous sparse array of pointers to all curves associated with a given property. Here's a rough outline of how such a struct might look: ```rust -struct Track { - curves: Vec>>>, +struct VariableCurve { + keys: Vec, + times: Vec, +} + +enum Track { + Translation(Vec>>>), + Scale(Vec>>>), + Rotation(Vec>>>), + // Optional. Potential representation for reflection based animation. + Reflect(Vec>>>>), } ``` @@ -171,15 +180,6 @@ The individual curves stored within are indexed by `ClipId`, and each pointer to a curve is non-None if and only if the original clip had a curve for the corresponding property. -A type erased version of `Track` will be needed to store the track in a -concrete map. The generic type parameter of the sample function will be used to -downcast the type erased version into the generic version. - -As mentioned in the prior primitives RFC, there will likely need to be a separate -clip storage dedicated to storing specialized `Transform` curves for the purposes -of skeletal animation that avoids both the runtime type casting and the dynamic -dispatch of `Arc>`. - ### Graph Evaluation ```rust struct Influence { @@ -205,7 +205,8 @@ and `B -> C` have the weights 0.5 and 0.25 respectively, the final weight for clip C is 0.125. If multiple paths to a end clip are present, the final weight for the clip is the sum of all path's weights. -After traversing the graph, the weights of all active inputs are clamped and normalized. +After traversing the graph, the weights of all active inputs are clamped and +normalized. By default, a dedicated system will run before sampling every app tick that evaluates every changed graph, relying on change detection to ensure that any @@ -224,26 +225,22 @@ struct EntityPath { } struct GraphBindings { - paths: BTreeMap, - bones: Vec, + paths: BTreeMap, } struct Bone { - properties: BTreeMap, + tracks: Vec, } - -#[derive(Resource)] -struct BoneBindings(SparseSet>); ``` -After graph evalution is building a global `BoneBindings` resource. This is a -newtype over a mapping between an animated entity (the key), and the -`AnimationGraph`s and corresponding `BoneId`s the entity is bound to. +After graph evalution is finding the target bones to animate. Each graph keeps a +map of all animated bones in a map from path to list of tracks. -`BoneBindings` allows us to safely mutate all animated entities in parallel. This -still requires use of `RefectComponent::reflect_unchecked_mut` or -`Query::get_unchecked`, but we can be assured that there is no unsound mutable -aliasing as each entity is only accessed by one thread at any given time. +All animation graphs will be forbidden from having overlapping hierarchies by +predicating sampling on an ancestor query from every animation graph for a graph +in its ancestor entities. This ensures any given animated bone in a entity +hierarchy is mutably borrowed only by one running animation graph. This allows +animation binding and sampling to run in parallel. `BTreeMap`s are used internally as they're typically more iteration friendly than `HashMap`. Iteration on `BTreeMap` is `O(size)`, not `O(capacity)` like @@ -251,30 +248,7 @@ HashMaps, and is array oriented which tends to be much more cache friendly than the hash-based scan. It also provides a consistent iteration order for the purposes of determinism. -Rough psuedo-code for building `BoneBindings`: - -```rust -fn build_bone_bindings( - graphs: Query<(Entity, &AnimationGraph), Changed>, - queue: Local>, // Parallel queue - mut bindings: ResMut, -) { - graphs.par_for_each(|(root, graph)| { - for (path, bone_id) in graph.bindings() { - if let Ok(target) = NameLookup::find(root, path) { - queue.push((target, (root, bone_id))) - } else { - warn!("Could not find bone at {}", path); - } - } - }); - - bindings.clear(); - update_bindings(&mut bindings, parallel_queue); -} -``` - -### `Animatable` Trait +### Value Blending: `Animatable` Trait To define values that can be properly smoothly sampled and composed together, a trait is needed to determine the behavior when interpolating and blending values of the type together. The general trait may look like the following: @@ -288,19 +262,14 @@ struct BlendInput { trait Animatable { fn interpolate(a: &Self, b: &Self, time: f32) -> Self; fn blend(inputs: impl Iterator>) -> Option; - unsafe fn post_process(&mut self, world: &World) {} } ``` `interpolate` implements interpolation between two values of a given type given a time. This typically will be a [linear interpolation][lerp], and have the `time` -parameter clamped to the domain of [0, 1]. However, this may not necessarily be -strictly be a continuous interpolation for discrete types like the integral -types, `bool`, or `Handle`. This may also be implemented as [spherical linear -interpolation][slerp] for quaternions. This will typically be required to -provide smooth sampling from the variety of curve implementations. If it is -desirable to "override" the default lerp behavior, newtype'ing an underlying -`Animatable` type and implementing `Animatable` on the newtype instead. +parameter clamped to the domain of [0, 1]. This may also be implemented as +[spherical linear interpolation][slerp] for quaternions. This will typically be +required to provide smooth sampling from the variety of curve implementations. `blend` expands upon this and provides a way to blend a collection of weighted inputs into one output. This can be used as the base primitive implementation for @@ -311,24 +280,9 @@ iterator is inherently ordered in some way, the result provided by `blend` must be order invariant for all types. If the provided iterator is empty, `None` should be returned to signal that there were no values to blend. -A blanket implementation could be done on types that implement `Add + -Mul`, though this might conflict with a need for specialized -implementations for the following types: - - `Vec3` - needed to take advantage of SIMD instructions via `Vec3A`. - - `Handle` - need to properly use `clone_weak`. - -An unsafe `post_process` trait function is going to be required to build values -that are dependent on the state of the World. An example of this is `Handle`, -which requires strong handles to be used properly: a `Curve` can -implement `Curve>` by postprocessing the `HandleId` by reading the -associated `Assets` resource to make a strong handle. This is applied only -after blending is applied so post processing is only applied once per sampled -value. This function is unsafe by default as it may be unsafe to read any -non-Resource or NonSend resource from the World if application is run over -multiple threads, which may cause aliasing errors if read. Other unsafe -operations that mutate the World from a read-only reference is also unsound. The -default implementation here is a no-op, as most implementations do not need this -functionality, and will be optimized out via monomorphization. +For the purposes of this RFC, this trait is only required to be implemented on +`bevy_math::Vec3A` and `bevy_math::Quat`, but it may be implemented on other +types for the purposes of reflection based animation in the future. [lerp]: https://en.wikipedia.org/wiki/Linear_interpolation [slerp]: https://en.wikipedia.org/wiki/Slerp @@ -352,8 +306,7 @@ This approach has a number of performance related benefits: - The state of the graph does not need to be evaluated for every sampling. - Curves can be sampled in `ClipId` order based on the influence map. This iteration should be cache friendly. - - No allocation is inherently required during sampling. (`Animatable::blend` - may require it for some types). + - No allocation is inherently required during sampling. - This process is (mostly) branch free and can be accelerated easily with SIMD compatible `Animatable::blend` implementations. @@ -361,46 +314,48 @@ Rough pseudo-code for sampling values with an exclusive system. ```rust fn sample_animators( - world: &mut World, // To ensure exclusive access - bindings: Res, + animation_graphs: Query<(Entity, &AnimationGraph)>, + mut transforms: Query<&mut Transform>, + children: Query<&Children>, ) { - let world: &World = &world; - let type_registry = world.get_resource::().clone(); - // Uses bevy_tasks::iter::ParallelIterator internally. - bindings.par_for_each_bone(move |bone, bone_id, graph| { - let graph = world.get::(); - for (field_path, track) in graph.get_bone(bone_id).tracks() { - assert!(field_path.type_id() != TypeId::of::()); - let reflect_component = reflect_component( - field_path.type_id(), - &type_registry); - let component: &mut dyn Reflect = unsafe { - reflect_component - .reflect_unchecked_mut(world, bone) - }; - track.sample( - &graph.influences(), - component.path_mut(field_path)); + animation_graphsgraphs.par_iter().for_each(|(root, graph)| { + let influences = graph.influences(); + for (entity_path, bone) in graph.bones().iter() { + let Some(transform) = get_descendant(root, entity_path, + transforms, children) else { continue }; + for track in bone.tracks() { + apply_track(*transform, track, influences); + } } }); } -impl Track { - fn sample(&self, influences: &GraphInfluences, output: &mut dyn Reflect) { - // Samples every track at the graph's targets - let tracks = self.curves - .zip(&influences) - .filter(|(curve, _)| curve.0.is_some()) - .map(|(curve, influence)| BlendInput { - value: curve.sample(influence.time), - weight: influence.weight, - ... - }); - - // Blends and uses Reflect to apply the result. - let result = T::blend(inputs); - output.apply(&mut result); - } +fn apply_track( + transform: &mut Transform, + track: &Track, + influences: &GraphInfluences, +) { + match track { + Track::Translation(ref curves) => { + let clips = influences.clips().iter(); + let curves = track.curves().iter(); + let blend_inputs = curves.zip(clips).map(|(curve, clip)| { + BlendInput { + value: curve.sample(clip.time), + weight: clip.weight, + } + }); + if let Some(blend_result) = Vec3A::blend(blend_inputs) { + transform.translation = blend_result;t dqt ;w + } + }, + Track::Scale(...) => { + ... + }, + Track::Rotation(...) => { + ... + }, + } } ``` @@ -410,20 +365,6 @@ from running. ## Rationale and alternatives -### Transforms only -An alternative (easier to implement) version that doesn't use `Reflect` can be -implemented with just a `Query<&mut Transform>` instead of a `&World`, though -`Query::get_unchecked` will still need to be used. This could be used as an -intermediate step between the current `AnimationPlayer` and the "animate -anything" approach described by the implementation above. - -### Generic Systems instead of Reflect -An alternative to the `Reflect` based approach is to use independent generic -systems that read from `BoneBindings`, but this requires registering each -animatable component and will not generically work with non-Rust components, it -will also not avoid the userspace `unsafe` unless the parallel iteration is -removed. - ### Relatonal `BoneBinding` as a Component Instead of using `BoneBindings` as a resource that is continually rebuilt every frame it's changed, `BoneBinding` could be a relational component, much like @@ -438,12 +379,6 @@ frame, and allows trivial parallel iteration via `Query::par_for_each`. However: - Using it as a component means there will be secondary archetype moves if a name or hierarchy changes, which increases archetype fragmentation. -### Combining Binding and Sampling Steps: Parallelizing on AnimationGraph -If we use the same `unsafe` tricks while parallelizing on `AnimationGraph`, -there's a risk of aliased mutable access if the same entity's components are -animated by two or more `AnimationGraph`s. - - ## Prior art This proposal is largely inspired by Unity's [Playable][playable] API, which has a similar goal of building composable time-sequenced graphs for animation, audio, @@ -488,22 +423,3 @@ graphs or state machines (i.e. Unity's Animation State Machine). Another potential extension is to allow this graph-like composition structure for non-animation purposes. Using graphs for low level composition of audio immediately comes to mind, for example. - -### Potential Optimizations -[#4985](https://github.com/bevyengine/bevy/issues/4985) details one potential -optionization. `ReflectComponent::reflect_component_unchecked` calls `World::get` -internally, which requires looking up the entity's location for every component -that is animated. If an alternative that allows using a cached `Entity{Ref, Mut}` -can save that lookup from being used repeatedly. Likewise, caching the -`ComponentId` for a given `TypeId` when fetching the component can save time -looking up the type to component mapping from `Components`. - -Using `bevy_reflect::GetPath` methods on raw strings requires string comparisons, -which can be expensvie when done repeatedly. -[#4080](https://github.com/bevyengine/bevy/issues/4080) details an option to -pre-parse the field path in to a sequence of integer comparisons instead. This -still incurs the cost of dynamic dispatch at every level, but it's signfigantly -faster than doing mutliple piecewise string comparisons for every animated field. - -A direct path for `Transform` based animation that avoids dynamic dispatch might -also save on that very common use case for the system. From 29138a51e8b54a43a5c6a5a835ae4be42ddfeacd Mon Sep 17 00:00:00 2001 From: james7132 Date: Tue, 18 Apr 2023 12:43:47 -0700 Subject: [PATCH 16/20] Typo --- rfcs/51-animation-composition.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index 246f603e..ed22e4a8 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -43,7 +43,7 @@ it does not always exclusively play one active animation clip, allowing users to blend any combination of the clips stored within. This produces a composite animation that would otherwise need to be authored by hand. -Each clip has its on assigned weight. As a clips weight grows relative to the +Each clip has its own assigned weight. As a clips weight grows relative to the other clips in the graph, it's influence on the final output grows. These weights may evolve over time to allow a smooth transition between animations. From f7a6c1212d1cd826dcd09611eca8b2bf0d48358f Mon Sep 17 00:00:00 2001 From: james7132 Date: Tue, 18 Apr 2023 12:44:22 -0700 Subject: [PATCH 17/20] More typo --- rfcs/51-animation-composition.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index ed22e4a8..b092331a 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -64,7 +64,7 @@ blend becomes more and more weighted towards the limp animation. Unlike the transition case, this is a persistent blend and is not driven by time but by gameplay design. -Each clip has its on assigned time. Some clips can advance faster or slower than +Each clip has its own assigned time. Some clips can advance faster or slower than others. This allows more fine grained control over individual animations than using the global time scaling feature to speed up or slow down animations. This can be tied to some gameplay element to produce more believable effects. For From 672aef4211454fef54acc4a9ae4861c22740b018 Mon Sep 17 00:00:00 2001 From: james7132 Date: Tue, 18 Apr 2023 12:57:54 -0700 Subject: [PATCH 18/20] Add section about masked blending --- rfcs/51-animation-composition.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index b092331a..4043d46b 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -423,3 +423,12 @@ graphs or state machines (i.e. Unity's Animation State Machine). Another potential extension is to allow this graph-like composition structure for non-animation purposes. Using graphs for low level composition of audio immediately comes to mind, for example. + +Other engines have a concept of [masked blending][masked_blending]. Moving only +the arm in a wave animation blending with a general body run, for example. +This is probably implemented by just omitting the curves from masked out bones +from the track (i.e. setting it to `None` in the track even if the curve is in +the original `AnimationClip`) when building the graph. The influence of that curve +will be entirely omitted from the final result. + +[masked_blending]: https://docs.unity3d.com/560/Documentation/Manual/class-AvatarMask.html From a39e098e95eb3044d24a108d8dad3ae0848ab5ac Mon Sep 17 00:00:00 2001 From: james7132 Date: Wed, 19 Apr 2023 09:47:37 -0700 Subject: [PATCH 19/20] Remove mention of exclusive systems --- rfcs/51-animation-composition.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index 4043d46b..d3bf8719 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -310,8 +310,6 @@ This approach has a number of performance related benefits: - This process is (mostly) branch free and can be accelerated easily with SIMD compatible `Animatable::blend` implementations. -Rough pseudo-code for sampling values with an exclusive system. - ```rust fn sample_animators( animation_graphs: Query<(Entity, &AnimationGraph)>, @@ -339,14 +337,14 @@ fn apply_track( Track::Translation(ref curves) => { let clips = influences.clips().iter(); let curves = track.curves().iter(); - let blend_inputs = curves.zip(clips).map(|(curve, clip)| { - BlendInput { + let blend_inputs = curves.zip(clips).filter_map(|(curve, clip)| { + curve.map(|curve| BlendInput { value: curve.sample(clip.time), weight: clip.weight, - } + }) }); if let Some(blend_result) = Vec3A::blend(blend_inputs) { - transform.translation = blend_result;t dqt ;w + transform.translation = blend_result; } }, Track::Scale(...) => { @@ -360,8 +358,10 @@ fn apply_track( ``` ## Drawbacks -The animation sampling system is an exclusive system and blocks all other systems -from running. +You cannot remove clips from a loaded animation graph. Even if you unload the +AnimationClip, the curves stored within are kept alive via the Arcs. This may +need to change in the future if we need alternative curve representation for +features like animation streaming. ## Rationale and alternatives From 00180aa65001975a0efd6cfa44a8845864c5aa5f Mon Sep 17 00:00:00 2001 From: James Liu Date: Wed, 19 Apr 2023 09:49:37 -0700 Subject: [PATCH 20/20] Apply suggestions from code review Co-authored-by: Nicola Papale --- rfcs/51-animation-composition.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/rfcs/51-animation-composition.md b/rfcs/51-animation-composition.md index d3bf8719..aae3d55e 100644 --- a/rfcs/51-animation-composition.md +++ b/rfcs/51-animation-composition.md @@ -149,14 +149,14 @@ a graph: Two main stores are needed internally for the graph: node storage and clip storage. Nodes are stored in a flat `Vec`, and a newtyped `NodeId` is used to keep a reference to a node within the graph. Due to this structure, nodes -cannot be removed from the graph. As a worksaround, a node can be effectively +cannot be removed from the graph. As a workaround, a node can be effectively removed by disconnecting all of it's incoming and outgoing edges. The root NodeId is always 0. Clip storage decomposes clips based on the properties each clip is animating. Instead of using `Handle`, each clip is instead internally assigned a auto-imcrementing `ClipId` much like `NodeId`. However, instead of -storing a `Vec`, a map of property paths to `Track` is stored. `Track` stores a contiguous sparse array of pointers to all curves associated with a given property. Here's a rough outline of how such a struct might look: @@ -365,7 +365,7 @@ features like animation streaming. ## Rationale and alternatives -### Relatonal `BoneBinding` as a Component +### Relational `BoneBinding` as a Component Instead of using `BoneBindings` as a resource that is continually rebuilt every frame it's changed, `BoneBinding` could be a relational component, much like `Parent` or `Children`. This removes the need to scan the named hierarchy every