Remove systems if their parent entity is removed #2232

arjo129 · 2023-11-07T09:54:00Z

🦟 Bug fix

Summary

In particular if a user despawns an entity, the associated plugin gets removed. This should prevent issues like #2165. TBH I'm not sure if this is the right way forward as a system should technically be able to access any entity in a traditional ECS.

The PR has now been reworked and greatly simplified. All we do is stop all worker threads if an entity is removed and then recreaye remaining threads.

Checklist

Signed all commits for DCO
Added tests
Updated documentation (as needed)
Updated migration guide (as needed)
Consider updating Python bindings (if the library has them)
codecheck passed (See contributing)
All tests passed (See test coverage)
While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining Signed-off-by messages.

🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸

codecov · 2023-11-08T08:34:56Z

Codecov Report

Attention: Patch coverage is 92.55319% with 7 lines in your changes missing coverage. Please review.

Project coverage is 64.78%. Comparing base (0f27ce0) to head (89a0f7d).
Report is 119 commits behind head on main.

❗ Current head 89a0f7d differs from pull request most recent head f3c57d0

Please upload reports for the commit f3c57d0 to get more accurate results.

Files	Patch %	Lines
src/SystemManager.cc	89.83%	6 Missing ⚠️
src/systems/pose_publisher/PosePublisher.cc	90.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2232      +/-   ##
==========================================
- Coverage   65.64%   64.78%   -0.87%     
==========================================
  Files         324      358      +34     
  Lines       30937    29213    -1724     
==========================================
- Hits        20308    18925    -1383     
+ Misses      10629    10288     -341

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

jrutgeer · 2023-11-09T08:52:30Z

@arjo129

I thought of this possible alternative implementation:

Only the system plugin itself knows which entities are necessary for its functioning,
- This could e.g. be the model entity it is attached to, but it could as well be any other entity,
- Hence it's the system plugin's responsibility to identify those entities,
To identify these entities, the system plugin adds a component used_by_plugin to each entity it acts upon,
- A used_by_plugin component of an entity identifies a plugin which acts on that entity,
Upon removal of entities, the ECM checks whether they have used_by_plugin components,
- If so, the SystemManager is triggered to remove the corresponding plugin.

A possible extension could be to also have a kind of severity flag in the component, to allow for different reactions from the SystemManager depending on which entities are removed.

E.g.: consider a system attached to a model and which acts on some subentities of that model:

If the model entity is removed, the system should also be removed,
But if a subentity is removed, maybe the system should not be removed yet, but rather perform some 'reconfigure' action instead.

arjo129 · 2023-11-09T09:04:51Z

I like the thought process behind your suggested API addition. However, in my mind that would be an added feature. We need to be able to remove systems safely. That is something that will take a fair deal of work right now and what this PR hopes to (eventually achieve). Your proposed API could be bolted on top in Ionic once we are able to perform safe removal. The current state of this PR is we can safely remove some plugins, but it causes weird breakages in the PosePublisher plugin which I need to investigate prior to making any changes. It is also entirely possible that I'm doing something wrong with the barriers. Either ways I need to revisit this another day when my head is clearer and I have more time.

arjo129 · 2023-11-24T05:09:22Z

I think this works now... Will have to write some unit tests to verify. But it does not go 💥 on the warehouse world.

azeey · 2023-12-01T05:33:15Z

I haven't had a chance to look into this deeply, but one thing to keep is how this interacts with the levels feature. If a model that contains a system is unloaded by the levels feature and gets loaded back again, what would be the behavior? Would the system get configured and start anew? If that works properly, that would cover most cases, but we might also want the ability for a system to stay loaded even when its parent entity is removed so that if the parent entity is loaded back in, the system might continue from where it left off.

arjo129 · 2023-12-01T06:17:10Z

Thanks for pointing that out. I'll go check out levels and see how this PR interacts with it. A cursory reading of the levels code though makes it seem like we despawn entities and then respawn them which makes me wonder if holding a reference "entity" in a system is correct. I've not looked into this deeply enough to say anything. Will update this PR when I try it. I'm currently out on my anual firefighting reservist.

Bogdanov-am · 2024-03-04T16:37:53Z

Is any work ongoing in this PR? I'm currently working on a simulation that requires frequent loading and unloading of models. The problem of unloaded plugins really worries me.

arjo129 · 2024-03-06T08:07:28Z

I just need to write some tests for this.

arjo129 · 2024-03-18T08:30:46Z

Tests have been written. I think this is ready for feedback.

landersson · 2024-05-01T13:28:21Z

Also hoping to see this merged soon.

arjo129 · 2024-05-02T01:51:24Z

If you'd like to get this checked in faster you could help with reviewing it. One thing to do is to checkout the relevant branch and run it on example worlds to see that things work as expected.

landersson · 2024-05-02T03:45:27Z

If you'd like to get this checked in faster you could help with reviewing it. One thing to do is to checkout the relevant branch and run it on example worlds to see that things work as expected.

Unfortunately, I don't have any experience with the gz-sim codebase and can't really comment on whether this is a suitable solution or not.

Just need a way to remove models without plugins like JointStatePublisher crashing.

arjo129 · 2024-05-02T04:40:26Z

Unfortunately, I don't have any experience with the gz-sim codebase and can't really comment on whether this is a suitable solution or not.

You can try it out and tell me if it crashes! There may be plugins I've not tested on!

src/SystemManager.hh

src/systems/pose_publisher/PosePublisher.cc

scpeters

nit: fix spelling in header file name

src/SystemCointainer.hpp

src/SystemContainer_TEST.cc

src/SystemManager.cc

src/SystemManager.hh

src/SystemContainer.hpp

src/SimulationRunner.cc

scpeters

this looks good to me! I just noted two places where comments could be improved, and then I will be ready to approve

src/SimulationRunner.hh

src/SystemManager.cc

azeey · 2024-07-11T20:56:22Z

See #2473

…h interface (#2473) --------- Signed-off-by: Addisu Z. Taddese <addisu@openrobotics.org> Signed-off-by: Steve Peters <scpeters@openrobotics.org> Signed-off-by: Arjo Chakravarty <arjoc@google.com> Co-authored-by: Steve Peters <scpeters@openrobotics.org> Co-authored-by: Arjo Chakravarty <arjoc@intrinsic.ai>

azeey · 2024-07-15T21:40:23Z

I haven't had a chance to look into this deeply, but one thing to keep is how this interacts with the levels feature. If a model that contains a system is unloaded by the levels feature and gets loaded back again, what would be the behavior? Would the system get configured and start anew? If that works properly, that would cover most cases, but we might also want the ability for a system to stay loaded even when its parent entity is removed so that if the parent entity is loaded back in, the system might continue from where it left off.

I've confirmed that when a level that contains a system get unloaded and loaded back again, the associated systems are also unloaded and loaded properly 🎉 . Regarding the ability for a system to stay loaded and continue where it left off, I propose we open a feature request and not address it right now since it's probably not going to be widely used.

Since there are a number of bug reports around this issue and since levels will work fine with this, I'm in favor of backporting this to Harmonic and earlier.

azeey

Thanks for this much needed feature/bug fix and for your patience in iterating!

landersson · 2024-07-15T22:22:29Z

Thanks everyone for your efforts, I will be making use of this ASAP.

arjo129 · 2024-07-15T23:15:48Z

Thanks everyone for your guidance! Seems like our homebrew build is blocking the merge.

scpeters · 2024-07-17T00:21:47Z

Thanks everyone for your guidance! Seems like our homebrew build is blocking the merge.

I can reproduce the test failures on my macOS machine. I'm investigating, but it's a bit strange.

The test failures may have been introduced by #2473 or #2475, since macOS CI did not run for either of those pull requests.

scpeters · 2024-07-17T00:28:03Z

Thanks everyone for your guidance! Seems like our homebrew build is blocking the merge.

I can reproduce the test failures on my macOS machine. I'm investigating, but it's a bit strange.

The test failures may have been introduced by #2473 or #2475, since macOS CI did not run for either of those pull requests.

I can reproduce the seg-faults when building normally and running tests with ctest
I tried reverting 7026b57, and it has no effect
The tests pass when run with lldb
The tests pass when recompiled with -DCMAKE_BUILD_TYPE=Debug

scpeters · 2024-07-17T00:35:06Z

actually the tests have been failing on main since https://build.osrfoundation.org/job/gz_sim-ci-main-homebrew-amd64/68/

scpeters · 2024-07-17T16:43:05Z

rerunning homebrew CI now that gazebosim/gz-math#606 and gazebosim/sdformat#1458 have been reverted after seeing that the reverts fixed tests in #2482

n particular if a user despawns an entity, the associated plugin gets removed. This should prevent issues like #2165. TBH I'm not sure if this is the right way forward as a system should technically be able to access any entity in a traditional ECS. The PR has now been reworked and greatly simplified. All we do is stop all worker threads if an entity is removed and then recreate remaining threads.

n particular if a user despawns an entity, the associated plugin gets removed. This should prevent issues like #2165. TBH I'm not sure if this is the right way forward as a system should technically be able to access any entity in a traditional ECS. The PR has now been reworked and greatly simplified. All we do is stop all worker threads if an entity is removed and then recreate remaining threads. Signed-off-by: Arjo Chakravarty <arjo@openrobotics.org>

github-actions bot added the 🌱 garden Ignition Garden label Nov 7, 2023

arjo129 mentioned this pull request Dec 11, 2023

System plugins keep executing after their model is removed. #2217

Closed

arjo129 self-assigned this Jan 18, 2024

arjo129 marked this pull request as ready for review March 18, 2024 08:25

arjo129 requested a review from mjcarroll as a code owner March 18, 2024 08:25

arjo129 changed the base branch from gz-sim7 to main March 18, 2024 08:26

arjo129 added 🏛️ ionic Gazebo Ionic and removed 🌱 garden Ignition Garden labels Mar 18, 2024

arjo129 requested a review from azeey May 2, 2024 02:10

arjo129 mentioned this pull request May 3, 2024

Prototype: specify order of execution for System::Update callbacks #2394

Closed

13 tasks

scpeters reviewed May 3, 2024

View reviewed changes

src/SystemManager.hh Outdated Show resolved Hide resolved

landersson reviewed May 3, 2024

View reviewed changes

src/systems/pose_publisher/PosePublisher.cc Outdated Show resolved Hide resolved

scpeters reviewed May 7, 2024

View reviewed changes

src/SystemCointainer.hpp Outdated Show resolved Hide resolved

src/SystemContainer_TEST.cc Outdated Show resolved Hide resolved

src/SystemManager.cc Outdated Show resolved Hide resolved

src/SystemManager.hh Outdated Show resolved Hide resolved

scpeters reviewed May 7, 2024

View reviewed changes

src/SystemContainer.hpp Outdated Show resolved Hide resolved

arjo129 commented May 8, 2024

View reviewed changes

src/SimulationRunner.cc Outdated Show resolved Hide resolved

azeey requested a review from scpeters June 10, 2024 18:38

scpeters reviewed Jun 14, 2024

View reviewed changes

src/SimulationRunner.hh Outdated Show resolved Hide resolved

src/SystemManager.cc Outdated Show resolved Hide resolved

azeey mentioned this pull request Jul 11, 2024

Use parentEntity available in systems instead of storing it for each interface #2473

Merged

azeey and others added 2 commits July 12, 2024 16:51

Merge branch 'main' into arjo/feat/remove_systems_with_entities

ed980d9

azeey approved these changes Jul 15, 2024

View reviewed changes

arjo129 enabled auto-merge (squash) July 15, 2024 23:14

This was referenced Jul 17, 2024

Buildfarmer log 2024/07/15 - 2024/07/22 osrf/buildfarm-tools#67

Closed

dontmerge: test graph init changes #2482

Closed

arjo129 merged commit 2994087 into main Jul 17, 2024
9 checks passed

arjo129 deleted the arjo/feat/remove_systems_with_entities branch July 17, 2024 17:39

This was referenced Jul 19, 2024

Crash when removing model with LogicalAudioSensorPlugin #2464

Closed

Crash when calling WorldControl service over model with AckermannSteering plugin #2465

Closed

arjo129 mentioned this pull request Jul 19, 2024

Backport #2232 to harmonic #2485

Merged

azeey mentioned this pull request Jul 22, 2024

Crash after adding and removing model lift_drag_demo_model #2459

Closed

arjo129 mentioned this pull request Jul 26, 2024

Backport #2232 to Fortress #2498

Merged

scpeters mentioned this pull request Aug 19, 2024

Backport to harmonic: Specify System::PreUpdate, Update execution order (#2487) #2530

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove systems if their parent entity is removed #2232

Remove systems if their parent entity is removed #2232

arjo129 commented Nov 7, 2023 •

edited

Loading

codecov bot commented Nov 8, 2023 •

edited

Loading

jrutgeer commented Nov 9, 2023 •

edited

Loading

arjo129 commented Nov 9, 2023 •

edited

Loading

arjo129 commented Nov 24, 2023

azeey commented Dec 1, 2023

arjo129 commented Dec 1, 2023

Bogdanov-am commented Mar 4, 2024

arjo129 commented Mar 6, 2024 •

edited

Loading

arjo129 commented Mar 18, 2024

landersson commented May 1, 2024

arjo129 commented May 2, 2024

landersson commented May 2, 2024

arjo129 commented May 2, 2024

scpeters left a comment

scpeters left a comment

azeey commented Jul 11, 2024

azeey commented Jul 15, 2024

azeey left a comment

landersson commented Jul 15, 2024

arjo129 commented Jul 15, 2024 •

edited

Loading

scpeters commented Jul 17, 2024

scpeters commented Jul 17, 2024

scpeters commented Jul 17, 2024

scpeters commented Jul 17, 2024

Remove systems if their parent entity is removed #2232

Remove systems if their parent entity is removed #2232

Conversation

arjo129 commented Nov 7, 2023 • edited Loading

🦟 Bug fix

Summary

Checklist

codecov bot commented Nov 8, 2023 • edited Loading

Codecov Report

jrutgeer commented Nov 9, 2023 • edited Loading

arjo129 commented Nov 9, 2023 • edited Loading

arjo129 commented Nov 24, 2023

azeey commented Dec 1, 2023

arjo129 commented Dec 1, 2023

Bogdanov-am commented Mar 4, 2024

arjo129 commented Mar 6, 2024 • edited Loading

arjo129 commented Mar 18, 2024

landersson commented May 1, 2024

arjo129 commented May 2, 2024

landersson commented May 2, 2024

arjo129 commented May 2, 2024

scpeters left a comment

Choose a reason for hiding this comment

scpeters left a comment

Choose a reason for hiding this comment

azeey commented Jul 11, 2024

azeey commented Jul 15, 2024

azeey left a comment

Choose a reason for hiding this comment

landersson commented Jul 15, 2024

arjo129 commented Jul 15, 2024 • edited Loading

scpeters commented Jul 17, 2024

scpeters commented Jul 17, 2024

scpeters commented Jul 17, 2024

scpeters commented Jul 17, 2024

arjo129 commented Nov 7, 2023 •

edited

Loading

codecov bot commented Nov 8, 2023 •

edited

Loading

jrutgeer commented Nov 9, 2023 •

edited

Loading

arjo129 commented Nov 9, 2023 •

edited

Loading

arjo129 commented Mar 6, 2024 •

edited

Loading

arjo129 commented Jul 15, 2024 •

edited

Loading