Run server and client in the same process #793

ahcorde · 2021-04-27T10:49:19Z

Signed-off-by: ahcorde ahcorde@gmail.com

🎉 New feature

Summary

Related to this issue #556

Test it

ign gazebo -r -v 4 --same-process shapes.sdf
ign gazebo -r -v 4 --same-process camera_sensor.sdf  # -> crash

Checklist

Signed all commits for DCO
Added tests
Added example and/or tutorial
Updated documentation (as needed)
Updated migration guide (as needed)
codecheck passed (See contributing)
All tests passed (See test coverage)
While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge

ahcorde · 2021-04-27T10:53:02Z

I made a comment in the related issue #556 (comment)

I want to make it work then I will clean up te code.

Signed-off-by: Louise Poubel <louise@openrobotics.org>

Signed-off-by: ahcorde <ahcorde@gmail.com>

ahcorde · 2021-06-02T15:34:38Z

I updated the base PR to match with the recent refactoring in the scene3D.

Current status

Gui and sensor thread shared the same scene.
- not able to visualize the objects in the scene 3D
- When I try to load the ImageDisplay then I get a crash
Ogre crashes when we try to use the wheel to zoom in and zoom out. In particular in the method OgreRayQuery::ClosestPoint. The vertex from the meshes are not fetched properly.
Ogre2 crashes on start in the method void Ogre2RenderTarget::Render() in the call engine->OgreRoot()->renderOneFrame();

to reproduce:

/home/ahcorde/ignition_fortress_leak/install/bin/ign gazebo --gui-config src/ign-gazebo/src/gui/gui.config camera_sensor.sdf -v 4 --same_process --render-engine ogre --render-engine-gui ogre --render-engine-server ogre

If you want to try Ogre two, you should modify the gui.config file and replace ogre with ogre2

/home/ahcorde/ignition_fortress_leak/install/bin/ign gazebo --gui-config src/ign-gazebo/src/gui/gui.config camera_sensor.sdf -v 4 --same_process --render-engine ogre2 --render-engine-gui ogre2 --render-engine-server ogre2

Limitations

everything should run with ogre or ogre2 but we cannot mix both renders when gui and server are running in the same process.

Signed-off-by: ahcorde <ahcorde@gmail.com>

ahcorde · 2021-06-04T16:52:54Z

I have two issues right now:

Sometimes ign-gazebo starts with some repeated entities in the Entity tree

When the entity is removed from the entity tree, it removed from the scene but not from the entity tree

Signed-off-by: ahcorde <ahcorde@gmail.com>

ahcorde · 2021-06-08T10:30:01Z

Decisions that I have made

There are two differences main way to launch ign-gazebo: with sensors and without sensors. This have some implications.

When we are runninig with sensors

There are two RenderUtils available: One in the sensor system plugin and another in the GUI. This is the easiest case because there are two system plugins that remove entities: Sensors and Physics which runs at the same speed. But the Scene 3D is running just at 30Hz. This may cause some issues in the Entity tree plugin. Some of the EachRemoved cycles are lost. I have created some events to be able to remove/add entities in the Entity tree plugin, this event are emitted in the renderutils class.

RemoveFromECM
AddToECM

The GzSceneManager should not update renderutil because the sensor thread will do it.

When running without sensors

In this case the physics system plugin will be the only one with access to the removeEach calls. In this case this plugin will send an event called UpdateGUIECM to avoid missing any removals.

There is another event called EnableSensors this event is used GzSceneManager. In this case the sensor system plugin is not updating the RenderUtils because it not running, so this event allow us to check if we should update the Scene 3D from the Gui system plugin.

EnableSensors
UpdateGUIECM

Render

On of the limitiation that we have, it is that we can only update the render scene in the same thread, otherwise we will get a crash. When running in the same process there are two Renderutils trying to update the scene3d in different threads, it's important to load in the first time the Scene3D to set the option useCurrentGLContext to 1. For this reason we need to create a event called Render, this event is emitted in the GzSceneManager plugin every time that an event Render is received in the eventFilter method.

Notes

Run the world without the sceneBroadcaster when using the --same_process flag
We can use Ogre or Ogre2 at the same time, we cann't mix both render engines when same process is activated.
Recommended branch of ign-gui https://github.com/ignitionrobotics/ign-gui/pull/new/ahcorde/6/new_scene

ahcorde · 2021-06-08T10:52:26Z

Recommended branch of ign-gui https://github.com/ignitionrobotics/ign-gui/pull/new/ahcorde/6/new_scene

src/systems/physics/Physics.cc

iche033 · 2021-06-09T00:42:54Z

With the latest changes, the events for keeping GUI plugins in sync with server side seems to be working from testing.

I also want to brainstorm ideas on how we could reuse more of the current ECM infrastructure. I think the problem all came down to the GuiRunner running the gui plugins at 30 Hz in a separate thread. What do you think of the following proposal:

Emit an event when SimulationRunner updates the systems
https://github.com/ignitionrobotics/ign-gazebo/blob/cc4cf71f45836ad6478ab8c69af5dc4a19201d5d/src/SimulationRunner.cc#L523-L527
In GuiRunner same process mode, we can connect to this event instead of running its own loop.
In call back we can call UpdatePlugins. We can also throttle it to some rate, e.g. 30Hz. by checking time since last update call.
In order not to miss important events like entities removed / added, we can use ECM's HasNewEntities, HasEntitiesMarkedForRemoval, HasOneTimeComponentChanges functions in the callback and check if we need bypass throttling and call UpdatePlugins
https://github.com/ignitionrobotics/ign-gazebo/blob/cc4cf71f45836ad6478ab8c69af5dc4a19201d5d/include/ignition/gazebo/EntityComponentManager.hh#L493-L502

The idea above is so that the gui plugins can keep using the EachRemoved call without having to connect to new events. This reduces the effort required to port other gui plugins (or user's own custom gui plugins) to work in the same process mode.

Signed-off-by: ahcorde <ahcorde@gmail.com>

chapulina · 2021-08-14T01:28:05Z

src/gui/plugins/entity_tree/EntityTree.cc

+  if (this->entityItems.find(_entity) != this->entityItems.end())
+  {
+    return;
+  }


Broke this off into #974

Signed-off-by: Louise Poubel <louise@openrobotics.org>

chapulina

I took a look at the sensors - 3D scene synchronization, and it concerns me that we're instantiating 2 RenderUtil classes that compete to create and update entities. I'm also wondering what impact the GUI running the render loop will have on the sensors' update rate. See more detailed comments below.

chapulina · 2021-08-17T01:35:52Z

src/systems/sensors/Sensors.cc

@@ -316,6 +331,45 @@ void SensorsPrivate::Run()
  this->renderThread = std::thread(&SensorsPrivate::RenderThread, this);
 }

+//////////////////////////////////////////////////
+void SensorsPrivate::Render()


Is this function ever called?

chapulina · 2021-08-17T01:48:32Z

src/rendering/SceneManager.cc

-    return rendering::VisualPtr();
+    auto vis = this->dataPtr->scene->VisualByName(name);
+    this->dataPtr->visuals[_id] = vis;
+    return vis;


Are these changes here because Sensors and Scene3D are trying to create the same entity? I think we should solve the underlying issue instead of protecting against it.

Right now I see lots of error messages:

$ ign gazebo sensors_demo.sdf --same-process [GUI] [Err] [SceneManager.cc:187] Visual: [ground_plane] already exists [GUI] [Err] [SceneManager.cc:187] Visual: [box] already exists [GUI] [Err] [SceneManager.cc:187] Visual: [cameras_alone] already exists [GUI] [Err] [SceneManager.cc:187] Visual: [camera_with_lidar] already exists [GUI] [Err] [SceneManager.cc:187] Visual: [rgbd_camera] already exists [GUI] [Err] [SceneManager.cc:187] Visual: [thermal_camera] already exists [GUI] [Err] [SceneManager.cc:187] Visual: [Construction Cone] already exists [GUI] [Err] [SceneManager.cc:1054] Visual: [sun::sunVisual] already exists [GUI] [Err] [SceneManager.cc:267] Entity with Id: [36] already exists in the scene [GUI] [Err] [SceneManager.cc:1098] Light with Id: [38] already exists in the scene [GUI] [Err] [SceneManager.cc:1054] Visual: [sun::sunVisual] already exists

We need to have a single authority on who's managing the lifecycle of entities in the scene when we run sensors and GUI in the same process. Otherwise, they're both trying to do duplicate work, which may eventually cause undefined behaviour.

My suggestion is that during sameProcess, we should have a single RenderUtil and SceneManager instantiated, instead of 2 trying to update the same scene.

chapulina · 2021-08-17T01:50:59Z

src/systems/sensors/Sensors.cc

+    if (!this->sameProcess)
+    {
+      IGN_PROFILE("Update");
+      this->renderUtil.Update();


Can you comment on the reasoning for having the GUI drive the render loop? Sensors are pickier about update rates, so I think they should have control.

Signed-off-by: ahcorde <ahcorde@gmail.com>

chapulina · 2021-09-22T15:39:00Z

Didn't make it into beta, can come in a minor release.

ahcorde requested review from iche033 and chapulina April 27, 2021 10:49

github-actions bot added the 🏯 fortress Ignition Fortress label Apr 27, 2021

ahcorde marked this pull request as draft April 27, 2021 10:49

ahcorde self-assigned this Apr 27, 2021

ahcorde mentioned this pull request Apr 27, 2021

Support running server and client in the same process #556

Open

diegoferigo mentioned this pull request Apr 27, 2021

Expose the Entity Component Manager #248

Open

chapulina and others added 4 commits May 12, 2021 14:08

Add scene manager GUI plugin that works with ign-gui's Scene3D

3a7d09c

Signed-off-by: Louise Poubel <louise@openrobotics.org>

cleanup

80161fb

Signed-off-by: Louise Poubel <louise@openrobotics.org>

Added some-process flag

400974d

Signed-off-by: ahcorde <ahcorde@gmail.com>

Avoid some crashes

ae902c3

Signed-off-by: ahcorde <ahcorde@gmail.com>

ahcorde changed the base branch from main to chapulina/6/scene_manager June 2, 2021 15:27

ahcorde force-pushed the ahcorde/same_process branch from 476a11a to ae902c3 Compare June 2, 2021 15:28

ahcorde added 4 commits June 3, 2021 22:09

Improved implementation

57cf18f

Signed-off-by: ahcorde <ahcorde@gmail.com>

added more improvemetns

343dba8

Signed-off-by: ahcorde <ahcorde@gmail.com>

uncomment code

999620d

Signed-off-by: ahcorde <ahcorde@gmail.com>

some fixes

9498255

Signed-off-by: ahcorde <ahcorde@gmail.com>

ahcorde added 2 commits June 8, 2021 10:16

Updates

ea7107f

Signed-off-by: ahcorde <ahcorde@gmail.com>

Updates

cc4cf71

Signed-off-by: ahcorde <ahcorde@gmail.com>

ahcorde marked this pull request as ready for review June 8, 2021 10:30

ahcorde requested a review from azeey as a code owner June 8, 2021 10:30

azeey reviewed Jun 8, 2021

View reviewed changes

src/systems/physics/Physics.cc Outdated Show resolved Hide resolved

try to fix Windows build

e1e1c6f

Signed-off-by: ahcorde <ahcorde@gmail.com>

ahcorde requested a review from chapulina August 13, 2021 09:56

ahcorde mentioned this pull request Aug 13, 2021

Standalone Ignition Gazebo executables #694

Closed

8 tasks

Merge branch 'main' into ahcorde/same_process

78f940e

chapulina mentioned this pull request Aug 14, 2021

Entity tree: prevent creation of repeated entity items #974

Merged

7 tasks

chapulina reviewed Aug 14, 2021

View reviewed changes

chapulina added 2 commits August 13, 2021 18:58

small cleanup tweaks, also fix OnPluginUpdate

d6d1f8a

Signed-off-by: Louise Poubel <louise@openrobotics.org>

Merge branch 'main' into ahcorde/same_process

74ed8e2

chapulina mentioned this pull request Aug 14, 2021

WIP [Same process] Updates to Server-GUI synchronization #975

Closed

chapulina reviewed Aug 17, 2021

View reviewed changes

chapulina added the beta Targeting beta release of upcoming collection label Sep 8, 2021

chapulina assigned mjcarroll Sep 14, 2021

ahcorde added 2 commits September 21, 2021 22:00

Merge remote-tracking branch 'origin/main' into ahcorde/same_process

baf095e

make linters happy

449e6e4

Signed-off-by: ahcorde <ahcorde@gmail.com>

chapulina removed the beta Targeting beta release of upcoming collection label Sep 22, 2021

chapulina added 🌱 garden Ignition Garden and removed 🏯 fortress Ignition Fortress labels Nov 16, 2021

adlarkin removed their request for review December 6, 2021 18:13

chapulina added the enhancement New feature or request label Jul 25, 2022

chapulina removed the 🌱 garden Ignition Garden label Aug 6, 2022

azeey added beta Targeting beta release of upcoming collection and removed beta Targeting beta release of upcoming collection labels Jul 31, 2023

azeey added beta Targeting beta release of upcoming collection and removed beta Targeting beta release of upcoming collection labels Jul 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Run server and client in the same process #793

Run server and client in the same process #793

ahcorde commented Apr 27, 2021

ahcorde commented Apr 27, 2021

ahcorde commented Jun 2, 2021

ahcorde commented Jun 4, 2021

ahcorde commented Jun 8, 2021 •

edited

Loading

ahcorde commented Jun 8, 2021

iche033 commented Jun 9, 2021

chapulina Aug 14, 2021

chapulina left a comment •

edited

Loading

chapulina Aug 17, 2021

chapulina Aug 17, 2021

chapulina Aug 17, 2021

chapulina commented Sep 22, 2021

Run server and client in the same process #793

Are you sure you want to change the base?

Run server and client in the same process #793

Conversation

ahcorde commented Apr 27, 2021

🎉 New feature

Summary

Test it

Checklist

ahcorde commented Apr 27, 2021

ahcorde commented Jun 2, 2021

Limitations

ahcorde commented Jun 4, 2021

ahcorde commented Jun 8, 2021 • edited Loading

Decisions that I have made

When we are runninig with sensors

When running without sensors

Render

Notes

ahcorde commented Jun 8, 2021

iche033 commented Jun 9, 2021

chapulina Aug 14, 2021

Choose a reason for hiding this comment

chapulina left a comment • edited Loading

Choose a reason for hiding this comment

chapulina Aug 17, 2021

Choose a reason for hiding this comment

chapulina Aug 17, 2021

Choose a reason for hiding this comment

chapulina Aug 17, 2021

Choose a reason for hiding this comment

chapulina commented Sep 22, 2021

ahcorde commented Jun 8, 2021 •

edited

Loading

chapulina left a comment •

edited

Loading