[rostest] Randomize Gazebo port number

[mikepurvis]

rostest has some nifty logic which allows rostests to run on top of each other, by randomizing the rosmaster's port number:

ros_comm/tools/rostest/src/rostest/rostest_parent.py

Line 84 in 98aa98e

run_id = roslaunch.core.generate_run_id()

Unfortunately, there can still be collisions with concurrent tests when Gazebo is involved, as there will be contention over the default GAZEBO_MASTER_URI value of http://localhost:11345. Would it be reasonable to speculatively set GAZEBO_MASTER_URI in the test environment, with a known-available port?

[dirk-thomas]

A short detour on how the random port number for rostest works: the feature of rostest to choose a random port for the ROS master was add in #468. It is achieved by creating the Master instance with a port 0 which will then let the master pick a random port. After the master is started and knows its port number each executable will get the chosen ROS master uri set in the environment:

ros_comm/tools/roslaunch/src/roslaunch/core.py

Line 201 in 4468f86

d[rosgraph.ROS_MASTER_URI] = master_uri

I understand your use case with running multiple tests using Gazebo. On the other hand nothing in these ROS packages knows about Gazebo at the moment. And the same request could be made for almost any other external software which needs a specific environment in order to run multiple instances in parallel. Therefore I hesitate to add something specifically for Gazebo. A "generic" feature which enables your use case for Gazebo but also covers similar use cases would be nicer.

I am not sure how this can be achieved best. Since Gazebo is run from one entry in the roslaunch file it is a bit different than the ROS master described above.

• Do any of the other nodes in the launch file also need the Gazebo uri?
  • If no, would it be an option to let Gazebo pick its own port (will require a change to Gazebo)?
  • If yes, the env var needs to be set externally. And the chosen port number must be unique and available.

[mikepurvis]

My primary interest is in running tests concurrently from multiple repos on the same buildslave (without going to Docker or whatever for port isolation), or by a parallel catkin_tools invocation (eg catkin/catkin_tools#397). So from either of those perspectives, it would be perfectly possible to just have a gazebo_testing package with an env hook that picks a random available port and just sets the variable.

However, that doesn't help the catkin_make run_tests case, since it would still be a single GAZEBO_MASTER_URI value. For that there'd need to be some kind of extra pre-test environment injection. Unsure on the best way to achieve that.

I agree in principal that it sucks to put something Gazebo-specific like this into rostest, but the complexity of a pluggable solution seems really high; given that it's just setting a var without any actual dependence, I wonder if it would be worth it.

[dirk-thomas]

I would imagine the environment variable to be set on a per test level. E.g. instead of calling add_rostest() you would call add_rostest_with_gazebo() which internally decides on a port number (which must be free) and then calls add_rostest() with a new argument ENV GAZEBO_MASTER_URI value.

This approach should work fine with catkin_make run_tests or any other way to run the tests. While this new CMake function could be added to rostest I think it would make more sense in e.g. https://github.com/ros-simulation/gazebo_ros_pkgs rostest would still need to be updated to add the new env argument though. How does that sound?

[mikepurvis]

That would be picking the port number at configure time rather than execution time, wouldn't it?

The idea of add_rostest_with_gazebo() is appealing, but I feel like it would need a little bit deeper integration than just wrapping add_rostest().

[dirk-thomas]

That is true.

The beauty with ROS master is that it picks the port on its own before any node from the launch file is started. And then that can be used to set the environment in all nodes. I don't see yet how something similar can be done with Gazebo (in a way that ensure collision free port numbers).

[mikepurvis]

Yes, in the case of setting up the env var in advance, it will still just be a random number best effort. Without cooperation from Gazebo, that's the best that's possible.

[dirk-thomas]

Since Gazebo is only one process in a launch file I don't see a good way how even with the support of Gazebo this could be improved.

@ros/ros_team Any ideas how to approach this?

[mikepurvis]

It would have to be a scheme where add_gazebo_rostest() launches a standalone gazebo instance (similar to the roscore), and passes its port number into the environment of the launch file with the actual tests, which expect to interact with a supplied instance rather than launching their own.

In the short term, I still think there's value in having an option to at least randomize it, or do some kind of round-robin scheme.

[wjwwood]

Setting the env variable to control the port Gazebo uses and randomizing that is the best option of which I can think.