Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds a tutorial explaining how to use Environmental Data infrastructure. #1806

Open
wants to merge 2 commits into
base: gz-sim7
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion examples/worlds/environmental_sensor.sdf
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
<?xml version="1.0" ?>
<!-- This example show cases how to load and unload environmental data. -->
<!-- This example show cases how to use a custom sensor to sample data.
You should be able to get data echoed on the topic
"/world/environmental_sensor_test/model/model_with_sensor/link/link/sensor/custom_sensor/environmental_sensor/humidity"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arjo129 meta: that's a pretty lengthy path. Can it be shortened?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks from the tutorial that it can be customized by <topic>, it would be worth adding that to this example file.

-->
<sdf version="1.6">
<world name="environmental_sensor_example">
<plugin
Expand All @@ -20,6 +23,7 @@
name="gz::sim::systems::EnvironmentalSystem">
</plugin>

<!-- Load the file-->
<plugin
filename="gz-sim-environment-preload-system"
name="gz::sim::systems::EnvironmentPreload">
Expand Down
173 changes: 173 additions & 0 deletions tutorials/environmental_data_and_visualization.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
\page environmental_data Visualizing Environment Data
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tutorial also needs to be linked from a new bullet in tutorials.md.in


Lot of times you may come accross environment properties that vary with time and
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Lot of times you may come accross environment properties that vary with time and
A lot of times you may come accross environment properties that vary with time and

space. For instance if you'd like to model properties such as gas distribution
and water temperature one needs to have some way of defining values such that
the values vary through out the environment. Gazebo's Environment Data
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
the values vary through out the environment. Gazebo's Environment Data
the values vary throughout the environment. Gazebo's Environment Data

infrastructure enables such visualization and sensing capabilities.
arjo129 marked this conversation as resolved.
Show resolved Hide resolved

# Loading Environment Data
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previewing the rendered webpage (build/gz-sim7/doxygen/html/environmental_data.html) locally shows that these headings are huge, bigger than the title. Probably need to knock all headings down a level. Same for below.

Suggested change
# Loading Environment Data
## Loading Environment Data


Currently gazebo is able to load data from CSV files and display them. It is
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Currently gazebo is able to load data from CSV files and display them. It is
Currently Gazebo is able to load data from CSV files and display them. It is

assumed the points are sampled at a fixed grid interval. For gaps in between the
sample points, Gazebo will interpolate the data.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arjo129 nit: it may worth describing the interpolation techniques that are in use (or at least adding a link to the code).


## Data Format

As mentioned earlier, the data should be a CSV file with grid based spacing.
The first line of the CSV file should be the heading for each column.
There should be a column for time (in seconds), the x coordinate, y coordinate
and z coordinate (in meters). You may have multiple columns of data like so:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arjo129 nit: it may be worth listing/explaining the reference frames in which these spatial coordinates can be defined.

```
timestamp,humidity,x,y,z
0,0,-1,-1,-1
0,0,-1,-1, 1
0,0,-1, 1,-1
0,0,-1, 1, 1
0,0, 1,-1,-1
0,0, 1,-1, 1
0,0, 1, 1,-1
0,0, 1, 1, 1
1,1,-1,-1,-1
1,1,-1,-1, 1
1,1,-1, 1,-1
1,1,-1, 1, 1
1,1, 1,-1,-1
1,1, 1,-1, 1
1,1, 1, 1,-1
1,1, 1, 1, 1

```
For the x and y coordinates you may also use latitude and longitude in degrees.
Heading names don't matter. Order also doesn't matter.
```
timestamp,humidity,latitude,longitude,altitude
1658923062,91,36.80029505,-121.788972517,0.8
1658923062,88,36.80129505,-121.788972517,0.8
1658923062,89,36.80029505,-121.789972517,0.8
1658923062,92,36.80129505,-121.789972517,0.8
1658923063,90,36.80029505,-121.788972517,0.8
1658923063,85,36.80129505,-121.788972517,0.8
1658923063,87,36.80029505,-121.789972517,0.8
1658923063,94,36.80129505,-121.789972517,0.8
```
> :warning: Note that only one Environment Data file per world is allowed. Also,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The warning syntax does not show in the rendered webpage. It's just small font as others and in italics.

The bigger problem is, it's showing up as two separate lines. So just a regular paragraph is better here.

Gazebo can only load data that fits into memory.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arjo129 meta: should we add a feature request to gz-common5 about lazy loading?


## Pre-Loading
If you are writing SDFormat files by hand or via a script you may want to
pre-load the data. You may use the `EnvironmentPreload` plugin for this. For
instance we could load the first file like so (by adding the environment to the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arjo129 nit: incomplete phrase?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
And there should be a command how to run the example SDF file, like

gz sim environmental_sensor.sdf

):
```xml
<plugin
filename="gz-sim-environment-preload-system"
name="gz::sim::systems::EnvironmentPreload">
<data>environmental_data.csv</data>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arjo129 nit: it may be worth mentioning how path resolution works here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
When I run

gz sim environmental_sensor.sdf

I get

[Err] [EnvironmentalSensorSystem.cc:460] No sensor data loaded
[Err] [EnvironmentPreload.cc:258] No environmental data file was found at environmental_data.csv

The csv file probably needs to be installed somewhere, to build/ or install/. I don't know which one gz sim is picking up, as there are three:

$ find . -name environmental_sensor.sdf
./src/gz-sim/examples/worlds/environmental_sensor.sdf
./install/share/gz/gz-sim7/worlds/environmental_sensor.sdf
./build/gz-sim7/test/worlds/environmental_sensor.sdf

But there are only 2 CSV files, only in src/:

$ find . -name environmental_data.csv
./src/gz-sim/test/worlds/environmental_data.csv
./src/gz-sim/examples/worlds/environmental_data.csv

gz sim is probably not finding them.

Even if I copy one of the CSV files to the current working directory, now I don't get the 2nd error about file not found, but I still get the first error "No sensor data loaded."

<dimensions>
<time>timestamp</time>
<space>
<x>x</x>
<y>y</y>
<z>z</z>
</space>
</dimensions>
</plugin>
```
If you would like to use Latitude and Longitude you may set the `reference` to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you would like to use Latitude and Longitude you may set the `reference` to
If you would like to use latitude and longitude you may set the `reference` to

`spherical` and the `units` to `degrees`.
```xml
<plugin
filename="gz-sim-environment-preload-system"
name="gz::sim::systems::EnvironmentPreload">
<data>environmental_data.csv</data>
<dimensions>
<time>timestamp</time>
<space reference="spherical" units="degrees">
<x>latitude</x>
<y>longitude</y>
<z>altitude</z>
</space>
</dimensions>
</plugin>
```

## Via GUI
To load the data, click on the 3 dots and search for the "Environment Loader":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For completeness, should instruct the user how to bring up the GUI we are starting from, e.g. I'm assuming it is

gz sim empty.sdf

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, with the new changes we need to preload

    <plugin
      filename="gz-sim-environmental-sensor-system"
      name="gz::sim::systems::EnvironmentalSystem">
    </plugin>

Otherwise nothing will work. Perhaps I should add an empty_environment_system.sdf?

@image html files/environment_data/environment_loader.png

Click on it and you will see the environment loader:

@image html files/environment_data/environment_loader_options.png

Select the file you would want to load by clicking on the arrow next to the
field named "Data file path".

@image html files/environment_data/environment_file_loaded.png
Copy link
Contributor

@mabelzhang mabelzhang Jun 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be an explanation of what the Reference fields mean - what the default ecef is, and the difference between global and spherical.


Select the appropriate fields and then click "load".
Copy link
Contributor

@mabelzhang mabelzhang Jun 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, the data doesn't seem to be loading for me.

Right when I click the button to browse files, I get this error

[GUI] [Err] [EnvironmentLoader.cc:194] No environmental data file was found at 

That's probably not supposed to happen. It seems it's trying to load too early, right when I click the browse button, before I even choose a file.
Update: This error disappeared after I updated the branches in the 2 PRs locally from gz-sim7 and gz-msgs9. But the following behavior is still the same.

Then after I select the environmental_data.csv in src/examples/worlds, select the right columns for the XYZ fields, and click Load, nothing happens - no more printouts. If the data loaded, I would expect a printout. If the data didn't load, I would expect an error.

Then if I proceed to the Visualization section, after I click the refresh button, there's still nothing in the drop-down list of topics.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, those were the branches I used. I also tried after merging from gz-msgs9 and gz-sim7, as the gz-msgs PR says the branch is outdated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for not getting back to this quicker but I've found the reason why. I've fixed it with c30639a.


@image html files/environment_data/environment_fields.png

The data is now loaded into the system.

# Visualizing the data

To visualize the data we will load 2 plugins. You must already have loaded the
data in. First we will search for "Environment Visualization" using the three
dots menu (second from top):

@image html files/environment_data/environment_viz_search.png

Click on it. You should see some sliders.

@image html files/environment_data/environment_viz_sliders.png

Next search for the "Point Cloud" tool.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is 3 panels to add in order to visualize some data... It's a lot of steps to remember. Without a tutorial, the interface is not intuitive at all, and a user would never guess it. Is there a way to simplify this UI? Or in the meantime, maybe even just have some tooltips in the panels to tell the user which panels they need to open next?


@image html files/environment_data/environment_pc_search.png

You should see a panel like so:

@image html files/environment_data/environment_pc_panel.png

Hit the refresh button to find the pointcloud generated b y the visuallization
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arjo129 nit:

Suggested change
Hit the refresh button to find the pointcloud generated b y the visuallization
Hit the refresh button to find the pointcloud generated by the visualization

plugin. You can select the field you wish to view using the "Float vector"
field. Once thats done, you should be able to see the data.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arjo129 nit:

Suggested change
field. Once thats done, you should be able to see the data.
field. Once that's done, you should be able to see the data.


@image html files/environment_data/environment_data_viz.png

# Using custom sensors to sense the data
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Custom sensor" is an overloaded phrase that, in the general Gazebo context, means custom sensor plugins. That is, the thing that was a lot of work to write in Gazebo-classic but made a lot easier in new Gazebo, which is an advertised feature of the new Gazebo. So I wouldn't use that phrase here. It would be very confusing if the user is familiar with that context.

Probably "custom environmental sensor" would be more appropriate.


Apart from just visuallization, there are custom sensors available for sampling
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arjo129 nit:

Suggested change
Apart from just visuallization, there are custom sensors available for sampling
Apart from just visualization, there are custom sensors available for sampling

the data. So if you would like to have your robot search for a maxima for
instance, you would need a custom sensor. It is easy to add such a sensor.
Simply add a sensor like so (replace column name with your own):
```xml
<sensor name="custom_sensor" type="custom" gz:type="environmental_sensor/{column_name}">
<always_on>1</always_on>
<update_rate>30</update_rate>
</sensor>
```
By default it'll echo to a topic generated by the scoped name.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By echo, do you mean publish or subscribe?

```bash
gz topic -e -t "/world/environmental_sensor_test/model/model_with_sensor/link/link/sensor/custom_sensor/environmental_sensor/humidity"
```
If you would like a better topic name you may always specify the topic like so
```
<sensor name="custom_sensor" type="custom" gz:type="environmental_sensor/{column_name}">
<always_on>1</always_on>
<update_rate>30</update_rate>
<topic>humidity</topic>
</sensor>
```
Humidity should now be available on the `/humidity` topic.
```bash
gz topic -e -t "/humidity"
```
# Using C++ API to integrate with your own plugins

You may want to use environmental data to model your own plugins for instance,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arjo129 nit:

Suggested change
You may want to use environmental data to model your own plugins for instance,
You may want to use environmental data to model your own plugins. For instance,

you may want to apply a force according to a pre-calculated wind model. For such
use cases we provide the [`Environment`](https://github.com/gazebosim/gz-sim/blob/gz-sim7/include/gz/sim/components/Environment.hh) component. To make querying the component easier you may take a
look at [`this method`](https://github.com/gazebosim/gz-sim/blob/4fa1f26a738cbdb3ff0c4a37fdde9506f3cd9b4e/include/gz/sim/Util.hh#L300).
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.