forked from IntelRealSense/librealsense
-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
PR IntelRealSense#2726 from PrasRsRos: Integration test template
- Loading branch information
Showing
17 changed files
with
2,522 additions
and
16 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,152 @@ | ||
| # Testing realsense2_camera | ||
| The test infra for realsense2_camera uses both gtest and pytest. gtest is typically used here for testing at the unit level and pytest for integration level testing. Please be aware that the README assumes that the ROS2 version used is Humble or later, as the launch_pytest package used here is not available in prior versions | ||
|
|
||
| ## Test using gtest | ||
| The default folder for the test cpp files is realsense2_camera/test. A test template gtest_template.cpp is available in the same folder. | ||
| ### Adding a new test | ||
| If the user wants to add a new test, a copy of gtest_template.cpp as the starting point. Please name the file as gtest_`testname`.cpp format so that the CMake detects the file and add as a new test. Please be aware that, multiple tests can be added into one file, the gtest_template.cpp has 2 tests, test1 and test2. | ||
|
|
||
| ### Adding a new test folder | ||
| It is recommended to use the test folder itself for storing all the cpp tests. However, if the user wants to add a different folder for a set of tests, please ensure that the file name format mentioned above is followed. The folder path is added to realsense_camera/CMakeLists.txt as below for the build to detect the tests within. | ||
|
|
||
| ``` | ||
| find_package(ament_cmake_gtest REQUIRED) | ||
| set(_gtest_folders | ||
| test #<-- default folder for the gtest sources | ||
| new_folder_for_test_but_why #<-- new folder name is added | ||
| ) | ||
| ``` | ||
|
|
||
| ## Test using pytest | ||
| The default folder for the test py files is realsense2_camera/test. Two test template files test_launch_template.py and test_integration_template.py are available in the same folder for reference. | ||
| ### Add a new test | ||
| To add a new test, the user can create a copy of the test_launch_template.py or test_integration_template.py and start from there. Please name the file in the format test_`testname`.py so that the CMake detects the file and add as a new test. Please be aware that, multiple tests can be added into one file, the test_integration_template.py itself has more than one test.i The marker `@pytest.mark.launch` is used to specify the test entry point. | ||
|
|
||
| The test_launch_template.py uses the rs_launch.py to start the camera node, so this template can be used for testing the rs_launch.py together with the rs node. | ||
|
|
||
| The test_integration_template.py gives a better control for testing, it uses few util functions and base test class from pytest_rs_utils.py located at the same location. However, it doesn't use the rs_launch.py, it creates the node directly instead. | ||
|
|
||
| The test_integration_template.py has two types of tests, one has a function "test_using_function". If the user wants to have a better control over the launch context for any specific test scenarios, this can be used. Both the function based test and class based tests use a default launch configuration from the utils. It's recommended to modify the camera name to a unique one in the parameters itself so that there are not clashes between tests. | ||
|
|
||
| It is expected that the class based test is used as the test format for most of the usecases. The class based test inherits from pytest_rs_utils.RsTestBaseClass and it has three steps, namely: init, run_test and process_data. Unless for the basic tests, the user will have to override the process_data function and check if the data received from the topics are as expected. Also, if the user doesn't want the base class to modify the data, use 'store_raw_data':True in the theme definition. Please see the test_integration_template.py for reference. | ||
|
|
||
| An assert command can be used to indicate if the test failed or passed. Please see the template for more info. | ||
|
|
||
| ### Adding a new test folder | ||
| It is recommended to use the test folder itself for storing all the pytests. However, if the user wants to add a different folder for a set of tests, please ensure that the file name format mentioned above is followed. The folder path should be added to realsense_camera/CMakeLists.txt as below for the infra to detect the new test folder and the tests within. | ||
|
|
||
| ``` | ||
| find_package(ament_cmake_pytest REQUIRED) | ||
| set(_pytest_folders | ||
| test #default test folder | ||
| test/templates | ||
| test/rosbag | ||
| new_folder_for_pytest #<-- new folder #but please be aware that the utils functions are in test/utils folder, | ||
| #so if the template is used, change the include path also accordingly | ||
| ) | ||
| ``` | ||
|
|
||
| ### Grouping of tests | ||
| The pytests can be grouped using markers. These markers can be used to run a group of tests. However, "colcon test" command doesn't pass a custom marker using (--pytest-args -m `marker_name`) to the pytest internally. This is because, the ament_cmake that works as a bridge between colcon and pytest doesn't pass the pytest arguments to pytest. So till this is fixed, pytest command has to be used directly for running a group of tests. Please see the next session for the commands to run a group py tests. | ||
|
|
||
| The grouping is specified by adding a marker just before the test declaration. In the test_integration_template.py `rosbag` is specified as a marker specify tests that use rosbag file. This is achieved by adding "@pytest.mark.rosbag" to the begining of the test. So when the pytest parses for test, it detects the marker for the test. If this marker is selected or none of the markers are specified, the test will be added to the list, else will be listed as a deselected test. | ||
|
|
||
| It is recommended to use markers such as ds457, rosbag, ds415 etc to differentiate the tests so that it's easier to run a group of tests in a machine that has the required hardware. | ||
|
|
||
| ## Building and running tests | ||
|
|
||
| ### Build steps | ||
|
|
||
| The command used for building the tests along with the node: | ||
|
|
||
| colcon build | ||
|
|
||
| The test statements in CMakeLists.txt are protected by BUILD_TESTING macro. So in case, the tests are not being built, then it could be that the macro are disabled by default. | ||
|
|
||
| Note: The below command helps view the steps taken by the build command. | ||
|
|
||
| colcon build --event-handlers console_direct+ | ||
|
|
||
| ### Prerequisites for running the tests | ||
|
|
||
| 1. The template tests require the rosbag files from librealsense.intel.comi, the following commands download them: | ||
| ``` | ||
| bag_filename="https://librealsense.intel.com/rs-tests/TestData/outdoors_1color.bag"; | ||
| wget $bag_filename -P "records/" | ||
| bag_filename="https://librealsense.intel.com/rs-tests/D435i_Depth_and_IMU_Stands_still.bag"; | ||
| wget $bag_filename -P "records/" | ||
| ``` | ||
| 2. The tests use the environment variable ROSBAG_FILE_PATH as the directory that contains the rosbag files | ||
| ``` | ||
| export ROSBAG_FILE_PATH=/path/to/directory/of/rosbag | ||
| ``` | ||
| 3. Install launch_pytest package. For humble: | ||
| ``` | ||
| sudo apt install ros-$ROS_DISTRO-launch-pytest | ||
| ``` | ||
| 4. As in the case of all the packages, the install script of realsesnse2_camera has to be run. | ||
| ``` | ||
| . install/local_setup.bash | ||
| ``` | ||
| 5. If the tests are run on a machine that has the RS board connected or the tests are using rosbag files, then its better to let the ROS search for the nodes in the local machine, this will be faster and less prone to interference and hence unexpected errors. It can be achieved using the following environment variable. | ||
| ``` | ||
| export ROS_DOMAIN_ID=1 | ||
| ``` | ||
|
|
||
| So, all put together: | ||
|
|
||
| ``` | ||
| sudo apt install ros-$ROS_DISTRO-launch-pytest | ||
| bag_filename="https://librealsense.intel.com/rs-tests/TestData/outdoors_1color.bag"; | ||
| wget $bag_filename -P "records/" | ||
| bag_filename="https://librealsense.intel.com/rs-tests/D435i_Depth_and_IMU_Stands_still.bag"; | ||
| wget $bag_filename -P "records/" | ||
| export ROSBAG_FILE_PATH=$PWD/records | ||
| . install/local_setup.bash | ||
| export ROS_DOMAIN_ID=1 | ||
| ``` | ||
|
|
||
| ### Running the tests using colcon | ||
|
|
||
| All the tests can be run using the below command: | ||
|
|
||
| colcon test --packages-select realsense2_camera | ||
|
|
||
| This command will invoke both gtest and pytest infra and run all the tests specified in the files mentioned above. Since the test results are stored in build/realsense2_camera/test_results folder, it's good to clean this up after running the tests with a new test added/removed. | ||
|
|
||
| The same command with console_direct can be used for more info on failing tests, as below: | ||
|
|
||
| colcon test --packages-select realsense2_camera --event-handlers console_direct+ | ||
|
|
||
| The test results can be viewed using the command: | ||
|
|
||
| colcon test-result --all --test-result-base build/realsense2_camera/test_results/ | ||
|
|
||
| The xml files mentioned by the command can be directly opened also. | ||
|
|
||
| ### Running pytests directly | ||
|
|
||
|
|
||
| User can run all the tests in a pytest file directly using the below command: | ||
|
|
||
| pytest-3 -s realsense2_camera/test/test_integration_template.py | ||
|
|
||
| All the pytests in a test folder can be directly run using the below command: | ||
|
|
||
| pytest-3 -s realsense2_camera/test/ | ||
|
|
||
| ### Running a group of pytests | ||
| As mentioned above, a set of pytests that are grouped using markers can be run using the pytest command. The below command runs all the pytests in realsense2_camera/test folder that has the marker rosbag: | ||
|
|
||
| pytest-3 -s -m rosbag realsense2_camera/test/ | ||
|
|
||
|
|
||
| ### Running a single pytest | ||
| The below command finds the test with the name test_static_tf_1 in realsense2_camera/test folder run: | ||
|
|
||
| pytest-3 -s -k test_static_tf_1 realsense2_camera/test/ | ||
|
|
||
| ### Points to be noted while writing pytests | ||
| The tests that are in one file are nromally run in parallel. So if there are multiple tests in one file, the system capacity can influence the test execution. It's recomended to have 3-4 tests in file, more than that can affect the test results due to delays. | ||
|
|
||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| // Copyright 2023 Intel Corporation. All Rights Reserved. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| #include <gtest/gtest.h> | ||
|
|
||
| TEST(realsense2_camera, test1) | ||
| { | ||
| std::cout << "Running test1..."; | ||
| ASSERT_EQ(4, 2 + 2); | ||
| } | ||
| TEST(realsense2_camera, test2) | ||
| { | ||
| std::cout << "Running test2..."; | ||
| ASSERT_EQ(4, 2 + 2); | ||
| } | ||
|
|
||
| int main(int argc, char** argv) | ||
| { | ||
| testing::InitGoogleTest(&argc, argv); | ||
| return RUN_ALL_TESTS(); | ||
| } | ||
|
|
Oops, something went wrong.