Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Frame metadata not available for Realsense d435 #3686

Closed
mattiagrassi93 opened this issue Apr 5, 2019 · 16 comments
Closed

Frame metadata not available for Realsense d435 #3686

mattiagrassi93 opened this issue Apr 5, 2019 · 16 comments
Assignees
@ev-mp
Labels
installation Linux

Comments

@mattiagrassi93
Copy link

| Required Info | |

| Camera Model | D435 |
| Firmware Version | 05.11.01.100 |
| Operating System & Version | Ubuntu 16.04 |
| Kernel Version (Linux Only) | 4.14.102 |
| Platform | Odroid XU-4 |
| SDK Version | 2.19.0 |

Issue Description

<Good morning,
I am trying to use a realsense d435 with ROS (kinetic, Realsense ROS v2.2.1) but, after I type
roslaunch realsense2_camera rs_camera.launch to start the camera, I encounter this warning:

Frame metadata isn't available! (frame_timestamp_domain = RS2_TIMESTAMP_DOMAIN_SYSTEM_TIME)

The following is the output of the terminal:

SUMMARY

PARAMETERS

  • /camera/realsense2_camera/accel_fps: 250
  • /camera/realsense2_camera/accel_frame_id: camera_accel_frame
  • /camera/realsense2_camera/accel_optical_frame_id: camera_accel_opti...
  • /camera/realsense2_camera/align_depth: False
  • /camera/realsense2_camera/aligned_depth_to_color_frame_id: camera_aligned_de...
  • /camera/realsense2_camera/aligned_depth_to_fisheye1_frame_id: camera_aligned_de...
  • /camera/realsense2_camera/aligned_depth_to_fisheye2_frame_id: camera_aligned_de...
  • /camera/realsense2_camera/aligned_depth_to_fisheye_frame_id: camera_aligned_de...
  • /camera/realsense2_camera/aligned_depth_to_infra1_frame_id: camera_aligned_de...
  • /camera/realsense2_camera/aligned_depth_to_infra2_frame_id: camera_aligned_de...
  • /camera/realsense2_camera/base_frame_id: camera_link
  • /camera/realsense2_camera/clip_distance: -2.0
  • /camera/realsense2_camera/color_fps: 30
  • /camera/realsense2_camera/color_frame_id: camera_color_frame
  • /camera/realsense2_camera/color_height: 480
  • /camera/realsense2_camera/color_optical_frame_id: camera_color_opti...
  • /camera/realsense2_camera/color_width: 640
  • /camera/realsense2_camera/depth_fps: 30
  • /camera/realsense2_camera/depth_frame_id: camera_depth_frame
  • /camera/realsense2_camera/depth_height: 480
  • /camera/realsense2_camera/depth_optical_frame_id: camera_depth_opti...
  • /camera/realsense2_camera/depth_width: 640
  • /camera/realsense2_camera/enable_accel: True
  • /camera/realsense2_camera/enable_color: True
  • /camera/realsense2_camera/enable_depth: True
  • /camera/realsense2_camera/enable_fisheye1: True
  • /camera/realsense2_camera/enable_fisheye2: True
  • /camera/realsense2_camera/enable_fisheye: True
  • /camera/realsense2_camera/enable_gyro: True
  • /camera/realsense2_camera/enable_infra1: True
  • /camera/realsense2_camera/enable_infra2: True
  • /camera/realsense2_camera/enable_pointcloud: False
  • /camera/realsense2_camera/enable_sync: False
  • /camera/realsense2_camera/filters:
  • /camera/realsense2_camera/fisheye1_frame_id: camera_fisheye1_f...
  • /camera/realsense2_camera/fisheye1_optical_frame_id: camera_fisheye1_o...
  • /camera/realsense2_camera/fisheye2_frame_id: camera_fisheye2_f...
  • /camera/realsense2_camera/fisheye2_optical_frame_id: camera_fisheye2_o...
  • /camera/realsense2_camera/fisheye_fps: 30
  • /camera/realsense2_camera/fisheye_frame_id: camera_fisheye_frame
  • /camera/realsense2_camera/fisheye_height: 480
  • /camera/realsense2_camera/fisheye_optical_frame_id: camera_fisheye_op...
  • /camera/realsense2_camera/fisheye_width: 640
  • /camera/realsense2_camera/gyro_fps: 400
  • /camera/realsense2_camera/gyro_frame_id: camera_gyro_frame
  • /camera/realsense2_camera/gyro_optical_frame_id: camera_gyro_optic...
  • /camera/realsense2_camera/infra1_frame_id: camera_infra1_frame
  • /camera/realsense2_camera/infra1_optical_frame_id: camera_infra1_opt...
  • /camera/realsense2_camera/infra2_frame_id: camera_infra2_frame
  • /camera/realsense2_camera/infra2_optical_frame_id: camera_infra2_opt...
  • /camera/realsense2_camera/infra_fps: 30
  • /camera/realsense2_camera/infra_height: 480
  • /camera/realsense2_camera/infra_width: 640
  • /camera/realsense2_camera/initial_reset: False
  • /camera/realsense2_camera/json_file_path:
  • /camera/realsense2_camera/linear_accel_cov: 0.01
  • /camera/realsense2_camera/odom_frame_id: camera_odom_frame
  • /camera/realsense2_camera/pointcloud_texture_index: 0
  • /camera/realsense2_camera/pointcloud_texture_stream: RS2_STREAM_COLOR
  • /camera/realsense2_camera/pose_frame_id: camera_pose_frame
  • /camera/realsense2_camera/pose_optical_frame_id: camera_pose_optic...
  • /camera/realsense2_camera/rosbag_filename:
  • /camera/realsense2_camera/serial_no:
  • /camera/realsense2_camera/unite_imu_method:
  • /rosdistro: kinetic
  • /rosversion: 1.12.14

NODES
/camera/
realsense2_camera (nodelet/nodelet)
realsense2_camera_manager (nodelet/nodelet)

ROS_MASTER_URI=http://localhost:11311

process[camera/realsense2_camera_manager-1]: started with pid [12576]
process[camera/realsense2_camera-2]: started with pid [12577]
[ INFO] [1554454322.947177752]: Initializing nodelet with 8 worker threads.
[ INFO] [1554454323.118452481]: RealSense ROS v2.2.1
[ INFO] [1554454323.118624857]: Running with LibRealSense v2.19.0
[ INFO] [1554454323.473666084]: getParameters...
[ INFO] [1554454323.723921248]: setupDevice...
[ INFO] [1554454323.724243958]: JSON file is not provided
[ INFO] [1554454323.724369209]: ROS Node Namespace: camera
[ INFO] [1554454323.724580210]: Device Name: Intel RealSense D435
[ INFO] [1554454323.724728794]: Device Serial No: 825312071439
[ INFO] [1554454323.724858919]: Device FW version: 05.11.01.100
[ INFO] [1554454323.724979920]: Device Product ID: 0x0B07
[ INFO] [1554454323.725182588]: Enable PointCloud: Off
[ INFO] [1554454323.725379589]: Align Depth: Off
[ INFO] [1554454323.725567548]: Sync Mode: Off
[ INFO] [1554454323.725993925]: Device Sensors:
[ INFO] [1554454323.726250510]: Stereo Module was found.
[ INFO] [1554454323.726395969]: RGB Camera was found.
[ INFO] [1554454323.726573678]: (Fisheye, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1554454323.726690845]: (Fisheye, 1) sensor isn't supported by current device! -- Skipping...
[ INFO] [1554454323.726805387]: (Fisheye, 2) sensor isn't supported by current device! -- Skipping...
[ INFO] [1554454323.726942888]: (Gyro, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1554454323.727059722]: (Accel, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1554454323.727175139]: (Pose, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1554454323.728142686]: setupPublishers...
[ INFO] [1554454323.742899093]: Expected frequency for depth = 30.00000
[ INFO] [1554454323.867709298]: Expected frequency for infra1 = 30.00000
[ INFO] [1554454323.962727730]: Expected frequency for infra2 = 30.00000
[ INFO] [1554454324.069527803]: Expected frequency for color = 30.00000
[ INFO] [1554454324.168976298]: setupStreams...
05/04 08:52:04,175 WARNING [3069132800] (sensor.cpp:366) Unregistered Media formats : [ UYVY ]; Supported: [ ]
[ INFO] [1554454324.189119565]: depth stream is enabled - width: 640, height: 480, fps: 30
[ INFO] [1554454324.190788365]: infra1 stream is enabled - width: 640, height: 480, fps: 30
[ INFO] [1554454324.192371747]: infra2 stream is enabled - width: 640, height: 480, fps: 30
05/04 08:52:04,198 WARNING [3069132800] (backend-v4l2.cpp:1248) Pixel format 36315752-1a66-a242-9065-d01814a likely requires patch for fourcc code RW16!
05/04 08:52:04,201 WARNING [3069132800] (sensor.cpp:366) Unregistered Media formats : [ RW16 ]; Supported: [ ]
[ INFO] [1554454324.234521540]: color stream is enabled - width: 640, height: 480, fps: 30
[ INFO] [1554454324.237992433]: insert Depth to Stereo Module
[ INFO] [1554454324.238208100]: insert Color to RGB Camera
[ INFO] [1554454324.238387976]: insert Infrared to Stereo Module
[ INFO] [1554454324.238560102]: insert Infrared to Stereo Module
[ INFO] [1554454324.415878401]: num_filters: 0
[ INFO] [1554454324.416154194]: SELECTED BASE:Depth, 0
[ INFO] [1554454324.422951478]: RealSense Node Is Up!
[ INFO] [1554454324.423094145]: Setting Dynamic reconfig parameters.
05/04 08:52:04,436 WARNING [2855269408] (ds5-timestamp.cpp:64) UVC metadata payloads not available. Please refer to installation chapter for details.
[ WARN] [1554454324.459806744]: Frame metadata isn't available! (frame_timestamp_domain = RS2_TIMESTAMP_DOMAIN_SYSTEM_TIME)
[ INFO] [1554454324.512459590]: Done Setting Dynamic reconfig parameters.
05/04 08:52:04,554 WARNING [2793563168] (ds5-timestamp.cpp:64) UVC metadata payloads not available. Please refer to installation chapter for details.

Can you help me with this problem?
Thank you very much.
@paLeziart
Copy link

paLeziart commented Apr 5, 2019 •
edited
Loading

I have the same problem. I tried to find a solution but as you can still retrieve images and pointclouds even with this warning I just gave up...

A solution that was given in October 2018 (IntelRealSense/realsense-ros#492)

If you follow distribution_linux.md the following command should solve the problem:
sudo apt-get install librealsense2-dkms

Maybe it will work for you...

@ev-mp
Copy link
Collaborator

ev-mp commented Apr 7, 2019

@mattiagrassi93 hi,
The Debian dkms package is designed for Ubuntu LTS kernels.
For Odroid running v4.14 kernel there is a kernel patch contributed by @freemanlo. You can find it here

@vladsterz
Copy link

Have you generated the makefile with metadata enabled in Cmake?

@mattiagrassi93
Copy link
Author

Thank you very much for your answer, I will try as soon as I can and update the issue.

@mattiagrassi93
Copy link
Author

I tried the patch that you linked @ev-mp
I got this error:

Applying realsense-powerlinefrequency-fix patch
patching file drivers/media/usb/uvc/uvc_ctrl.c
cp: cannot stat '/lib/modules/4.14.102-138/build/.config': No such file or directory

I don't know how to fix it. As someone experienced the same issue? What can I do?

@ev-mp
Copy link
Collaborator

ev-mp commented Apr 14, 2019

@freemanlo , apparently there are some kernel versions mismatches. Can you assist?

@RealSenseCustomerSupport
Copy link
Collaborator

Hi @mattiagrassi93,

Try below patch which is modified especially for D435i. Let me know how it works.
https://github.com/freemanlo/librealsense/blob/odroid_build_4.14/scripts/patch-realsense-ubuntu-odroid-xu4-4.14.sh

@akirayou
Copy link
Contributor

I found another situation D435i and Raspi2/Ubuntu mate18.
Which using realsenest-ubuntu-lts.sh for the driver patch.

I got USB PID 0x0ad6 but not listed in the drriver.
So I added it, it works.

But I don't know ,is "0x0ad6" correct one.
Is it uvc driver bug (lack of device list) or firmware bug(wrong PID)?

My workaround:
akirayou@264347e#diff-722c465090b13fcdbd0d0a7f376a149bR182

@ev-mp
Copy link
Collaborator

ev-mp commented May 20, 2019

@akirayou hello,
The PID 0x0ad6 is a rather an outdated enumeration assigned to the device in USB2 mode.
With this enumeration ID in place only the essential Depth control/streaming capabilities are enabled.
What is the FW version for the D435i you're using ?

@akirayou
Copy link
Contributor

My D435i version was 05.11.01.100,and I update to 05.11.04.00 today.
But the PID (with USB 2.1 cable) still 0x0ad6
I checked it by windows device manger (USB\VID_8086&PID_0AD6&REV_50B4&MI_00)
Thank you.

@RealSenseCustomerSupport
Copy link
Collaborator

@mattiagrassi93, @akirayou,
Still need any support for this topic? Suggest to have USB3 connection with full capabilities.

@akirayou
Copy link
Contributor

Yes I know USB3 is right way. But I want to use them on RasPi which have no USB3 port.
D435 is useful as an obstacle sensor for navigation stacks in ROS.

(I want to make like this https://twitter.com/search?q=D435%20T265&src=typd )

Only low resolution depth image is needed for navitgation stack, and USB2 have enough band width for the depth image.

thank you

@Furane
Copy link

Furane commented Jun 26, 2019

I have the same problem. I can't get metadata from realsense (specially timestamp) using an odroid xu4.
I already installed the patch from freemanlo, did not work. I also checked ON the option ENFORCE_METADATA in cmake and I got no result.
Did you find a solution?

@Tomatower
Copy link

After seeing the identical issue; By accident I connected the camera to a USB2-only port.
Solved it by plugging the Camera into an USB3-Port.

Maybe the warning could be a bit clearer here :)

@RohithKuditipudi
Copy link

I'm having this same problem but on a different version of the kernel (4.15). Am using USB3 and tried sudo apt-get install realsense2-dkms.

@ev-mp
Copy link
Collaborator

ev-mp commented Jul 28, 2019

@Furane hi, there are no ready-to-run kernel patches or dkms packages for odroid xu, (See above comment)
The easiest way to get the metadata functional is to rebuild the SDK with libuvc backend
(#4516).
If you're experienced with building kernels (especially monolithic) from source, then porting the patches to odroid distro is a viable option that may range between couple of days to a week plus. In that case we'll also try to provide assistance for technicalities. Yet, based on personal experience - I'd recommend this only as a last resort after exhausting the other options.

@Tomatower , the metadata functionality for video4linux is a kernel-enabled feature, and once configured conveys meta-streams for D400 cameras regardless of USB2/USB3 modes. Please file a dedicated ticket if you reproduce this.

@RohithKuditipudi , I'm following this up in your other report (#4516).

@ev-mp ev-mp self-assigned this Jul 28, 2019
@ev-mp ev-mp closed this as completed Aug 18, 2019
@MartyG-RealSense MartyG-RealSense mentioned this issue Apr 21, 2020
Closed
@MartyG-RealSense MartyG-RealSense mentioned this issue Feb 5, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ev-mp ev-mp

Labels
installation Linux
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
9 participants
@akirayou @Tomatower @Furane @vladsterz @ev-mp @RohithKuditipudi @mattiagrassi93 @RealSenseCustomerSupport @paLeziart