Open_Manipulator_AR_Marker Timestamp Issue (Realsense D435)

[MarcoDotIO]

Hello,

I am opening this issue in regards to an issue I've been having involving RViz and the D435. This issue revolves around a time synchronization issue occurring between the two, with this error being outputted. Also note, this error occurs commonly between multiple machines. In my case, I am running on a single machine. :

[ERROR] [1575440969.925629154]: Lookup would require extrapolation into the future. Requested time 3150585447.856007099 but the latest data is at time 1575440969.935363491, when looking up transform from frame [camera_color_optical_frame] to frame [world]

Here are the following pieces of information to gather from:
Distro - Ubuntu 18.04.3 LTS
Linux kernel - 5.0.0-36-generic
RViz version - 1.13.5
Realsense Firmware - 5.12.0.0
Published topics - https://cdn.discordapp.com/attachments/643580745113403455/651648739567599616/output.txt
Package summary - https://ghostbin.co/paste/t2e7c

If any other information is needed, I'll provide it.

[doronhi]

Sorry, I don't understand the issue.
realsense2_camera does not provide a "world" frame. That comes out of another node.
So I guess the error message has something to do with configuration.
I am aware of some timestamp issues involving the "unite_imu_method" option but it seems you did not use it.
Can you provide some kind of test, set of instructions, that I can perform and reproduce the issue on my computer? Preferably with as little number of nodes as possible.

[RealSenseCustomerSupport]

---

@MaidenOfTheMain Could you please provide more information as doronhi requested so that we can look into it further? Thanks!

[RealSenseCustomerSupport]

---

@MaidenOfTheMain Could you please update? Thanks!

[MarcoDotIO]

@MaidenOfTheMain Could you please update? Thanks!

Hello,

Sorry about the long term delay. Anyways I am still having issues with this problem, so I will be more than happy to provide the information needed. The steps I've performed are exactly as listed on this webpage provided by Robotis:
http://emanual.robotis.com/docs/en/platform/openmanipulator_x/ros_applications/#ros-applications

I installed the ROS packages described along with the drivers for the Realsense 2, as they listed. I launched it as they described it as well. And that is where, in the terminal where the command they list for launching ar_pose.launch, is the error starts showing up.

However, as of late, I've gotten the same message, but with times closer to each other, something like this:

[ERROR] [1581310070.265468835]: Lookup would require extrapolation into the past. Requested time 1581307462.967501640 but the earliest data is at time 1581310064.897853526, when looking up transform from frame [camera_color_optical_frame] to frame [world]

I'm using a different USB C cable that came provided, and the cable can be found here:
https://www.tabletdockshop.com/en/charging-cable-50cm-100cm-black-usb-usb-c-connecto.html

Edit: I should mention I tried the same steps above, same exact steps, on Ubuntu 16.04. And the error doesn't occur. So is this issue revolving around ROS / Ubuntu / Linux versions?

[RealSenseCustomerSupport]

---

@MarcoDotIO Did you apply the patch for kernel 5.0 as below? If not, please try to patch and test again.

https://github.com/IntelRealSense/librealsense/blob/development/scripts/realsense-camera-formats-bionic-5.patch
https://github.com/IntelRealSense/librealsense/blob/development/scripts/realsense-metadata-bionic-5.patch

[MarcoDotIO]

@MarcoDotIO Did you apply the patch for kernel 5.0 as below? If not, please try to patch and test again.

https://github.com/IntelRealSense/librealsense/blob/development/scripts/realsense-camera-formats-bionic-5.patch
https://github.com/IntelRealSense/librealsense/blob/development/scripts/realsense-metadata-bionic-5.patch

Thank you for the quick response. For anyone who's also having this problem, including myself, would you be able to provide a resource on applying such a patch? I have no experience with patching the Linux kernel myself.

[RealSenseCustomerSupport]

---

@MarcoDotIO Please refer to the script below to apply the patch.
https://github.com/IntelRealSense/librealsense/blob/master/scripts/patch-ubuntu-kernel-4.16.sh

Or else you can also apply the patch manually.

[MarcoDotIO]

@RealSenseCustomerSupport Thank you for providing the patch file, though it seems like the shell file pertains to kernel version 4.16 rather than 5.0.0. And I did try running it and had this expected output from the script:

The patch is applicable for kernel version 4.16. / For earlier kernels please use patch-realsense-ubuntu-lts.sh script

Should I go ahead and use patch-realsense-ubuntu-lts.sh?

[RealSenseCustomerSupport]

---

@MarcoDotIO Currently no such patch apply script specified for kernel 5.0, and you can refer to the steps in the script of kernel 4.16 to apply the patch manually for kernel 5.0. Or else you can use DKMS package referring to https://github.com/IntelRealSense/librealsense/blob/master/doc/distribution_linux.md

[doronhi]

There is a problem with frame timestamps. It was fixed in librealsense PR#5751 and will be released in the next version.

It could be that the problem your are having is related to frames' timestamps.
Try running the following commands, a different terminal, while rviz prints its error messages:
rostopic echo /camera/color/image_raw/header/stamp
or
rostopic echo /camera/depth/image_rect_raw/header/stamp

Are the timestamps advancing normally or are they going back and forth?

[RealSenseCustomerSupport]

---

@MarcoDotIO Any other questions with the reply of doronhi? Looking forward to your reply. Thanks!

[RealSenseCustomerSupport]

---

@MarcoDotIO Could you please update? Thanks!

[RealSenseCustomerSupport]

---

@MarcoDotIO PR#5751 IntelRealSense/librealsense#5751 merged in latest SDK v2.33.1 . Could you please have a try? Thanks!

[MarcoDotIO]

Hello,

Apologies about the delay, the problem with the AR markers not being mapped seems to be solved with the latest update with SDK v2.33.1. Thank you guys so much for helping out!