How to add a RGB-D sensor to the robot in Isaac Sim

[ge79puv]

Hey there,

now I use Isaac SIm 4.1.0, and I am trying to add a RGB-D sensor to the wirst of my manipulation robot. I found there is already Realsense 455 intergrated in the software, so I tried to add this directly. However, there is an error about the "missing reference found". And I also cannot find the corresponding usd file mentioned in https://docs.omniverse.nvidia.com/isaacsim/latest/features/environment_setup/assets/usd_assets_sensors.html. Could you tell me how to solve this problem?
Moreover, it's better for me to add Realsense D435i sensor into the simulator, because there is already one available in the labor. Could you tell me if there is any method to add another RGB-D sensor in the Isaac Sim?

Best regards,
Yunfeng

[hang-yin]

Hi Yunfeng @ge79puv , thanks for reaching out!

Unfortunately OmniGibson is not compatible with everything Isaac Sim supports. For you question specifically, if you are using OmniGibson instead of vanilla Isaac, you can definitely add extra RGB-D cameras. I'm going to need a little bit of time tomorrow to make a small infra change for wrist camera (same question here #1004), but after that you should be able to do it. This will give you the proper camera stream, but would not show up as a physical camera in the scene. If you do want that, one way you could do it is by modifying the urdf of the robot you are using and importing it as a custom robot into OmniGibson: https://behavior.stanford.edu/omnigibson/tutorials/custom_robot_import.html. For questions about custom robots, @wensi-ai can probably answer them a lot better than me.

[hang-yin]

Hi @ge79puv , please take a look here #1004 (comment) ! Let me know if this helps!