Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Getting T265 fisheye image? #857

Closed
triplehoon opened this issue Apr 26, 2022 · 8 comments
Closed

Getting T265 fisheye image? #857

triplehoon opened this issue Apr 26, 2022 · 8 comments

Comments

@triplehoon
Copy link

Hi. I am trying to use RTAPMAP C++ with T265 + D400 device.

And I use rtabmap::CameraRealSense2 set as dual mode.

And I wanna use T265 Fisheye camera frame, but the only I can takeImage() from D455.

Is there any way to get T265 fisheye camera image?

@sumitsarkar1
Copy link

in dual mode T265 provides odometry and D400 provides point cloud...based on odometry the point cloud of D400 get stitched...If you can point out what exactly you want to use the T265 camera frame for may be it will be more helpful..

@triplehoon
Copy link
Author

@sumitsarkar1
I just need to get a T265 fisheye camera image while I using Rtabmap.
T265 image is not for any SLAM algorithm, just want it to show up

@sumitsarkar1
Copy link

sumitsarkar1 commented May 7, 2022

I guess on ROS platform you can do...just view the topic published in rviz...if your are using standalone RtabMap then just check insideCameraRealSense2.cpp if its possible

@triplehoon
Copy link
Author

@sumitsarkar1
Thanks for your advice about using ROS. But I am using standalone RtabMap, because it is very hard to use ROS with my Windows-based program.
So I've checked CameraRealSense2.cpp.

/// CameraRealSense2.cpp line 1427, captureImage() func which is called by takeImage() func
else if(is_right_fisheye_arrived)
				{
					cv::Mat left,right;
					if(rectifyImages_ && stereoModel_.left().isValidForRectification() && stereoModel_.right().isValidForRectification())
					{
						left = stereoModel_.left().rectifyImage(cv::Mat(rgbBuffer_.size(), rgbBuffer_.type(), (void*)rgb_frame.get_data()));
						right = stereoModel_.right().rectifyImage(cv::Mat(depthBuffer_.size(), depthBuffer_.type(), (void*)depth_frame.get_data()));
					}
					else
					{
						left = cv::Mat(rgbBuffer_.size(), rgbBuffer_.type(), (void*)rgb_frame.get_data()).clone();
						right = cv::Mat(depthBuffer_.size(), depthBuffer_.type(), (void*)depth_frame.get_data()).clone();
					}

					if(stereoModel_.left().imageHeight() == 0 || stereoModel_.left().imageWidth() == 0)
					{
						stereoModel_.setImageSize(left.size());
					}

					data = SensorData(left, right, stereoModel_, this->getNextSeqID(), stamp);
				}

I see the data is kind of getting T265 image, but I can't get SensorData from T265 when I use takeImage() function from the Camera.

I can change source code, and re-build Rtabmap, but I think It is kind of a bad turnaround.

@sumitsarkar1
Copy link

try to do cv::imshow("fisheye",left)

@matlabbe
Copy link
Member

To get fisheye images from T265 with CameraRealsense2, you should use realsense sdk <=2.42 or fix it in latest realsense release.

You would need to calibrate T265 to be used by rtabmap, this can be done following those instructions: http://official-rtab-map-forum.206.s1.nabble.com/Slam-using-Intel-RealSense-tracking-camera-T265-tp6333p6336.html

@triplehoon
Copy link
Author

triplehoon commented May 12, 2022

@matlabbe
Thanks for your comments.
I calibrated T265 and set SDK <= 2.42.

But I can't access to T265 sensor.

My code is

	rtabmap::CameraRealSense2* camera = new rtabmap::CameraRealSense2;
	camera->setResolution(1280,720);
	
	const rtabmap::Transform extrinsics(0.006f, 0.0025f, 0.0f, 0.0f, 0.0f, 0.0f);
	camera->setOdomProvided(true);
	camera->setDualMode(true, extrinsics);
	if (!camera->init(".","943222111057")) //943222111057: T265 serial number
	{
		std::cout << "Setting error \n";
	}
	if (camera->stereoModel_.left().isFisheye())
	{
		std::cout << "Calibrated!" << std::endl;
	}
	

But I get false value for stereoModel_.left().isFisheye().
Also, I can't get fish-eyed images when I checked breakpoint at CameraRealSense2.cpp

///  CameraRealSense2.cpp / SensorData CameraRealSense2::captureImage(CameraInfo * info) / line 1408
	else if(is_left_fisheye_arrived)
			{
				if(odometryOnlyLeftStream_) // <<< set break point here
				{

@matlabbe
Copy link
Member

You should not set dual mode, this makes ignoring T265 image data, only odometry is kept.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants