Makefile Arm SSE fix

[jetsonhacks]

Currently the Makefile for librealsense does not work on the Jetson TK1. This patch checks to see if SSE is available on the CPU before enabling the -msse flags. The TK1 is an ARM processor, which does not have SSE instructions, and the compiler chokes when it sees a SSE flag enabled.

[Maghoumi]

Thanks a lot Jim (@jetsonhacks) ! I wanted to fix that for the longest time but didn't have access to a TK1. I will look at the request later.
Also, could you please confirm that the latest version of this fork works on TK1? I updated to the original repo a couple of days ago and somebody told me that it stopped working for them on the Odroid board. It still works on my TX1.
Thanks again!

[jetsonhacks]

Hi Mehran,

In the request, I tried to look to see if SSE is available on the
processor instead of examining the architecture.

The latest fork does not work with the TK1, the RGB stream is black. I
haven't been able to figure out why yet. There could be a conflict with
the image patch, but I haven't explored enough to know.

I have recompiled a TK1 Linux kernel with the patches in the
librealsense library on the TK1 on an older version, V4L2 seems to work ok.

I've been playing with workarounds to get the camera running without the
plug/replug between program invocations:

https://github.com/jetsonhacks/setupR200-Jetson

The issue appears to be that the video streams halt after being used the
first time. The workaround tries to clear the halts before use. A call
to r200JetsonSetup() or videoStreamsClearHalts() at the beginning of a
program should get everything lined up and ready to go.

This generally clears the -EPIPE error that I'm sure you've seen.
However it doesn't appear to be 100% effective, sometimes the video
stalls on the streams and the streams timeout. Usually running the
program a few times will clear this and everything will work normally
again. The camera does not appear to shut down correctly when a program
is terminated (at the very least the streams stall and the red light
stays on). I don't know where this occurs in the usb stack, on the
camera, drivers, or kernel. I've spent way too long looking at it, I
think to investigate further takes a USB hardware analyzer and a better
debugging setup than I have.

I also converted the Qt Creator files over to use the UVC backend:

https://github.com/jetsonhacks/R200-UVC-Examples

In the examples folder there is sample which uses the workaround
mentioned above. Eventually I'm hoping to consolidate all of this.

Looking through the changes to your code base, I think one thing to
consider is submitting a pull request to the Intel repository with your
SSE workaround patches. That way you don't have to end up maintaining
the entire code base that they're creating there.

Best,

Jim

On 6/2/16 11:09 AM, Mehran Maghoumi wrote:

Thanks a lot Jim (@jetsonhacks https://github.com/jetsonhacks) ! I
wanted to fix that for the longest time but didn't have access to a
TK1. I will look at the request later.
Also, could you please confirm that the latest version of this fork
works on TK1? I updated to the original repo a couple of days ago and
somebody told me that it stopped working for them on the Odroid board.
It still works on my TX1.
Thanks again!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#2 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AIDJzeUTMg9aH0VK6usxqVWKOgt8utHLks5qHxxegaJpZM4Is03r.

[jetsonhacks]

Two more points of information:

1. There have been changes to the GLFW library recently that break everything on the Jetson TK1. The symptom is that the app is started, and a graphics window does not open, sometimes with the machine going into a hard lock. The latest version that the Jetson TK1 works on is glfw 3.1.2, the current revision appears to be 3.2. It could be that other machines have this issue too.

The script that loads the glfw library in the librealsense scripts folder loads the 'latest' tag, which equates to 3.2 (or greater going forward).

2. Reverting the last change to src/image.cpp (removing the patch for the UDOO fix) fixes the black RGB stream on the Jetson TK1. There is also a conflict marked against the Intel repository master in that file.

[AutoModelCar]

@jetsonhacks Could you link to the repository which is worked! I received a black image as well.

[Maghoumi]

OK thanks for the responses @jetsonhacks, I will look into this soon.
@AutoModelCar You could revert to a commit before I updated everything to reflect the latest changes in the original repository.

[ZahraBoroujeni]

I am working on SSE2NEON, I could compile librealsense using neon, and RS_FORMAT_Y8 works good, but RS_FORMAT_RGB8 still does not work correctly.
https://github.com/ZahraBoroujeni/librealsense
I think the problem is in "_mm_alignr_epi8" function which I could not find proper function in neon for it.
or maybe _mm_shuffle_epi8 which I rewrote it using for()!!

[Maghoumi]

OK, I've been able to reproduce the problem on my Jetson TX1. I'm now getting the hard freezes. I am currently looking into it and it may take a couple of days. I apologize for any inconveniences.

@jetsonhacks I will incorporate this pull request after I've fixed the problem with the sensor not working at all.

Looking through the changes to your code base, I think one thing to consider is submitting a pull request to the Intel repository with your SSE workaround patches. That way you don't have to end up maintaining the entire code base that they're creating there.

Unfortunately, that is not possible currently as I'm 99% sure the PR will get rejected due to the ARM support not being robust at all. I will plan to send them a PR once I've found a reliable ARM NEON implementation.

@ZahraBoroujeni That's great news! I will look into your repository when I find some time on my hands.

[Maghoumi]

I pushed a new version and incorporated @jetsonhacks fixes.
@ZahraBoroujeni @jetsonhacks @AutoModelCar Let me know if the latest version fixes your issues.

[ZahraBoroujeni]

I can confirm that I can see the RGB/IR/Depth images now with the new version.

My Platform and situation:

• Odroid xu4
• patched ubuntu 14.04 (add new uvcvideo.ko)
• sr300
• make librealsense using "make BACKEND=LIBUVC && make install"
• 30 Hz
• ROS indigo
• glfw 3.2/3.1.2 (works for both the same!)

Situation: The second run works. I do not need to plug/unplug the camera, I need to run the program twice!
and Always the error below occurs after first run (odd runs):

[ERROR] [1356999361.466719947]: RealSense Camera - Error calling rs_create_context ( api_version:4 ): 
libusb_bulk_transfer(...) returned LIBUSB_ERROR_TIMEOUT 

[FATAL] [1356999361.635365197]: Service call failed!
[SR300_nodelet_manager-3] process has died [pid 14572, exit code 1, cmd /opt/ros/indigo/lib/nodelet/nodelet manager __name:=SR300_nodelet_manager __log:=/root/.ros/log/39a0b54c-53a6-11e2-9cb6-7cdd908e0595/SR300_nodelet_manager-3.log].
log file: /root/.ros/log/39a0b54c-53a6-11e2-9cb6-7cdd908e0595/SR300_nodelet_manager-3*.log
[camera_nodelet-4] process has died [pid 14585, exit code 255, cmd /opt/ros/indigo/lib/nodelet/nodelet load realsense_camera/RealsenseNodelet SR300_nodelet_manager rgb/image_raw:=app/camera/rgb/image_raw camera/depth/image_raw:=depth/image_raw camera/color/image_raw:=rgb/image_raw camera/infrared1/image_raw:=ir/image_raw camera/infrared2/image_raw:=ir2/image_raw camera/depth/points:=depth/points camera/depth/camera_info:=depth/camera_info camera/color/camera_info:=rgb/camera_info camera/infrared1/camera_info:=ir/camera_info camera/infrared2/camera_info:=ir2/camera_info __name:=camera_nodelet __log:=/root/.ros/log/39a0b54c-53a6-11e2-9cb6-7cdd908e0595/camera_nodelet-4.log].
log file: /root/.ros/log/39a0b54c-53a6-11e2-9cb6-7cdd908e0595/camera_nodelet-4*.log

[Maghoumi]

@ZahraBoroujeni well glad to hear that at least it works most of the times! I'd blade that on the fact that your board is not "officially" supported. Heck, people are still having problems with "supported" boards too (you can tell by looking at the issues reported in the original repository). Also, RealSense sensors are all still dev-kits and are not consumer-ready yet, so issues are expected.
Looking forward to a working ARM NEON version : )
By the way, what did you use to patch your kernel? Do you have a how-to guide for that or something?

[jetsonhacks]

@Maghoumi this works with the Jetson TK1 and R200 camera.

[ZahraBoroujeni]

@Maghoumi sorry for the late answer. To Patch the kernel for odroid,

• git clone the Linux kernel https://github.com/hardkernel/linux
• apply patch, if patch does not work, you can manually change three files inside drivers/media/usb/uvc/ (uvc_driver.c uvcvideo.h videodev2.h) based on the patch file, you should add some lines to the three files, and then you can make a new patch for yourself for future.
• We should cross compile the Linux kernel for the arm Processor, use old-config and make a new config : make ARCH=arm CROSS_COMPILE=path/to/compiler/bin/arm-linux-gnueabihf- oldconfig
• make ARCH=arm CROSS_COMPILE=/path/to/compiler/bin/arm-linux-gnueabihf- -j 4 zImage
• go to uvc folder: cd drivers/media/usb/uvc
• make -C /path/to/linux M=$PWD ARCH=arm CROSS_COMPILE=/path/to/compiler/bin/arm-linux-gnueabihf- modules
• Now you have replace uvcvideo.ko file in your arm board with new one! you can copy the file to lib/modules/uname -r/kernel/drivers/media/usb/uvc/

[Maghoumi]

@ZahraBoroujeni Thanks.
The problem is that Jetson boards (the TX1 at least) are still using kernel v3.10! I will soon check and see if the patch can still be applied to that kernel on the TX1 (as mentioned in the beginning of the post, TK1 seems to be working fine).

[jetsonhacks]

@Maghoumi
I built patch files that have been tested on the TK1 for the UVC kernel module, available at:
https://github.com/jetsonhacks/installLibrealsense
They are located in the UVCKernelPatches directory. Note that on the TK1 I switched uvcvideo to be a module instead of built-in.

I have not looked at bringing this code over to the TX1, but I think that the sources for both kernels for uvcvideo are the same, so it should work.

[I0x0I]

The arm.patch in jetsonhacks' repo works for me, maybe consider adding some macros for arm devices in src/uvc-v4l2.cpp?
I'm using

• Odroid XU4
• Ubuntu 15.04 (kernel 3.10.82-56 with new uvcvideo.ko)
• Intel R200
• latest version librealsense with patched V4L2 backend

When running programs for the first time, there might arise error like

RealSense error calling rs_create_context(api_version:5):
UVCIOC_CTRL_QUERY:UVC_SET_CUR error 5, Input/output error

dmesg is like

[ 1994.530289] [c0] uvcvideo: Failed to query (SET_CUR) UVC control 1 on uni.
[ 2331.871794] [c0] uvcvideo: Failed to set UVC commit control : -32 (exp. 34).
[ 2331.877472] [c0] uvcvideo 4-1.2:1.1: resume error -5
[ 2331.884556] [c0] uvcvideo: Failed to set UVC commit control : -32 (exp. 34).
[ 2331.890215] [c0] uvcvideo 4-1.2:1.3: resume error -5

replug the device seems to solve that.

The frame rate of cpp-alignimages is 3 to 4 HZ and takes up 144% cpu resource, is it because I'm using R200 instead of SR300?

Thanks to all of you guys!

[ZahraBoroujeni]

It seems it does not patched correctly! Because it does not find UVC commit control!
You can try the image from the linke below! it is reported that the image(ubunt 14.04) works also for R200. But I did not test it personally.
https://github.com/AutoModelCar/AutoModelCarWiki/wiki/Flashing-the-eMMC

"Hint: do not pay attention about the name, the image works for xu4, "

[I0x0I]

Thanks for your help ;-) @ZahraBoroujeni, it turns out that it's a missing OpenGL library causing the cpp-alignimages and other examples run at low frame rates, I fixed it with cv::imshow and now it works at 30HZ.
I also recompiled the kernel to make the xhci-hcd to be an external module, which enables me to reload the module instead of physically replug the device when the mentioned errors occur. This trick seems to only work on the V4L2 backend library.