SR305 depth image incorrect on Jetson nano with BUILD_WITH_CUDA=true

[rkent]

---

Required Info
Camera Model	SR305
Firmware Version	3.27.3.0
Operating System & Version	Ubuntu 18.04 (Jetson Nano)
Kernel Version (Linux Only)	Linux 4.9.140-tegra aarch64
Platform	Jetson Nano
SDK Version	commit bc64e52
Language	C++
Segment	depth camera
CUDA version	10.0.326

Issue Description

I tried installing anSR305 depth camera on a Jetson Nano. I've used both the binary distribution, as well as compiling from source. In both cases, the depth camera returns incorrect results. Looking at the image on realsense-viewer, it appears to be an actual grayscale image instead of a depth field. The reported depth from depth field, when it should be about .3 meters, reports instead something much smaller, like .02 meters. This same camera works fine on an x86 Ubuntu system.

After playing with the API for a couple of days to see if I could debug the problem, I sort of by accident discovered that the problem goes away if I compile with BUILD_WITH_CUDA=false instead of true.

[MartyG-RealSense]

Hi @rkent Is CUDA installed on your Nano please?

https://devtalk.nvidia.com/default/topic/1048926/jetson-nano/unable-to-install-cuda-on-jetson-nano-help/71911

[rkent]

Yes CUDA is installed. /usr/local/cuda-10.0

I checked paths. I had the PATH correct but not the LD_LIBRARY_PATH. I fixed that, did a clean recompile, but it did not fix the issue or change its nature.

[MartyG-RealSense]

When you installed using the Debian packages, did you use the JetsonHacks Nano '5 minute installation's guide, please?

https://www.jetsonhacks.com/2019/12/22/install-realsense-camera-in-5-minutes-jetson-nano/

This method utilises Intel's dedicated install instructions for Jetson:

https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_jetson.md

Since the Librealsense Debian packages have CUDA support included in them, this method would seem to be the most straightforward way to achieve a CUDA-enabled Librealsense on Nano if you have not followed this method already.

[rkent]

"achieve a CUDA-enabled Librealsense on Nano" is not really my goal. My goal was to use the SR305 somehow on the Nano, which so far I can only achieve by building myself with CUDA disabled. I am reporting my experiences in case it help others. I don't believe I need CUDA support to accomplish my goals.

To answer your question directly, I tried all kinds of things for days to try to get the camera to work, including the various Jetson hacks recommendations. But to demonstrate cleanly the issue, I did a clean test of the jetson hacks install. I reimaged an SD card with a fresh image from Nvidia (jetbot_image_v0p4p0.zip). I then followed the directions in "Install RealSense Camera in 5 minutes – Jetson Nano" namely:

$ git clone https://github.com/JetsonHacksNano/installLibrealsense
$ cd installLibrealsense
$ ./installLibrealsense.sh

I restart, plugin my SR305 (using the supplied USB-3.0 cable), and run realsense-viewer. Same experience as before. RGB camera works fine. For the depth camera, at first you see nothing. Playing around, post-processing must be turned off to see anything, particularly the threshold filter. This is because, by default, it throws away anything with a reported depth less than 0.1 meters. But once I get a picture, I see that the reported depth is low, like .03 meters. Changing the color pattern to a black-to-white pattern, it is clear that I am seeing an intensity image, not a depth image.

I can use the same camera on my x86 Ubuntu computer and it works fine, the reported depths make sense and the image from the depth sensor is clearly a depth image and not an intensity image.

I am happy to do other tests if it would be helpful to diagnose this better if it would be helpful to others. But for my own purposes, by building librealsense with CUDA disabled I can get the camera to do what I need it to do.

[MartyG-RealSense]

Thanks so much for your detailed testing and reporting! It's very valuable to have guidance for current and future SR305 Nano users.

[rkent]

I spoke too soon saying '"achieve a CUDA-enabled Librealsense on Nano" is not really my goal'. With CUDA disabled, running the SR305 required 100% of two of the four processors. So I investigated further what the issue is.

Turns out that whatever the problem is occurs only in a single file (src/proc/depth-formats-converter.cpp) If I disable the use of CUDA just in that file (by replacing #ifdef RS2_USE_CUDA with #ifdef RS2_USE_CUDA_NOT) then depth images work. Now one CPU at 85% is needed to run the camera, so whatever that change did is not really needed for significant CUDA performance improvements.

I've implemented that change in my fork https://github.com/rkent/librealsense which is intended for my personal use.

[MartyG-RealSense]

Thanks very much for the update - I'm glad you were able to develop a solution that works for you!

[MartyG-RealSense]

Do you require any further assistance with this case or can it be closed, please? Thanks!

[rkent]

That depends on the intent of your use of issues. I am accustomed to issues being documentation of known bugs needing fixing. If that is the case, then I have done my best to document a bug, and isolate it to a few lines of code. That bug still remains and affects use of your product. By that definition, this case should not closed.

As for my person use, I have developed a workaround that works for me. I have no further personal need of assistance.

So you decide according to your policies whether this should be closed.

[MartyG-RealSense]

The effort that you have made in documenting your problem really is appreciated.

An element for consideration in whether an issue should be officially classified as a Bug to be investigated is whether it can be reproduced by others. As your report is the only one that I know of in relation to depth image integrity on CUDA on SR305, I will close it for now. The information will still be findable by others in a web search and the matter can be revisited in future if others report the same problem. Thanks again!