[realsense2] high CPU utilization

[jangernert]

Required Info
Camera Model	D435
Firmware Version	librealsense 2.8.0
Operating System & Version	Windows 10
CPU	Intel Core i7 860 (2.8GHz)

First off: The RealSense Viewer that came with the SDK doesn't have this problem for some reason.

But compiling the samples on my pc and writing my own code results in CPU usage >90%.
Compiling the capture example results in CPU spikes of 100% with frames actually being displayed, followed by CPU utilization of 0% and with no new frames being displayed.

But I think that is because of my slow USB 3.0 ports (PCIe extention card) of the workstation I'm on. The RealSense Viewer also displays no frame data if I activate multiple streams with medium or one stream with high resolution.

When I now edit the rs-capture.cpp example and only activate a 640x480 color stream my USB connection has no trouble keeping up anymore. But that also means the CPU utilization is constantly >90%.

rs2::context* ctx = new rs2::context();
rs2::config* conf = new rs2::config();
conf->disable_all_streams();
conf->enable_stream(RS2_STREAM_COLOR, 0, 640, 480, RS2_FORMAT_RGB8, 30);
rs2::pipeline* pipe = new rs2::pipeline(*ctx);
pipe->start(*conf);

We also tried to run the D435 on other hardware with a better USB connection. This of course fixes the problem with only being able to run low res streams, but the CPU utilization is about equally high on those maschines as well.

Profiling the example shows around 70% of the time is spent evaluating while(app). Or to be more precise glfwSwapBuffers(win); which doesn't really make sense to me. So I deactivated all the visualisation code and only gather frames in a while loop. Profiling that code still results in CPU >90% with over 95% of that time spent inside realsense2.dll.

#include <librealsense2/rs.hpp>

int main(int argc, char * argv[]) try
{
    rs2::colorizer color_map;

    rs2::context* ctx = new rs2::context();
    rs2::config* conf = new rs2::config();
    conf->disable_all_streams();
    conf->enable_stream(RS2_STREAM_COLOR, 0, 640, 480, RS2_FORMAT_RGB8, 30);
    rs2::pipeline* pipe = new rs2::pipeline(*ctx);
    pipe->start(*conf);

    while(true)
    {
        rs2::frameset data = pipe->wait_for_frames();

        rs2::frame depth = color_map(data.get_depth_frame());
        rs2::frame color = data.get_color_frame();
    }

    return EXIT_SUCCESS;
}

Any Idea why that is, and why the RealSense Viewer manages to display the same streams with only ~5% CPU?

[jangernert]

I built the master branch of librealsense2 with debugging symbols and profiled my application once again.
The app is just getting color frames via the C API in an infinite loop without displaying them.
Around 15% of the time is spent in rs2_release_frame . About 37% of the time is spent in unpack_yuy2. And most of the CPU time (39%) is spent inside OnReadSample of win-uvc.cpp with 33% just being stream.callback(profile, f, continuation);.

Still not sure what is going on. Or if I messed something up in my code.

[halton]

@jangernert development branch is the right branch

[jangernert]

Okay I'll try again with development ;) But wasn't master synced with development when the 2.8 release happened? master is just around 60 commits behind development.

[jangernert]

Same behaviour with librealsense2 built from development-branch =(

[juergenr]

Same Problem here.
I have > 90% CPU usage for the example apps (capture, hello-world, multicam, pointcloud) too.
Using Windows 10 1709, 64 bit, Intel i7-4790 4GHz
Version 2.8.0, RealSense SR300

[dorodnic]

Hello @juergenr, @jangernert, thanks for reporting this.

Regarding CPU spikes with the SDK (but not the Viewer) - I believe it is related to use of OpenMP. Please try compiling the library after running cmake .. -DBUILD_WITH_OPENMP=false. (See build configuration)
We disable this optimization in the public Viewer to reduce dependencies of the binary release (OpenMP will require Visual Studio Redistributable even when compiling with /MT)
However, disabling OpenMP will increase RGB latency and can cause RGB frame drops in HD / Full-HD resolutions on some systems.

We are actively looking into ways to reduce CPU % and latency internally.
Right now the base implementation of color conversions is SSE2 optimized, optional AVX version can improve things slightly. The other hotspot is projection / alignment logic that is currently done on the CPU. GPU alternatives are also something we are looking into and wish to offer.

[juergenr]

I can confirm, that with -DBUILD_WITH_OPENMP=false the problem is gone.
Thanks!

[jangernert]

I finally had the opportunity to test with the D435 again, and can confirm as well that without OpenMP the CPU goes down significantly.