Aligned Point Cloud using wrong intrinsics?

[jb455]

Required Info
Camera Model	D400
Firmware Version	05.09.11.00
Operating System & Version	Win10
Platform	PC
SDK Version	2.13.0
Language	C++/C#

Hi,
I'm trying to create a point cloud from a depth frame which has been aligned to colour, but I'm getting bad results. I think the problem is that the point cloud is being calculated using the depth camera intrinsics, instead of the colour camera intrinsics which, to my understanding, should be used after alignment.
I'm trying to follow through align.cpp to confirm this and it looks as though create_aligned_profile overwrites the 'to' (colour in this case) intrinsics with the 'original' (depth) intrinsics, causing the issue later on when the point cloud is calculated.

Can anyone confirm this please?

PS: It would be useful to add a 'GetVertices' method to the Points class in the C# wrapper as below, to make the method for obtaining the vertices clearer and easier (with similar for TextureCoordinates):

public Vertex[] GetVertices()
{
    var vertexArray = new Vertex[Count];
    points.CopyTo(vertexArray);
    return vertexArray;
}

[dorodnic]

Hi @jb455
You might be correct regarding align, we need to double-check. That said, why not generate point-cloud from the original depth frame? If I understand the flow correctly, the resulting XYZ values should be the same regardless of view-port and extra alignment is just introducing noise.
Regarding the extra getter - sounds great. Please consider submitting a pull-request. We can certainly integrate your proposal ourselves, but I believe community contribution via GitHub is more elegant and often more efficient.

[jb455]

Hi, thanks for the reply.
The reason I need the point cloud aligned to colour is so I can point to pixels in the colour image and get their vertices so I can do things like measuring on the colour image. So I need the vertices array to be the same size and order as the colour image to make this easy.

Regarding the pull request, do I need to create a fork to my personal git, clone that, make my changes, then submit a pull request from my fork using the web GUI?

[dorodnic]

I see. We will reproduce and address.
We added some step by step instructions about GitHub contributing here.
Please let me know if you have any questions, simplifying community involvement is high priority for us.

[zeraphil]

Just wanted to point out, I'm doing a similar application (need to select color pixels and get their vertices) and am experiencing similar issues. Is there a stopgap workaround?

[jb455]

You can call rs2_deproject_pixel_to_point on each pixel with the correct intrinsics to calculate the point cloud manually.
It's slower, but I only need to calculate the point cloud once so it's not too much of an issue for me.

[zeraphil]

Thanks. Would you happen to know if that method is exposed in C#? I've been searching for it but can't find it in the source or docs...

[jb455]

It is exposed in the API but doesn't look like it's implemented in the wrapper. Should be easy enough to add yourself - just add it in NativeMethods.cs then call it in a static method somewhere. It would probably be easier to just translate the C++ code I linked though and create a new method in your app; you only really need lines 52-53 & 63-65 for most purposes.

[zeraphil]

Got it, I agree. Thanks for the help!

[dorodnic]

Merged to the development branch and going to be part of 2.16 release