How to implement a (more realistic) depth sensor with shadows?

Typically: "How do I... ", "How can I... " questions
Post Reply
Uli_v
Posts: 35
Joined: 14 Dec 2018, 18:07

How to implement a (more realistic) depth sensor with shadows?

Post by Uli_v » 06 Sep 2019, 17:46

Hi,

I saw that V-REP supports rendering with shadows. Can this also work for depth sensors?

I would like to have a depth sensor (based on a structured light, like Intel RealSense SR300) that produces shadows. Shadows (missing depth information) are caused because the location of the IR projector and IR camera on the sensor is different. This causes that some surfaces in the IR camera are visible that do not have IR pattern (I call these "shadows"). In the depth image, these pixels won't have any depth information (e.g. depth value = 0). How can I easily implement a more realistic depth sensor with shadows in V-REP? Thanks.

Uli

coppelia
Site Admin
Posts: 7330
Joined: 14 Dec 2012, 00:25

Re: How to implement a (more realistic) depth sensor with shadows?

Post by coppelia » 09 Sep 2019, 06:29

Hello Uli,

ok, I see what you are trying to achieve...
Basically, if there was a possibility to render a scene with the OpenGL3 mode, but by selecting only one light source and rendering a binary image (1=lit, 0=shadow), then one could modify the depth map appropriately (i.e. by clearing depth map areas where the RGB image is 0). Is that correct?
Currently this is not possible. Probably that could be implemented with not too much effort. Also, in next release, the vision sensor filters will be gone. Instead, there will be a vision callback that allows to handle image processing in a much more flexible way than currently.

Cheers

Uli_v
Posts: 35
Joined: 14 Dec 2018, 18:07

Re: How to implement a (more realistic) depth sensor with shadows?

Post by Uli_v » 09 Sep 2019, 15:13

Yes, the description your approach is correct. I had something like that in mind. Could that be added in the next release? When is the next release due? Thanks.

Post Reply