Vision Sensor and C++ remote API's

Typically: "How do I... ", "How can I... " questions
Post Reply
Msalmanch
Posts: 13
Joined: 06 Feb 2017, 19:07

Vision Sensor and C++ remote API's

Post by Msalmanch » 22 Mar 2017, 08:24

I am working on a project which incorporates using Image Sensor in communication with visual studio through remote APIs with C++. I am attaching the code for reference. The code I am using exhibits some strange behaviors listed below

1. simxGetVisionSensorImage initially returns "0" 10 to 15 times, during which image and resolution variables gives the value to which they are initialized and image returns bad pointer.

2. After 10 to 15 time the response of simxGetVisionSensorImage changes to "1" during which only resolution of x of image is read in variable resy, however, resx still remains to value to which it is initialized. Upon debugging the image field presents 0 and nothing else.

I am confused where the exact issue lies, any suggestions would be highly appreciated.

Code: Select all

   #include <iostream>
#include <conio.h>
#include <stdlib.h>
using namespace std;
extern "C" 
{
  #include "extApi.h"
}



int main(int argc, char** argv)
{
	simxFinish(-1);                //closes all connections
	simxInt clientID=simxStart("127.0.0.1",19999,'True','True',2000,5);   //connect to v-rep
	simxInt camera;//reciever of object handles
	simxInt resx=0,resy=0;
	simxUChar* Image=0;
	if (clientID != -1)
	{
		simxAddStatusbarMessage(clientID,"Hello, V-rep can you hear me",simx_opmode_oneshot);       //send a message to v-rep
		simxGetObjectHandle(clientID,"Camera1",&camera,simx_opmode_blocking);//get handle of vision sensor
		simxGetVisionSensorImage(clientID,camera,(&resx,&resy),&Image,0,simx_opmode_streaming);
}
	else
		{
		cout << "Connection Impossible, possible sources are simulation is not running, port number is not true, ip address is not true" << endl;
		return (0);
		}
	while (clientID != -1)
	{
		simxGetVisionSensorImage(clientID,camera,(&resx,&resy),&Image,0,simx_opmode_buffer);
		&Image[1];
	cout << resy << endl;
	}
	
      getch();
	}

coppelia
Site Admin
Posts: 7330
Joined: 14 Dec 2012, 00:25

Re: Vision Sensor and C++ remote API's

Post by coppelia » 22 Mar 2017, 13:57

Hello,

there are two reasons why an image wouldn't come at first:

1. At simulation start, there is not image, until the first time simHandleVisionSensor was called on that vision sensor (by default in the main script). But I don't believe this is your problem.
2. When you start streaming, it will take a few ms until the first image has arrives. So it is very important to check for the return value of your function. You don't even check the return value. Something like:

Code: Select all

int retVal=simxGetVisionSensorImage(clientID,camera,&resolution,&Image,0,simx_opmode_buffer);
if (retVal==simx_return_ok)
    cout << resolution[1]; 
Also you are using a funny construction for the x/y resolution, this is the problem.

Cheers

Msalmanch
Posts: 13
Joined: 06 Feb 2017, 19:07

Re: Vision Sensor and C++ remote API's

Post by Msalmanch » 22 Mar 2017, 14:23

Thanks for your reply, many things are clear now. Can you please let me know the exact construction for x/y resolution i.e. deceleration of variables and using them in function. I am stuck for quite a while now, thanks in advance.

coppelia
Site Admin
Posts: 7330
Joined: 14 Dec 2012, 00:25

Re: Vision Sensor and C++ remote API's

Post by coppelia » 22 Mar 2017, 15:21

Here some code for receiving and sending an image in C++, from a remote API client:

Code: Select all

#include <stdio.h>
#include <stdlib.h>

extern "C" {
    #include "extApi.h"
}

int main(int argc,char* argv[])
{
    int portNb=0;
    int leftMotorHandle;
    int rightMotorHandle;
    int sensorHandle;

    if (argc>=5)
    {
        portNb=atoi(argv[1]);
        leftMotorHandle=atoi(argv[2]);
        rightMotorHandle=atoi(argv[3]);
        sensorHandle=atoi(argv[4]);
    }
    else
    {
        printf("Indicate following arguments: 'portNumber leftMotorHandle rightMotorHandle sensorHandle'!\n");
        extApi_sleepMs(5000);
        return 0;
    }

    int clientID=simxStart((simxChar*)"127.0.0.1",portNb,true,true,2000,5);
    if (clientID!=-1)
    {
        int driveBackStartTime=-99000;
        float motorSpeeds[2];

        int passive;
        int active;
        simxGetObjectHandle(clientID,"PassiveVision_sensor",&passive,simx_opmode_blocking);
        simxGetObjectHandle(clientID,"Vision_sensor",&active,simx_opmode_blocking);
        int resolution[2];
        simxUChar* image;
        simxGetVisionSensorImage(clientID,active,resolution,&image,0,simx_opmode_streaming);

        while (simxGetConnectionId(clientID)!=-1)
        {
            if (simxGetVisionSensorImage(clientID,active,resolution,&image,0,simx_opmode_buffer)==0)
            {
                simxSetVisionSensorImage(clientID,passive,image,resolution[0]*resolution[1]*3,0,simx_opmode_oneshot);
            }


            simxUChar sensorTrigger=0;
            if (simxReadProximitySensor(clientID,sensorHandle,&sensorTrigger,NULL,NULL,NULL,simx_opmode_streaming)==simx_return_ok)
            { // We succeeded at reading the proximity sensor
                int simulationTime=simxGetLastCmdTime(clientID);
                if (simulationTime-driveBackStartTime<3000)
                { // driving backwards while slightly turning:
                    motorSpeeds[0]=-3.1415f*0.5f;
                    motorSpeeds[1]=-3.1415f*0.25f;
                }
                else
                { // going forward:
                    motorSpeeds[0]=3.1415f;
                    motorSpeeds[1]=3.1415f;
                    if (sensorTrigger)
                        driveBackStartTime=simulationTime; // We detected something, and start the backward mode
                }
                simxSetJointTargetVelocity(clientID,leftMotorHandle,motorSpeeds[0],simx_opmode_oneshot);            
                simxSetJointTargetVelocity(clientID,rightMotorHandle,motorSpeeds[1],simx_opmode_oneshot);           
            }
            extApi_sleepMs(5);
        }
        simxFinish(clientID);
    }
    return(0);
}
Cheers

Msalmanch
Posts: 13
Joined: 06 Feb 2017, 19:07

Re: Vision Sensor and C++ remote API's

Post by Msalmanch » 24 Mar 2017, 14:46

Thanks a lot, the resolution problem is solved. One last issue, how to access the image. I need to access the image array so that i can detect the rgb values. Please can you give an example code for accessing the image. I think I am using the wrong syntax.

Thanks for all help in advance. :)

idkhowtovrep
Posts: 2
Joined: 27 Aug 2019, 04:32

Re: Vision Sensor and C++ remote API's

Post by idkhowtovrep » 04 Sep 2019, 09:23

Hi,

I'm using your last code example on receiving and sending image in c++.

I'm not too sure where I'm going wrong, as the program is just stuck in the while loop.

The code I'm using is:

Code: Select all

while (simxGetConnectionId(clientID) != -1)
		{
			std::cout << "while" << std::endl;
			if (simxGetVisionSensorImage(clientID, camera, resolution, &image, 0, simx_opmode_buffer) == 0)
			{
				simxSetVisionSensorImage(clientID, passive, image, resolution[0] * resolution[1] * 3, 0, simx_opmode_oneshot);
				std::cout << "if" << std::endl;
			}
		}
Thank you in advance for the help

coppelia
Site Admin
Posts: 7330
Joined: 14 Dec 2012, 00:25

Re: Vision Sensor and C++ remote API's

Post by coppelia » 05 Sep 2019, 05:36

You are trying to read the image from the internal buffer (operation mode simx_opmode_buffer). You first need to enable streaming with operation mode simx_opmode_streaming:

Code: Select all

simxGetVisionSensorImage(clientID, camera, resolution, &image, 0, simx_opmode_streaming);
while (simxGetConnectionId(clientID) != -1)
{
    std::cout << "while" << std::endl;
    if (simxGetVisionSensorImage(clientID, camera, resolution, &image, 0, simx_opmode_buffer) == 0)
    {
        simxSetVisionSensorImage(clientID, passive, image, resolution[0] * resolution[1] * 3, 0, simx_opmode_oneshot);
        std::cout << "if" << std::endl;
    }
}
Cheers

Post Reply