How the Vision Set works


The Vision Set uses a 2D camera to detect objects and calculate their positions in space. This detection allows you to trigger the actions of your choice.

To allow such functions with a 2D camera, we developed a system based on landmarks that define workspaces in 3 dimensions (the third dimension being the height, taking the height of the landmarks as origin).


For each action related to Vision, the camera will give you a relative position in the workspace reference that the robot will translate to its own reference.


Please note that any action related to Vision will need to be done in a workspace, which means all 4 markers should be detectable by the camera

On the figure below, you can see an image from Ned’s camera streaming with the Landmarks filter. This filter makes sure that the robot is able to detect all 4 markers.

Workspace debug


The first landmark is slightly different from the others : it has only 3 blacks areas, which allows to sort the landmarks clockwise


To use the Vision related functions, two conditions are required:
  • Workspace’s landmarks positions are set and mechanically fixed
  • The landmarks delimiting the workspace are visible
    • Objects that are too close to the landmarks are not entirely visible by the camera
    • The tool used at the end of the arm can reduce the visual field of the camera


Please note that these two conditions need to be met to allow the Vision related operations.