Use the Vision Set with Blockly¶
Blockly’s blocks have been designed to be used with the Vision Set. It keeps Ned’s Ecosystem easy to use with Niryo Studio.
In this section, we will see how to use the Vision Set with these blocks.
Allows the user to choose a color among the list of available colors.
This color will be used in the Vision Pick Block and the Is detected block.
Allows the user to choose a shape among the list of available shapes.
The selected shape will be used in the Vision Pick Block and the Is detected block.
Vision Pick Block¶
This block will make Ned pick an object matching the Color Block and Shape Block.
The picking process is:
Opening the gripper
Going over the object (5 cm)
Closing the gripper
Going back to 2.
The pick will happen in the selected workspace (default_workspace in the example)
height offset is the height at which the end effector will come
to take the object, based on the workspace height.
Examples for height offset:
With the Vacuum Pump equipped, you can use the height of the object.
With the Standard Gripper equipped, it can be half the height of the object to take it from its middle.
This block acts as a condition and so, returns a Boolean.
Vision Pick with Observation Pose Block¶
Equivalent to a Move Pose to the Observation Pose followed by a vision pick.
Is detected block¶
Represents whether an object is matching Color Block and Shape Block is present in the workspace.
This block returns a Boolean.
Please find below some use cases in Blockly. For each example, the sequence will be detailed step by step and you will have to set an observation position.
In order to realise these examples, you need to have your robot calibrated, and its tool plugged & scanned.
Moreover, you will need to register your workspace. In the following examples,
the name used is
default_workspace but feel free to chose yours!
Simple Vision pick and place¶
Ned has to pick an object from its workspace using vision. Then, it should place it anywhere.
Solution & explanation¶
To pick an object using Vision, you can use the Vision Pick Block.
It will return
True if an object is detected and picked, so it has to be
put in a logic condition ⇨ if the action happens,
we can place the picked object.
You can add an action in case the picking did not happen with
In this case, the robot will turn on the learning mode.
We need to see all 4 landmarks when the Vision Pick is called. In order to do that, we have to move over the workspace. We can do that by using a Move Pose block, but for the sake of this User Guide, we use the Vision Pick with Observation Pose Block
So, a complete pick and place process could be resumed in this easy sequence, which defines an observation position and picks an object if there is a matching object in the workspace to place it elsewhere, or activates learning mode if no object matches the chosen attributes.
Empty the Workspace with Vision pick and place¶
Ned has to pick any object in the workspace then place them all on the same spot.
The workspace can have any number of object without fixed color/shape.
Solution & explanation 1¶
To make the Vision pick task, you can use the Vision Pick with Observation Pose Block
which will return
True each time the action happens.
By using a loop as a condition (here,
repeat while), the
robot will continue the process until it picked all the objects of the workspace.
Place from Pose Block is added in order to place each picked object.
In this sequence, Ned will pick all the objects it can detect on the workspace then activate its learning mode.
Multi-reference packaging with Vision pick & place¶
Ned has to take all the objects from the workspace and place them at different positions depending on their attributes ⇨ the red objects will be placed in an area and the other ones in a different area.
Solution & explanation 2¶
You will start the process by using the Vision Pick with Observation Pose Block to interact with the red objects only. If the action happens, the object will be placed on a certain pose.
else if condition is added. That will be triggered if the first vision pick
did not happen, which means that there are no red objects anymore.
With this program, Ned will try to pick a red object, then place it in an area. If there is no red object in the workspace, Ned will take another object to place it in another area.
Now, we can use a loop to repeat this sequence for a given amount of objects using variables.
Vision pick with the Conveyor Belt¶
Ned has to take a number of objects from a workspace position on a Conveyor Belt, then place it in another area.
Solution & explanation 3¶
With Blockly, Ned is not able to pick a moving object, but it can control the Conveyor Belt! So, we can do:
Move Ned to an observation position above the Conveyor Belt with
Turn on the Conveyor Belt with
Wait to see an object entering the workspace with
Repeat until object detected
Stop the Conveyor Belt with
Pick the object and place it in the position of your choice with Vision Pick Block