********************************** Use the Vision Set with Blockly ********************************** `Blockly `_'s blocks have been designed to be used with the Vision Set. It keeps Ned's Ecosystem easy to use with Niryo Studio. In this section, we will see how to use the Vision Set with these blocks. Blocks Description ---------------------------- Color Block ^^^^^^^^^^^^^^^^^^ .. image:: ../images/blockly/color_block.png :alt: Color block :height: 25px Allows the user to choose a color among the list of available colors. This color will be used in the :ref:`source/how_to_use_blockly:Vision Pick Block` and the :ref:`source/how_to_use_blockly:Is Detected Block`. Shape Block ^^^^^^^^^^^^^^^^^^ .. image:: ../images/blockly/shape_block.png :alt: Shape block :height: 25px Allows the user to choose a shape among the list of available shapes. The selected shape will be used in the :ref:`source/how_to_use_blockly:Vision Pick Block` and the :ref:`source/how_to_use_blockly:Is Detected Block`. Vision Pick Block ^^^^^^^^^^^^^^^^^^^^^^^ .. image:: ../images/blockly/vision_pick_block.png :alt: Vision pick block :height: 125px This block will make Ned pick an object matching the :ref:`source/how_to_use_blockly:Color Block` and :ref:`source/how_to_use_blockly:Shape Block`. The picking process is: #. Opening the gripper #. Going over the object (5 cm) #. Going down #. Closing the gripper #. Going back to 2. The pick will happen in the selected workspace (default_workspace in the example) The ``height offset`` is the height at which the end effector will come to take the object, based on the workspace height. Examples for height offset: - With the Vacuum Pump equipped, you can use the height of the object. - With the Standard Gripper equipped, it can be half the height of the object to take it from its middle. This block acts as a condition and so, returns a Boolean. Vision Pick with Observation Pose Block ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. image:: ../images/blockly/vision_pick_w_obs_block_cut.png :alt: Vision pick block :height: 150px Equivalent to a Move Pose to the Observation Pose followed by a vision pick. Is detected block ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. image:: ../images/blockly/is_detected_block.png :alt: Is detected block :height: 100px Represents whether an object is matching :ref:`source/how_to_use_blockly:Color Block` and :ref:`source/how_to_use_blockly:Shape Block` is present in the workspace. This block returns a Boolean. Examples ---------------------- Please find below some use cases in Blockly. For each example, the sequence will be detailed step by step and you will have to set an observation position. .. important:: In order to realise these examples, you need to have your robot calibrated, and its tool plugged **& scanned**. Moreover, you will need to register your workspace. In the following examples, the name used is ``default_workspace`` but feel free to chose yours! Simple Vision pick and place ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Task """""""""""""""""""""" Ned has to pick an object from its workspace using vision. Then, it should place it anywhere. Solution & explanation """"""""""""""""""""""""""""""""""" To pick an object using Vision, you can use the :ref:`source/how_to_use_blockly:Vision Pick Block`. It will return ``True`` if an object is detected and picked, so it has to be put in a logic condition ⇨ if the action happens, we can place the picked object. .. image:: ../images/blockly/tuto_1_1.png :alt: Simple vision pick 1 :width: 750px :align: center You can add an action in case the picking did not happen with ``else``. In this case, the robot will turn on the learning mode. .. image:: ../images/blockly/tuto_1_2.png :alt: Simple vision pick 2 :width: 750px :align: center We need to see all 4 landmarks when the Vision Pick is called. In order to do that, we have to move over the workspace. We can do that by using a Move Pose block, but for the sake of this User Guide, we use the :ref:`source/how_to_use_blockly:Vision Pick with Observation Pose Block` .. image:: ../images/blockly/tuto_1_3.png :alt: Simple vision pick 3 :width: 750px :align: center So, a complete pick and place process could be resumed in this easy sequence, which defines an observation position and picks an object if there is a matching object in the workspace to place it elsewhere, or activates learning mode if no object matches the chosen attributes. Empty the Workspace with Vision pick and place ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Task 1 """""""""""""""""""""""""""""""""" Ned has to pick any object in the workspace then place them all on the same spot. The workspace can have any number of object without fixed color/shape. Solution & explanation 1 """""""""""""""""""""""""""""""""" To make the Vision pick task, you can use the :ref:`source/how_to_use_blockly:Vision Pick with Observation Pose Block` which will return ``True`` each time the action happens. By using a loop as a condition (here, ``repeat while``), the robot will continue the process until it picked all the objects of the workspace. .. image:: ../images/blockly/tuto_2_1.png :alt: Infinite vision pick 1 :width: 750px :align: center Then, a ``Place from Pose Block`` is added in order to place each picked object. .. image:: ../images/blockly/tuto_2_2.png :alt: Infinite vision pick 2 :width: 750px :align: center In this sequence, Ned will pick all the objects it can detect on the workspace then activate its learning mode. Multi-reference packaging with Vision pick & place ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Task 2 """""""""""""""""""""""""""""""""" Ned has to take all the objects from the workspace and place them at different positions depending on their attributes ⇨ the red objects will be placed in an area and the other ones in a different area. Solution & explanation 2 """""""""""""""""""""""""""""""""" You will start the process by using the :ref:`source/how_to_use_blockly:Vision Pick with Observation Pose Block` to interact with the red objects only. If the action happens, the object will be placed on a certain pose. .. image:: ../images/blockly/tuto_3_1.png :alt: Multi-reference vision pick 1 :width: 750px :align: center Then, a ``else if`` condition is added. That will be triggered if the first vision pick did not happen, which means that there are no red objects anymore. .. image:: ../images/blockly/tuto_3_2.png :alt: Multi-reference vision pick 2 :width: 750px :align: center With this program, Ned will try to pick a red object, then place it in an area. If there is no red object in the workspace, Ned will take another object to place it in another area. Now, we can use a loop to repeat this sequence for a given amount of objects using variables. .. image:: ../images/blockly/tuto_3_3.png :alt: Multi-reference vision pick 3 :width: 750px :align: center Vision pick with the Conveyor Belt ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Task 3 """""""""""""""""""""""""""""""""" Ned has to take a number of objects from a workspace position on a Conveyor Belt, then place it in another area. Solution & explanation 3 """""""""""""""""""""""""""""""""" With Blockly, Ned is not able to pick a moving object, but it can control the Conveyor Belt! So, we can do: #. Move Ned to an observation position above the Conveyor Belt with ``Move pose`` #. Turn on the Conveyor Belt with ``Control Conveyor`` #. Wait to see an object entering the workspace with ``Repeat until object detected`` #. Stop the Conveyor Belt with ``Stop Conveyor`` #. Pick the object and place it in the position of your choice with :ref:`source/how_to_use_blockly:Vision Pick Block` .. image:: ../images/blockly/tuto_4_1.png :alt: Vision pick with Conveyor :width: 750px :align: center