####################################################################
Leap Motion hand tracking controller
####################################################################

**V.1.1**

.. image:: /images/leap_motion/leapmotion_controller.png
                     :alt: Mouse control application
                     :width: 100%
                     :align: center

Difficulty: ``medium``

Time: ``~30 min``

.. note::
    This tutorial is working from:
        | The version v3.0.0 of the ned_ros_stack 
        | The version v3.0.0 of Niryo Studio 
        | The version v3.1.0 of PyNiryo

You can find the project in our `GitHub <https://github.com/NiryoRobotics/ned_applications/tree/master/examples/Leap_Motion>`__.

.. raw:: html

    <div class="youtube">
        <iframe width="560" height="315" src="https://www.youtube.com/embed/mQpFMWQy858" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
    </div>

Introduction
--------------------------

This documentation contains all the information on how to set up and use the Leap Motion and control Ned/Ned2 with it.
A Leap Motion is an optical hand tracking module that captures the movements of your hands with unparalleled accuracy.
With this application, you will be able to control Ned/Ned2 with your hand with a Leap Motion hand tracker.

Requirements
---------------------------------
To run this application you will need a Leap Motion controller from Ultraleap and a Ned/Ned2 (real or simulated).

You can buy and find all the information about the Leap Motion following this
link: `Leap Motion <https://www.ultraleap.com/product/leap-motion-controller/>`__.

How to set up the Leap Motion
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. note::
    For this application you don't have to set up the Leap Motion because it's already done in the package. You can still have a look at it if you want to know how to use the Leap Motion.

Before using the Leap Motion with Ned/Ned2, you will need to set up it.
You can first have a look at this `video <https://vimeo.com/71036624>`__ to learn how to set up your Leap Motion.

**Download the Leap Motion controller SDK**

First you will need to download the Leap Motion SDK on the repository you want.
You can download it `here <https://developer.leapmotion.com/sdk-leap-motion-controller/>`__.

Then you will have to install the package you just downloaded. To do that you will need to open a terminal and go on the repository where you downloaded the previous SDK.

.. code-block:: bash

 cd <your leap motion repository>
 sudo dpkg -i Leap-2.3.1+31549-x64.deb

**Run the Leap Motion**

Now you can plug the Leap Motion to your computer and run it:

.. code-block:: bash

 sudo leapd

If your Leap Motion is working and properly connected you should have this:

      .. image:: /images/leap_motion/capture_leap.png
                    :alt: Ned
                    :width: 500px
                    :align: center

If you want to, you can launch the Leap Motion Visualizer on a terminal to have a first look on how it is working:

.. code-block:: bash

 cd <your leap motion repository>
 Visualizer

You should see a 3D simulated environment and if you put your hands up to the Leap Motion you can see them.

      .. image:: /images/leap_motion/leap_main.png
                    :alt: Ned
                    :width: 500px
                    :align: center

Now that your Leap Motion is well working you can use it with Ned/Ned2.

How to launch the application
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. note::
    Before running the application you will need to launch the Leap Motion (see below how to do it).

**Link the Leap library to your code**

Before doing that, you need to link the Leap library to your code. To do so, open the robot_V3.py script.
You will need to change the path of the Leap library on the second line:

.. code-block:: python

 import os, sys, inspect, thread, time
 sys.path.insert(0, "YOUR_PATH/LeapDeveloperKit_2.3.1+31549_linux/LeapSDK/lib")
 import Leap

**Run the robot**

To launch this application you will need Ned/Ned2 (real or simulated).

.. note::
    We advise you to first run this application in simulation to get used to controlling the robot with the Leap Motion.

According to the choice you make you will need to change the IP address of the robot on the code robot_V3.py.

.. code-block:: python

 robot_ip = "127.0.0.1" #"192.168.1.15" => ip of the robot

.. danger::
    Before running the application, make sure the environment around the real robot is clear.


To launch the application you need to run the code/robot_V3.py script:

.. code-block:: python

 cd <your leap motion repository/code>
 python leap_motion_ned.py


After launching the application, you should see this menu:

      .. image:: /images/leap_motion/menu_leap.png
                    :alt: Ned
                    :width: 500px
                    :align: center

      .. image:: /images/leap_motion/menu2_leap.png
                    :alt: Ned
                    :width: 500px
                    :align: center

If you open your hand, the gripper should open and if you close your hand it should close.
If you move your hand, the robot will do the same movement. 

.. note::
    We advise you to move your hand slowly not to have unreachable or sudden movement.

Tips
---------------
To have a fluid movement, we advise you to use an Ethernet connexion with Ned/Ned2.