Use the Vision Set to play Tic-Tac-Toe with Ned


If you are using a Niryo One, please refer to this tutorial.

Time has come to fight a worthy opponent, Ned, at Tic-Tac-Toe!


This application uses our PyNiryo API, Ned as well as our Vision Set. The image processing allows the robot to detect every object on the workspace, used as the game board, and the AI makes Ned a fearsome opponent, depending on the level of difficulty chosen at the beginning of every game.

Click here to download the Github repository to play Tic-Tac-Toe with Ned.


  • Ned

  • The Vacuum Pump end effector

  • 5 blue pawns and 5 green pawns

  • The Vision Set as well as the workspace

  • Optional: The slope from the Conveyor Belt

You can also customize your own workspace.


Setting up Ned

Overview Ned, its workspace and pawns

There are some variables you need to change in the and tic_tac_toe_config.yaml files from the github repository:


  • The IP address of your robot

In tic_tac_toe_config.yaml:

  • The workspace’s name: start by calibrating the workspace with Niryo Studio. The workspace’s name must be the same one as the one specified in the file (named by default default_ws).
  • The observation pose: if your workspace is not in front of the robot, you will have to modify the “observation_pose” variable so as to make the robot detect the four landmarks on the workspace.


In order to set the observation pose, activate the learning mode, place the robot above the workspace so it can detect the 4 landmarks, in the Pose panel of Niryo Studio, click Set to current. Now, the observation pose is stored and you just have to copy the values in the observation_pose variable.

It is very important to firmly attach your robot, the slope and the workspace.

Setting up the slope

In case you use it, it is necessary to locate the slope before the game starts. In order to do it, first launch the program with:

  • Place the robot’s suction cup on the pawn at the bottom of the slope.

  • Press enter.

You can also use the –reset argument in order to modify the slope’s position:

python3 --reset

or use the menu by using the –menu argument.


If the slope’s position has changed since your last game, it will be necessary to submit its new position to the robot.

How it works:

To implement Tic-Tac-Toe with Ned, we are using the Vision Set to picture the workspace. Then, the pawns’ positions in the workspace are extracted thanks to image processing functions. Finally, objects are placed in a 3×3 matrix used as a game board.

With the algorithm and according to the chosen difficulty, the robot will calculate its next move. A command will then be sent to the robot so as to make it place the pawn in the right position.

This application is composed of several python files that implement different classes:

  • displays logs at the beginning, manages options and the slope position, and run the main loop which uses the other files.
  • implements the TicTacToe class, which provides all the calculation methods for the Tic-Tac-Toe game, and manages the playing grid.
  • implements the RobotPlayer class, which provides all actions methods concerning Ned itself, such as “pick_place_on_grid”.
  • implements the ObjectDetection class, which manages both the display part and the objects detection part.


To launch the program, use:


This will start the Tic-Tac-Toe application with default settings, meaning:

  • 1 game only

  • You notify Ned you played your turn by moving it (you can move any joint you want)

  • The AI difficulty is set to 5 out of 1000 (further explainations in the List Of Commands section)

  • You are using the slope

Change settings

In order to change those settings, either you can:

  • directly change their value in the file, in the param dictionnary under the line:
    if __name__ == '__main__':
  • or you can launch the program using the –menu argument with:
    python3 --menu

    That will allow you to choose a bunch of settings when launching the program. These settings will offer you the possibility to change the way the game is played by the robot:

    Launch program with --menu option

Other features

Our AI also has the following features:

  • The program is able to see if the Human player is cheating. In this case, it will launch an animation and will then play its turn.
  • The difficulty can be adapted, from a never losing AI to an AI playing in a random way (cf. the list of commands).
  • If the –noslope option is chosen, the robot will automatically clean the workspace before each game or else the robot will put the pawns out of the workspace.
  • Each time the robot analyzes the workspace, a window opens and shows the image processing that the robot uses to locate the pawns.

List of commands:


You can only use one command when launching the program.


python3 --105

Number is a integer between 0 and 1000. This command allows you to adjust the AI’s random side. The AI is unbeatable if you set the difficulty to 0, and it plays in an absolutely random way if the difficulty is set to 1000.

{–loop | -l}

python3 --loop
python3 --l

This command triggers the relaunch of any ended game.

{–button | -b}

python3 --button
python3 --b

This command allows you to tell the robot your turn is done by pressing a button plugged in Ned’s GPIO port.

{–red | -r}

python3 --red
python3 --r

This command allows you to tell the robot your turn is done by moving a red pawn in the workspace. You can place it under the 3x3 grid board. Then, Ned will place this pawn on the first landmark, waiting for its turn.

{–noslope | -s}

python3 --noslope
python3 --s

This command gives you the ability to use the workspace instead of the slope to store the robot’s pawns. Before using this command, you must place the robot’s green pawns on the workspace at the beginning of the game (you can also add them in the storing slope during the game). When this setting is on, the robot will automatically tidy the workspace at the beginning of each game.


This command only works with round pawns since the square ones are too big to be placed on the workspace at the beginning.


python3 --reset

Reset the position of the slope, that is stored in a file tic_tac_toe_config.yaml.

Launching a game:

Launching a game with standard settings

  • If you use the slope:

    Start by putting five green pawns on Ned’s slope, then launch:



    When you play for the first time, or when the slope has been moved, Ned will have to know its new position. Check the Setting-up the slope part of this application to do that.

  • If you don’t use the slope:

    Start by putting five round green pawns on the side of the workspace dedicated to the pawns, then launch:

    python3 --noslope

Then a graphic interface will appear. It displays:

  • What Ned saw the last time it watched the workspace from above

  • Several grids gathering information about blue and green pawns positions

Moreover, a grid will also be displayed in the program terminal, to let you know where Ned saw each pawn, like:







The left part is where Ned’s pawns are stored, and the 3x3 grid on top right corner is the Tic-Tac-Toe board.

If you start the game

  • Place a pawn on the workspace, wherever you want

  • Once the pawn is placed, indicate Ned your turn is over, according to the chosen option for the turn_end argument. Ned will scan the workspace and play its turn.

  • Repeat the operation until the end of the game!

When the game is over, Ned will go back to its standby position.

If Ned starts the game

First, you have to indicate Ned it should start, according to the chosen option for the turn_end argument. It will scan the workspace and place its first pawn. Once your pawn is on the workspace, move the robot’s arm again and so on, until the end of the game.