Using Pixy2 and Arduino for Vision Processing
Pixy is a small, fast camera capable of detecting objects that you can teach, simply through pressing a button. For this blog, we will be talking about Pixy2. Pixy2 can perform the same functions as its predecessor, but instead has new algorithms that detect and track lines for use with line-following, all at 60 fps.
How to use Pixy2
In order to operate the PIxy2 camera, you will need the following:
The Pixycam itself
A micro USB-to-USB cable
Pixymon (Pixymon v2 Windows version 3.0.24 (exe))
Light ring
The first step is to get the camera to locate the object. The first step you would need to do is to open Pixymon and connect the camera to your laptop.
PixyMon will connect to your Pixy2 and start displaying color connected components data. This is the default data that are streamed from Pixy2, consisting of a list of detected objects, their locations, and sizes, overlaid over raw video.
Getting the camera to track the object requires setting a signature. To that, hold the object in front of the camera and on Pixymon go to Action -> Set Signature “...”. The video will freeze and allow you to click and drag the selected region you want Pixy2 to use to learn the object.
Booyah! Your Pixy2 can track the object.
In the case that you want to target retro-reflective tape, just set the camera brightness low and put the camera around the center of the light ring. It will give you the ability to track the glowing tape.
Note: Although it’s great, it ain’t good for targeting at a long distance. It’s best that you use it to target an object on the FRC field or to target retro-reflective tape at close range. If you want long-range targeting, use Limelight 2+. For more, check my other blog:
https://www.team3749.org/post/using-pixy2-and-arduino-for-vision-processing
Using Pixy2 with an Arduino
Ayyyyyy so now you got the Pixycam to read a target. However, the real question is how would you get data from the target? That’s where the Arduino comes in. Any programmer should know what it is. But in case you don’t know, an Arduino is an open-source electronics platform based on easy-to-use hardware and software. Arduino boards are able to read inputs and turn it into an output. You can tell your board what to do by sending a set of instructions to the microcontroller on the board.
The Pixy2 has already been made with microcontrollers in mind. All you need is:
An Arduino board (preferably an Uno board)
Arduino software ( https://www.arduino.cc/en/Main/Software )
USB-to-USB-B cable
The first thing you should do is connect the Pixycam to the board using the specialized Arduino cable given to you in the Pixy2 box.
Next, download the latest Arduino library “arduino_pixy2-x.y.z.zip” here. Bring up the Arduino IDE and import the Pixy library by selecting Sketch -> Include Library -> Add .ZIP Library… (or if you're using an older version Sketch -> Import Library) in the Arduino IDE, and then browsing to the Arduino zip file that you just downloaded.
Next, load the “ccc_hello_world” example by selecting it in File -> Examples -> Pixy2.
Bring up the Serial Monitor which can be found in Tools->Serial Monitor. Also be sure to set the Serial Monitor baud rate to 115200 baud, which is selectable at the bottom of the Serial Monitor window.
You will get this data:
More information about the Arduino code for Pixy2 will be found in this API
Here is some example code that we’ve used to target the power cells for aligning the robot during the 2020 FRC Game Season: Infinite Recharge: https://github.com/Team-Optix-3749/Team3749-2020/blob/master/Team3749-2020/ArduinoVisionCode.txt
Note: this example will only print messages if Pixy is running the “color_connected_components” program and an object that matches one of its color signatures is visible. This is what PixyMon looks like when Pixy is running the color_connected_components program and it has detected objects:
Getting Data from Arduino to RoboRIO
Now that the Arduino can create data from the Pixycam, now it’s time to deliver that data to the RoboRIO.
Programming is simple. All you need is the SerialPort class from the WPI library and your epic parsing skills.
I’m not gonna go detailed as to what you should do, but here is some example code from the 2020 FRC Game Season: Infinite Recharge: https://github.com/Team-Optix-3749/Team3749-2020/blob/master/Team3749-2020/src/main/java/frc/robot/subsystems/Vision.java
There you go! Just connect the Arduino Board to the Roborio using the USB-to-USB-B cable and you are all set.