Teach Pixy an Object 2

Version 3 (Rich LeGrand, 01/15/2015 04:06 pm)

1 1 Rich LeGrand
h1. Teach Pixy an Object
2 1 Rich LeGrand
3 1 Rich LeGrand
Teaching Pixy an object is super easy, but first let's talk about which objects will work well with Pixy.  Pixy uses a hue-based color filtering algorithm to detect objects.  Since Pixy uses hue (color), the object needs to have a distinct hue.  Here are some objects that are good because they have good, distinct hues.
4 1 Rich LeGrand
5 1 Rich LeGrand
!{width: 10%}http://i74.photobucket.com/albums/i241/cmucam/tennisball_zps93bb0163.jpg!
6 1 Rich LeGrand
!{width: 10%}http://i74.photobucket.com/albums/i241/cmucam/pinkpenguin_zps99f4cb1e.jpg!
7 1 Rich LeGrand
8 1 Rich LeGrand
Here are some bad objects because either there is no hue (black, white or gray) or the hue is not distinct.  
9 1 Rich LeGrand
10 1 Rich LeGrand
!{width: 10%}http://i74.photobucket.com/albums/i241/cmucam/Black_Stapler_zpsc3024407.jpg!
11 1 Rich LeGrand
!{width: 10%}http://i74.photobucket.com/albums/i241/cmucam/stone_zpsd3435436.png!
12 1 Rich LeGrand
13 1 Rich LeGrand
Keeping these guidelines in mind, choose an object to teach Pixy.  (Apply power to Pixy via battery or USB cable if you haven't already.  When you power up Pixy, it will go through a series of LED flashes.  Wait for the led to turn off before teaching Pixy.)  Hold down the button on top of Pixy.  After about 1 second, the LED will turn on, first white, then red, then other colors, but when it turns red, release the button.  
14 1 Rich LeGrand
15 2 Rich LeGrand
When you release the button, Pixy will enter what's called "light pipe" mode, where the LED color is the color of the object that Pixy has "locked" onto.  Pixy will lock onto objects in the center of its video frame, so hold the object directly in front of pixy between 6 and 20 inches from the lens.  Pixy uses a region growing algorithm to try to determine which pixels are part of your object and which pixels are part of the background.  Pixy uses these Pixels to create a statistical model of your object so it can detect it reliably under different lighting conditions.  So use the LED color as feedback to determine if Pixy has a good lock on the object, and use the following guidelines to judge:
16 1 Rich LeGrand
17 2 Rich LeGrand
# When Pixy has locked onto your object, the LED color should match the color of your object.  
18 2 Rich LeGrand
# The brighter the LED, the better the lock.  Move the object a little and see if you can maximize the LED brightness. 
19 1 Rich LeGrand
20 2 Rich LeGrand
When you are satisfied, press and release the button, like you'd click your mouse.  The LED will flash a couple times indicating that Pixy has now "learned" your object.  It will now start tracking your object (or objects). 
21 1 Rich LeGrand
22 2 Rich LeGrand
The video below is a good short "how-to" regarding teaching Pixy objects.  
23 1 Rich LeGrand
24 3 Rich LeGrand
25 2 Rich LeGrand
26 2 Rich LeGrand
h3. Multiple Signatures
27 3 Rich LeGrand
28 2 Rich LeGrand
Pixy can learn up to seven color signatures. We can teach Pixy the 1st color signature by releasing the button when the LED turned red. If we had continued to hold down the button, the led would have turned, orange, yellow, etc., indicating the remaining color signatures. Here are the signatures in order:
29 2 Rich LeGrand
30 1 Rich LeGrand
# Red
31 1 Rich LeGrand
# Orange
32 1 Rich LeGrand
# Yellow
33 1 Rich LeGrand
# Green
34 1 Rich LeGrand
# Cyan (light blue)
35 1 Rich LeGrand
# Blue
36 1 Rich LeGrand
# Violet
37 1 Rich LeGrand
38 2 Rich LeGrand
The color signature number is determined by _when_ you release the button.  Release the button when the LED is yellow and you're teaching signature 3.  Release the button when the LED is blue and you're teaching signature 6.  These colors _are not related to the hue of the object_.  The colors are used only to indicate the signature number. So, for example, signature 1 can be a yellow object, even though signature 1 is indicated by a red LED, and signature 2 could be a pink object even though signature 2 is indicated by an orange LED.
39 2 Rich LeGrand
40 1 Rich LeGrand
h3. Other Notes on Teaching
41 1 Rich LeGrand
42 2 Rich LeGrand
After you teach Pixy a signature, it saves the signature in flash, so when you power cycle your Pixy, it will remember the signatures you taught it and continue to track objects that match these signatures.    
43 2 Rich LeGrand
44 2 Rich LeGrand
If you accidentally find yourself teaching signature 2 when you meant to teach signature 1 (i.e. you released the button when it was orange instead of read), for example, just hold down the button until the LED turns off.  This is how you tell Pixy to cancel teach mode.  You can then start over by holding the button down again. 
45 1 Rich LeGrand
46 1 Rich LeGrand
h3. White Balance
47 1 Rich LeGrand
48 1 Rich LeGrand
Some types of lighting (such as incandescent) have a reddish hue and others (such as fluorescent) have a bluish hue.  The lighting can affect your color signatures.  For example, if you teach an object under incandescent lighting and move into a room with fluorescent lighting, the color signatures will likely no longer work as well.  You can either re-teach all signatures or you can adjust the white balance.
49 1 Rich LeGrand
50 2 Rich LeGrand
When you first apply power to Pixy, it will spend the first 5 or so seconds determining the correct white balance to use.  It will then disable automatic white balance.  If you wish to readjust the white balance, hold down the button until the LED turns white and release.  It happens quickly, so be prepared!  Pixy is now in automatic white balance mode.  You can hold a white sheet of paper in front of Pixy so Pixy can adjust the white balance.  It only takes 2 or 3 seconds to adjust the white balance, after which you can press and release the button (like a mouse click). The LED will flash, indicating success, and now Pixy is "rebalanced".  
51 1 Rich LeGrand
52 2 Rich LeGrand
h3. Teach through PixyMon
53 1 Rich LeGrand
54 2 Rich LeGrand
You can also teach Pixy an object through PixyMon.  This may be useful if the object you want to teach is small or if you want more control over which pixels are used for teaching. Begin by plugging in the USB cable between Pixy and your computer and running PixyMon.  
55 1 Rich LeGrand
56 1 Rich LeGrand
Now hold the object you want to teach in front of Pixy and select *Action➜Set signature 1* from the pulldown menu.
57 1 Rich LeGrand
58 1 Rich LeGrand
!{width: 50%}http://i74.photobucket.com/albums/i241/cmucam/Image127_zps75a6cdef.jpg!
59 1 Rich LeGrand
60 1 Rich LeGrand
Now select you region you want Pixy to use to learn the object.  
61 1 Rich LeGrand
62 1 Rich LeGrand
!{width: 50%}http://i74.photobucket.com/albums/i241/cmucam/Image128_zpsff146617.jpg!
63 1 Rich LeGrand
64 2 Rich LeGrand
That's it! After you select the region, Pixy will "learn" the object and automatically go into "cooked" video mode so can verify how well your color signature is working.
65 1 Rich LeGrand
66 1 Rich LeGrand
!{width: 50%}http://i74.photobucket.com/albums/i241/cmucam/Image131_zpsf63a2ef0.jpg!
67 1 Rich LeGrand
68 2 Rich LeGrand
h3. *Signature tuning*
69 2 Rich LeGrand
70 2 Rich LeGrand
71 1 Rich LeGrand
72 1 Rich LeGrand
You can teach Pixy other objects in this way, up to seven total color signatures are possible.  Note that Pixy always saves color signatures in flash, so they will be available the next time you apply power to Pixy. 
73 1 Rich LeGrand
74 1 Rich LeGrand
If you haven't already, from here you might be interested in [[Hooking up Pixy to a Microcontroller like an Arduino|hooking Pixy up to a microcontroller]].  
75 1 Rich LeGrand
76 1 Rich LeGrand
77 1 Rich LeGrand
78 1 Rich LeGrand
79 1 Rich LeGrand
80 1 Rich LeGrand
81 1 Rich LeGrand