Simple CMUcam4 Program

Added by Kwabena Agyeman almost 8 years ago

BRYANTAYJH.09 said:

Hi, this is my first experience on the arduino and cmucam4 and i am still learning the basics of programming.
I am trying to program the cmucam4 to track a black object 1-3metres away from the camera and also be able to locate whether the object is at the left, right or center of the camera. There would be 3 LEDs attached to the arduino digital outputs and if the camera detects a black object on the left, the left LED would light up. The same thing goes for the center and right LEDs. I referred to the examples but have still no idea on making the values to detect black and also allowing it to locate the object and light up the leds on the arduino. Could you provide a basic program that would basically do the above for reference? Your help is greatly appreciated.

I said:

Here's a simple program that does what you asked. I haven't tested it myself yet, but I think it should work. I just built it off of the color tracking template code.

/***************************************************************************//**
* @file
* Example Code
*
* @version @n 1.0
* @date @n 9/19/2012
*
* @authors @n Kwabena W. Agyeman
* @copyright @n (c) 2012 Kwabena W. Agyeman
* @n All rights reserved - Please see the end of the file for the terms of use
*
* @par Update History:
* @n v1.0 - Initial Release - 9/19/2012
*******************************************************************************/

#include <CMUcam4.h>
#include <CMUcom4.h>

#define YUV_MODE false // Change this to true to switch to YUV mode.

// More about the YUV color space http://en.wikipedia.org/wiki/YUV

// Also read http://cmucam.org/projects/cmucam4/wiki/Tips_and_Tricks

// Feed the color to track into these six variables.

#define RED_MIN 230
#define RED_MAX 255
#define GREEN_MIN 230
#define GREEN_MAX 255
#define BLUE_MIN 230
#define BLUE_MAX 255

// The above tracking values currently track bright lights.

#define LED_BLINK 5 // 5 Hz
#define WAIT_TIME 5000 // 5 seconds

#define LEFT_LED_PIN 6
#define MIDDLE_LED_PIN 5
#define RIGHT_LED_PIN 3

#define PIXELS_THRESHOLD 0 // The percent of tracked pixels needs to be greater than this 0=0% - 255=100%.
#define CONFIDENCE_THRESHOLD 50 // The percent of tracked pixels in the bounding box needs to be greater than this 0=0% - 255=100%.

#define NOISE_FILTER_LEVEL 2 // Filter out runs of tracked pixels smaller than this in length 0 - 255.

CMUcam4 cam(CMUCOM4_SERIAL);

void setup()
{
  cam.begin();

  // Wait for auto gain and auto white balance to run.

  cam.LEDOn(LED_BLINK);
  delay(WAIT_TIME);

  // Turn auto gain and auto white balance off.

  cam.autoGainControl(false);
  cam.autoWhiteBalance(false);

  cam.LEDOn(CMUCAM4_LED_ON);

  cam.colorTracking(YUV_MODE);

  pinMode(LEFT_LED_PIN, OUTPUT);
  pinMode(MIDDLE_LED_PIN, OUTPUT);
  pinMode(RIGHT_LED_PIN, OUTPUT);

  cam.noiseFilter(NOISE_FILTER_LEVEL);
}

void loop()
{
  CMUcam4_tracking_data_t data;

  // Start streaming...

  cam.trackColor(RED_MIN, RED_MAX, GREEN_MIN, GREEN_MAX, BLUE_MIN, BLUE_MAX);

  for(;;)
  {
    cam.getTypeTDataPacket(&data); // Get a tracking packet at 30 FPS or every 33.33 ms.

    // Process the packet data safely here.

    // More info http://cmucam.org/docs/cmucam4/arduino_api/struct_c_m_ucam4__tracking__data__t.html

    if(data.pixels > PIXELS_THRESHOLD) // We see the color to track.
    {
      if(data.confidence > CONFIDENCE_THRESHOLD) // The pixels we are tracking are in a dense clump - so maybe they are a single object.
      {
        // Okay, so we now know we are looking at a clump of pixels that are in the color bounds we wanted to track.

        // Linearly makes the LED get brighter as the object moves towards the left.
        analogWrite(LEFT_LED_PIN,  255 - map(CMUCAM4_MIN_NATIVE_COLUMN + data.mx, 0, CMUCAM4_MAX_NATIVE_COLUMN, 0, 255));

        // Linearly makes the LED get brighter as the object moves towards the middle.
        analogWrite(MIDDLE_LED_PIN, 255 - map(abs((CMUCAM4_NATIVE_H_RES/2) - data.mx), 0, (CMUCAM4_NATIVE_H_RES/2), 0, 255));

        // Linearly makes the LED get brighter as the object moves towards the right.
        analogWrite(RIGHT_LED_PIN, 255 - map(CMUCAM4_MAX_NATIVE_COLUMN - data.mx, 0, CMUCAM4_MAX_NATIVE_COLUMN, 0, 255));

        // Might want to make the above functions non-linear as the brightness function of an LED is a non-linear... the effect will look better.
      }
      else
      {
        // Turn the LED's off.

        analogWrite(LEFT_LED_PIN, 0);
        analogWrite(MIDDLE_LED_PIN, 0);
        analogWrite(RIGHT_LED_PIN, 0);
      }
    }
    else
    {
      // Turn the LED's off.

      analogWrite(LEFT_LED_PIN, 0);
      analogWrite(MIDDLE_LED_PIN, 0);
      analogWrite(RIGHT_LED_PIN, 0);
    }
  }

  // Do something else here.
}

/***************************************************************************//**
* @file
* @par MIT License - TERMS OF USE:
* @n Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"), to
* deal in the Software without restriction, including without limitation the
* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
* sell copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
* @n
* @n The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
* @n
* @n THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*******************************************************************************/

So, to figure out the color of the object you want to track you're going to need to do this:


The CMUcam4 features a frame dump command called "DF" (Dump Frame). The "DF" command saves a picture of what the CMUcam4 sees to the microSD card. By using the "DF" command you can take pictures of all the colored objects you want to track with the CMUcam4. In general, you'll want to use the "DF" command to take 160x120 pictures because 160x120 is the same resolution the CMUcam4 uses to track colors. Once you have all the pictures of the colored objects you want to track you then need to use the "UM" (Unmount Disk) command to unmount the microSD card from the CMUcam4 before removing the microSD card from the CMUcam4. Otherwise, when you plug your microSD card into your computer your operating system will complain about the microSD card not being removed properly.

Anyway, once you connect your microSD card to your PC you can then look at the CMUcam4 frame dumps. The CMUcam4 saves pictures in the BMP file format so that any program will be able to open them. However, first you'll need to rotate the image by 90 degrees before being able to view it properly. After rotating the image you can then examine the pixel color values for the object you want to track using the eye dropper tool like in Microsoft Paint.

You'll want to take a few samples of the red, green, and blue color channels of the object you want to track. Try to figure out the area of color that the object spans. Eventually, you should end up with a minimum and maximum value for the red, green, and blue color channels. You'll then want to feed these minimum and maximum values back to the CMUcam4 to track the colored object. You'll probably want to increase the range of the min and max values you find to compensate for changes in lighting. Otherwise, if you have the CMUcam4 hooked up to a TV, you'll be able notice the object moving in and out of being tracked as the lighting changes on the object.

If you have a program that can display the histogram of the image like Photoshop, then you'll want to crop the image around the color of the object you want to track. By using the image histogram feature of the program you'll be able to see the distribution of the pixels in the red, blue, and green color channels. This makes figuring out what color of the object you want to track much easier. You'll also want to increase the range of colors you are tracking to compensate for changes in lighting like in the above paragraph and similarly feed the minimum and maximum red, green, and blue values of the color you want to track back to the CMUcam4 so that it tracks the object.

This calibration process requires a bit of iteration, but, the CMUcam4 is just like any other sensor, you can't expect it to magically work without effort. However, the CMUcam4's video output feature lets you see what the CMUcam4 sees in real time so that you can more quickly finish the calibration process and verify that everything is working for you application.


Figuring out the color bounds is the hardest part. Hook up a TV to the CMUcam4 so you can see what color it is tracking. Once you get the color down then you'll be good to go. It takes some work to figure out what the color bounds need to be. Also, try out YUV mode.

If you don't have a serial programmer to talk to the CMUcam4 over the serial port using a terminal client program then you can use the example "paparazzi" programs to save picture of what the CMUcam4 is looking at to the SD card. Don't move the object your tacking a picture of. The DF feature builds a picture of the object you are looking at out of multiple frames from the camera.

Good luck,


Replies (16)

RE: Simple CMUcam4 Program - Added by bryan tay almost 8 years ago

Thanks for the quick reply. This has really help me alot in understanding more on the cmucam4 as well as the arduino. I have managed to get the required color parameters for me to track the object. However i do not fully understand how to change led brightness from linear to non linear. Is it to use for( i =0; i>=1; i++) <== just an example. Or could you give me an example?
Another question i want to ask is whether replacing the leds with thrusters and making a robot that can track black lines follows the same concept as the example program you had given me or is there any more neccessary information i should know before starting this project?

RE: Simple CMUcam4 Program - Added by Kwabena Agyeman almost 8 years ago

Hi Bryan,

You can make the LED brightness non-linear by using the C sin() and cos() functions.

For example, you can make the LED brightness non-linear by using the following code:

        // Makes the LED get brighter as the object moves towards the left.
        int left_temp = map(CMUCAM4_MIN_NATIVE_COLUMN + data.mx, 0, CMUCAM4_MAX_NATIVE_COLUMN, 0, 90); // Convert to degrees...        
        int left_cos = ((int) (cos( /* convert to radians */ ((((float) left_temp) * PI) / 180.0) ) * 255)); // Scale 255 by cos... cos will go from 1 to 0... 
        analogWrite(LEFT_LED_PIN, left_cos);  

        // Makes the LED get brighter as the object moves towards the middle.
        analogWrite(MIDDLE_LED_PIN, 255 - map(abs((CMUCAM4_NATIVE_H_RES/2) - data.mx), 0, (CMUCAM4_NATIVE_H_RES/2), 0, 255));

        int middle_temp = map(data.mx, 0, CMUCAM4_MAX_NATIVE_COLUMN, -45, 45) * 2; // Convert to degrees...        
        int middle_cos = ((int) (cos( /* convert to radians */ ((((float) middle_temp) * PI) / 180.0) ) * 255)); // Scale 255 by cos... cos will go from 1 to 0... 
        analogWrite(MIDDLE_LED_PIN, middle_cos); 

        // Makes the LED get brighter as the object moves towards the right.
        int right_temp = map(CMUCAM4_MAX_NATIVE_COLUMN - data.mx, 0, CMUCAM4_MAX_NATIVE_COLUMN, 0, 90); // Convert to degrees...        
        int right_cos = ((int) (cos( /* convert to radians */ ((((float) right_temp) * PI) / 180.0) ) * 255)); // Scale 255 by cos... cos will go from 1 to 0... 
        analogWrite(RIGHT_LED_PIN, right_cos); 

You can play with the above code to have a more or less narrow cutoff between LEDs. If you don't understand what's going on look at the cos() function from -90 to 90 degrees (-pi/2 to pi/2). Then imagine 3 copies of the functions existing on a line where one cos() is on the left edge, another on the right edge, and one cos() is in the middle.

Now, you can use the above code to do simplistic line following. If you have an Arduino Mega then you can do more complicated and robust line following using the CMUcam4 line mode feature. The line mode feature for the CMUcam4 causes the camera to send 80x60 binary images back to the Arduino at 30 FPS for post processing of all the tracked pixels in the image.

Using line mode, you can then take horizontal slices out of the 80x60 image. You would then compute the center of mass of the "1" pixels in the slice of the image. This is simply the sum of the all the "1" pixel position in the slice you look at - very easy. By taking let's say, 10 horizontal slices from the 80x60 binary image you would then know the center of mass of 10 points in the binary image.

If you imagine that the line we want to follow is perfectly displayed vertically across the 80x60 image, then the 10 center of mass points we just found correspond to 10 points on the line. Once you know this, all you have to do is tell the robot to try to make the 10 points it just found line up with the center of the image with it's motors. This effectively will cause the robot to want to lock on to the line and follow it.

Either using line mode or just the centroid will work to follow a line. Having both methods at the same time, however, will be the best. Non-Arduino Mega's can't handle the 80x60 (600 bytes) binary images the CMUcam4 sends because they cause the non-Mega Arduinos to run out of RAM.

Thanks,

RE: Simple CMUcam4 Program - Added by Edwin Tan Yong Zhi almost 8 years ago

Hi,

Is is possible to replaces those LEDs with a esc to control brushless motor? I am making a robot to follow the line, if the robot is to the left the right motor will have higher speed to make it align to the line.

Regards
Edwin

RE: Simple CMUcam4 Program - Added by Kwabena Agyeman almost 8 years ago

I mean, I suppose you can replace the LED's with an ESC.

If you understand what the code if doing you can put whatever you want there.

I wouldn't just copy the code verbatim and hope that it works. You'll have to tailor what I did above to your application.

Thanks,

RE: Simple CMUcam4 Program - Added by Andy He over 7 years ago

Dear Kwabena,

I noticed that in your code above and in the DemoObjectTrackingTemplate, there are lines like "(CMUCAM4_NATIVE_H_RES/2) - data.mx" and "CMUCAM4_MIN_NATIVE_COLUMN + data.mx". I am also doing a similar project and I was wondering what these two lines means. Thanks!

RE: Simple CMUcam4 Program - Added by Andy He over 7 years ago

Oh I know what they mean now, thx!

RE: Simple CMUcam4 Program - Added by Kwabena Agyeman over 7 years ago

They are just defines... if you find the source where they come from it says what they are.

Thanks,

RE: Simple CMUcam4 Program - Added by Sam Jackson over 6 years ago

Hi Kwabena, I am using this program as a test for a tracking system for a project, when I use it all the LED's turn on when the target is infront of it, and then it can't detect at the sides, using an arduino uno so I can't debug easily, I know its recommended to use a mega but only have the uno for this project. Would you be able to give me a bit of help? Thanks.

RE: Simple CMUcam4 Program - Added by Kwabena Agyeman over 6 years ago

Hi Sam,

Please explain what your problem is.

Thanks,

RE: Simple CMUcam4 Program - Added by Sam Jackson over 6 years ago

Don't worry anymore, after a bit of playing about I've realised what my issue was.

RE: Simple CMUcam4 Program - Added by Fonfon Bzoubida over 6 years ago

Hi Kwabena,
I just received my CMUcam4, so I wanted to try it with your program (above)
But when I verified it, I had this error message :
------------------------------------------------------------------------------------------------------------
P:\Mes documents\Arduino\libraries\CMUcam4\CMUcom4.cpp: In member function 'size_t CMUcom4::write(const char*)':
P:\Mes documents\Arduino\libraries\CMUcam4\CMUcom4.cpp:111: error: void value not ignored as it ought to be
P:\Mes documents\Arduino\libraries\CMUcam4\CMUcom4.cpp: In member function 'size_t CMUcom4::write(const uint8_t*, size_t)':
P:\Mes documents\Arduino\libraries\CMUcam4\CMUcom4.cpp:126: error: void value not ignored as it ought to be
-------------------------------------------------------------------------------------------------------------
As if my libraries have mistakes, but I downloaded it on the official website
Can you help me ?
Thanks

RE: Simple CMUcam4 Program - Added by Kwabena Agyeman over 6 years ago

Do you have the newest library? Never seen that error before... maybe Arduino updated their code. If so, just return the amount written in those functions.

RE: Simple CMUcam4 Program - Added by Fonfon Bzoubida over 6 years ago

Finally my arduino version was too old, I changed it and it works
thanks !

(1-16/16)