Friday, August 12, 2011

Episode_12_FPS / Frame per Second


Well, I do not want to talk much more about what it is FPS, you can find yourself in google. I just want to help you if your camera doesn't support with FPS.

Get Frame information in OpenCV:
  • int fps = (int) cvGetCaptureProperty(capture, CV_CAP_PROP_FPS);
  • float posMsec = cvGetCaptureProperty(capture, CV_CAP_PROP_POS_MSEC);
  • int posFrames = (int) cvGetCaptureProperty(capture, CV_CAP_PROP_POS_FRAMES);
  • float posRatio = cvGetCaptureProperty(capture, CV_CAP_PROP_POS_AVI_RATIO);
So I have a tips to solve for this solution.
#include "time.h"

int main (int argc, char* argv[])
{
 int fps,fpsfix,second;
 ...
 ...
 SYSTEMTIME st; //time & date function in c ++
 ...
 while(1) //insert in your looping
 {
   // how to make fps !!
   sprintf(f, "%d", fpsfix);
   fps = fps++;
   GetSystemTime(&st); //using GetSystemTime function.
   if (second != st.wSecond) //if the second is not same as before then record fps
   {   
   fpsfix = fps;
   fps = 0;
   }
   second = st.wSecond;

   cvPutText (frame,"fps : ",cvPoint(10,230), &font2, cvScalar(0,255,0));
   cvPutText (frame,f,cvPoint(40,230), &font2, cvScalar(0,255,0));
   ...
 }
 return 0;
}


picture 0_frame_per_second

Tuesday, August 9, 2011

Episode_11_Add_images_with_difference_size

"Can I play with Mr.Smile ?"
Of Course ,you can!! Now in this episode
I will play with Mr.Smile in OpenCV

Maybe you are familiar adding image from image with the same sizes. How about with difference size? Have you never tried before?? So this is the right time for you to learn more about image processing, it's look like a Gif format isn't it?
picture 0_Add_images_difference_size

How to make it:
  1. First we need to prepare two different size images. if you need a transparent background, you should make a black background like as My Mr.Smile icon.

    picture 1_Example_of_images_with_difference_size

  2. Now prepare second layers with black background and different sizes. First layer, I use 1024x768 pixels and the second layer 640x480 pixels. Why?? When I put the image exceeds 640x480 size with Mr.Smile icon, the program will crash.


    picture 2_The_image_exceeds_640x480_size

    IplImage* src, * res, * roi, *cmb, *roi2, *thr;
    src = cvLoadImage("Smile.jpg",1);
    cmb = cvLoadImage("Scientist.jpg",1);
    res = cvCreateImage(cvSize(640,480), 8, 3); //640x480 for Scientist.jpg
    thr = cvCreateImage(cvSize(640,480), 8, 3); //640x480 for Scientist.jpg
    roi = cvCreateImage(cvSize(1024,768), 8, 3); //1024x768
    roi2 = cvCreateImage(cvGetSize(cmb), 8, 1); //for mask

    picture 3_first_layer_&_second_layer

  3. Now we use ImageROI to make it two layers with same size.

    /* First ROI 1024x768 */
    CvRect rect = cvRect(150, 400, src->width, src->height); // change coordinates
    cvSetImageROI(roi, rect);
    cvAdd(src, roi, roi, NULL);
    cvResetImageROI(roi);

    picture 4_Mr.Smile_icon_add_to_first_layer
    /* Second ROI 640x480 */
    CvRect rect2 = cvRect(250, 250, res->width, res->height);
    cvSetImageROI(roi, rect2);
    cvAdd(res, roi, res, NULL);
    cvResetImageROI(roi);

    picture 5_First_layer_add_to_second_layer

  4. After that we use image processing to make a mask layer.
    /* Filtering image to make a mask */
    cvThreshold(res,thr,50,255,1);
    cvCvtColor(thr,roi2,CV_BGR2GRAY);
    cvThreshold(roi2,roi2,250,255,1);
    cvNot(roi2,roi2);
    cvAdd(cmb,res,res,roi2);
    picture 6_Mask_layer
  5. Finish!! Let's show the result

    picture 7_Result
Full Source Code :

#include "cv.h"
#include "cxcore.h"
#include "highgui.h"

int main(int argc, char** argv)
{
IplImage* src, * res, * roi, *cmb, *roi2, *thr;
src = cvLoadImage("Smile.jpg",1);
cmb = cvLoadImage("Scientist.jpg",1);
res = cvCreateImage(cvSize(640,480), 8, 3); //640x480 for camera
thr = cvCreateImage(cvSize(640,480), 8, 3); //640x480 for camera
roi = cvCreateImage(cvSize(1024,768), 8, 3); //1024x768
roi2 = cvCreateImage(cvGetSize(cmb), 8, 1); //for mask

/* prepare the 'ROI' image */
cvZero(res);
cvZero(roi);
cvZero(roi2);

/* First ROI 1024x768 */
CvRect rect = cvRect(350, 350, src->width, src->height); // change coordinates
cvSetImageROI(roi, rect);
cvAdd(src, roi, roi, NULL);
cvResetImageROI(roi);
/* Second ROI 640x480 */
CvRect rect2 = cvRect(250, 250, res->width, res->height);
cvSetImageROI(roi, rect2);
cvAdd(res, roi, res, NULL);
cvResetImageROI(roi);

/* Filtering image to make a mask */
cvThreshold(res,thr,50,255,1);
cvCvtColor(thr,roi2,CV_BGR2GRAY);
cvThreshold(roi2,roi2,250,255,1);
cvNot(roi2,roi2);
cvAdd(cmb,res,res,roi2);

cvShowImage("src", res);
cvShowImage("res", src);
cvWaitKey(0);
/* be tidy */
cvDestroyAllWindows();
cvReleaseImage(&src);
cvReleaseImage(&res);
cvReleaseImage(&roi);
return 0;

}
More of my experiments using cvadd

Wednesday, July 20, 2011

Episode_10_Memory_Leak (The scariest nightmare for a programmer)


Memory leak often occurs when a computer program is unable to release the memory back to the operating system. In programming, a memory leak happens when an object is stored in a memory but cannot be accessed by the running code and generally can only be diagnosed by a programmer with access to the program source code. However, many people refer to any unwanted increase in memory usage as a memory leak, though this is not strictly accurate from a technical perspective.

picture 0_memory_leak_OpenCv
That's an example when memory leak occured in my program, so I diagnosed my source code. It usually happens when we loop the program continuously. Well, if you make a program, you must check step by step your source code whether it is stable or not. After you run your program, you can check with task manager and search your name of the program ***.exe. It will show you how much memory is used by your program.

picture 1_check_memory_myprogram_with_task_manager

So After I have diagnosed and repaired my source code, it shows that I just used 18.064k of 2,00GB memory.

picture 2_myprogram_more_stable_after_diagnose
With your looping source code, I say again that you must be careful with your code that is unable to release memory back to the operating system. I will give an Example in OpenCv source code (C++):

Source Code here :

...
while(1){
  ...
  ...
  result = cvCreateImage(size, frame->depth, frame->nChannels); // its wrong
  tes = cvLoadImage("smile.png",1); //wrong
  cvNamedWindow("original", CV_WINDOW_AUTOSIZE); //wrong again
  cvSaveImage("Temp.jpg",img); //also wrong
  ...
  ...
  }
return0;
}

And many other events that you can find with your source code that is important to located it in your looping section.
...
...
cvNamedWindow("original", CV_WINDOW_AUTOSIZE);
result = cvCreateImage(size, frame->depth, frame->nChannels);
while(1){
  ...
  ...
  if (Load == TRUE) tes = cvLoadImage("smile.png",1);
  if (Save == TRUE) cvSaveImage("Temp.jpg",img);
  ...
  ...
  }
return0;
}

Note : It is simple but important especially it can drive you crazy when your program is finished but unstable and consuming huge amounts of memory.


Sunday, July 17, 2011

Episode_9_Trying_OpenCV_with_Python


Well let's try using OpenCV with Python binding on Win 7... Why using Python? this is open source programming language.. On the other side, it's very helpfull us when the project requires a mathematical algorithm like as MATLAB.


Here How I used OpenCV with python.
  1. Of course we need OpenCV2.1 from Sourceforge. (Why I using OpenCV2.1 couse the library OpenCV2.1 is more suitable with Pyhton 2.6.4 than the others).
  2. Install Pyhton 2.6.4 (you can download from python.org).
  3. Make some edits to your Environment Variables : MyComputer -> Properties - > Advanced system settings -> Environment Variables.
  • Add C:/Python26;C:/OpenCV2.1/bin to PATH
  • Add C:/OpenCV2.1/Python2.6/Lib/site-packages to PYTHONPATH
Ok Test some examples.. you get it in opencv example C:\OpenCV2.1\samples


picture 0_Directories_OpenCV_python_in_mycomputer

Test camera.py
picture 1_camera.py

Warning !!! Behind me is a gay... be carefull if u see him in 202 Laboratory..

Wednesday, May 11, 2011

Episode_8_Template_Matching_with_WebCam

Template matching is a technique in digital image processing for finding small parts of an image which match a template image. It can be used in manufacturing as a part of quality control, a way to navigate a mobile robot, or as a way to detect edges in images. For more information about Template matching, you can see in http://en.wikipedia.org/wiki/Template_matching.

I use some combination such as mouse handler, img ROI, and template matching.
#include "cv.h"
#include "cxcore.h"
#include "highgui.h"

int c;
IplImage *frame,*img, *img1;
IplImage *tpl;
IplImage *res;
double minval, maxval;
int frame_width, frame_height;
int tpl_width, tpl_height;
int res_width, res_height;
int drag = 0;
CvPoint minloc, maxloc;
CvPoint point;
bool pause = FALSE;
bool crop = FALSE;

void mouseHandler(int event, int x, int y, int flags, void* param)
{
/* user press left button */
if (event == CV_EVENT_LBUTTONDOWN && !drag)
{
point = cvPoint(x, y);
drag = 1;
}

/* user drag the mouse */
if (event == CV_EVENT_MOUSEMOVE && drag)
{
cvDestroyWindow("template");
img = cvCloneImage(frame);
cvRectangle(img,point,cvPoint(x, y),CV_RGB(255, 0, 0),1, 8, 0);
cvCopy(img,frame, NULL);
cvShowImage("reference", img);
pause = TRUE;
crop = FALSE;
}

/* user move the mouse */
if (event == CV_EVENT_MOUSEMOVE && !drag)
{
point = cvPoint(x, y);
printf("x = %d y = %d \n",point.x,point.y);
}

/* user release left button */
int a,b;
if (event == CV_EVENT_LBUTTONUP && drag)
{
img = cvCloneImage(frame);
if ((x == point.x) || (y == point.y)) goto jmp;
if ((point.x > x) && (point.y < y)) goto jmp;
if ((point.x < x) && (point.y > y)) goto jmp;
if ((point.x > x) && (point.y > y)) {
 a = point.x;
 b = point.y;
 point.x = x;
 point.y = y;
 x = a;
 y = b;
}
cvSetImageROI(img,cvRect(point.x,point.y,x - point.x,y - point.y));
cvShowImage("template", img);

/* load template image */
cvSaveImage("Temp.jpg",img);
tpl = cvLoadImage("Temp.jpg");
/* get image's properties */
tpl_width = tpl->width;
tpl_height = tpl->height;
res_width = frame_width - tpl_width + 1;
res_height = frame_height - tpl_height + 1;

cvResetImageROI(img);
jmp:
drag = 0;
pause = FALSE;
crop = TRUE;
}
}

int main(int argc, char* argv[])
{
CvCapture* capture = cvCaptureFromCAM(0);
cvSetCaptureProperty( capture, CV_CAP_PROP_FRAME_WIDTH, 320 );
cvSetCaptureProperty( capture, CV_CAP_PROP_FRAME_HEIGHT, 240 );

cvNamedWindow( "reference", CV_WINDOW_AUTOSIZE );
cvSetMouseCallback("reference", mouseHandler, NULL);

/* get image's properties */
frame=cvQueryFrame(capture);
frame_width = frame->width;
frame_height = frame->height;

while(1)
{
frame=cvQueryFrame(capture);

if (crop){
 /* create new image for template matching computation */
 res = cvCreateImage( cvSize( res_width, res_height ), IPL_DEPTH_32F, 1 );
 /* choose template matching method to be used */
 cvMatchTemplate( frame, tpl, res, CV_TM_SQDIFF );
 cvMinMaxLoc( res, &minval, &maxval, &minloc, &maxloc, 0 );
 /* draw red rectangle */
 cvRectangle( frame,cvPoint( minloc.x, minloc.y ),cvPoint( minloc.x + tpl_width, minloc.y + tpl_height ),cvScalar( 0, 0, 255, 0 ), 2, 0, 0 );
}

if (!pause) cvShowImage( "reference", frame );
/* wait until user press a key to exit */
c=cvWaitKey(10);
// escape key terminates program
if(c == 27)
break;
}
/* free memory */
cvDestroyWindow( "reference" );
cvReleaseImage( &frame );


return 0;
}
This my first video,


Episode_7_Using_Image_ROI

If you ever use paint application, I am sure that you know what is the use of crop tool. In this episode 7, I will tell you how to crop the image using opencv. it's very important when you want to take data in specific areas. Now Let's try it !!
#include "cv.h"
#include "cxcore.h"
#include "highgui.h"

IplImage* frame, * img1;
CvPoint point;
int drag = 0;
int key = 0;
bool pause = FALSE;

void mouseHandler(int event, int x, int y, int flags, void* param)
{
    /* user press left button */
    if (event == CV_EVENT_LBUTTONDOWN && !drag)
    {
        point = cvPoint(x, y);
        drag = 1;
    }

    /* user drag the mouse */
    if (event == CV_EVENT_MOUSEMOVE && drag)
    {
        cvDestroyWindow("result");
        img1 = cvCloneImage(frame);
        cvRectangle(img1,point,cvPoint(x, y),CV_RGB(255, 0, 0),1, 8, 0);
        cvCopy(img1,frame, NULL);
        cvShowImage("origin", img1);
        pause = TRUE;
    }

    /* user move the mouse */
    if (event == CV_EVENT_MOUSEMOVE && !drag)
    {
        point = cvPoint(x, y);
        printf("x = %d y = %d \n",point.x,point.y);
    }

    /* user release left button */
    int a,b;
    if (event == CV_EVENT_LBUTTONUP && drag)
    {
        img1 = cvCloneImage(frame);
        if ((x == point.x) || (y == point.y)) goto jmp;
        if ((point.x > x) && (point.y < y)) goto jmp;
        if ((point.x < x) && (point.y > y)) goto jmp;
        if ((point.x > x) && (point.y > y)) {
            a = point.x;
            b = point.y;
            point.x = x;
            point.y = y;
            x = a;
            y = b;
        }
        cvSetImageROI(img1,cvRect(point.x,point.y,x - point.x,y - point.y));
        cvNot(img1, img1); // or do whatever with the ROI
        cvShowImage("result", img1);
        cvResetImageROI(img1);
        //cvCopy(img1,frame, NULL); //other tricks
        //cvShowImage("result1", img1);
        jmp:
        drag = 0;
        pause = FALSE;
    }

    /* user click right button: reset all */
    if (event == CV_EVENT_RBUTTONUP)
    {
        cvShowImage("result", frame);
        drag = 0;
    }
}

int main(int argc, char *argv[])
{
    /* create a window for the video */
    cvNamedWindow( "origin", CV_WINDOW_AUTOSIZE );
    cvSetMouseCallback("origin", mouseHandler, NULL);

    while( key != 'q' )
    {
    frame = cvLoadImage("uniq.jpg");
    if (!pause) cvShowImage("origin", frame);
    key = cvWaitKey( 30 );
    }

    cvDestroyWindow("origin");
    cvReleaseImage(&frame);
    cvReleaseImage(&img1);
    return 0;
}


And the result

picture 0_image_ROI

Wednesday, May 4, 2011

Episode_6_Filtering_Images

FILTERING IMAGE
In the previous episode, you have learned how to load an image with OpenCV (Episode 2). For this episode, we will try how to editing of the image in the real time such as brightness, negatve, bluring, removing noise, etc. We will now move on to higher-level methods that treat the images as images, and not just as arrays of colored (or grayscale) values. When we say “image processing”, we mean just that: using higher-level operators that are defined on image structures in order to accomplish tasks whose meaning is naturally defi ned in the context of graphical, visual images.


Part A. Eroding the ImageI won't explain in detail what is erode? because you can learn it yourself. Here I will Erode the image 2 times;

#include "cv.h"
#include "cxcore.h"
#include "highgui.h"

int main(int argc, char* argv[])
{
IplImage *img = cvLoadImage("Step_forward.jpg");
IplImage *img2= cvCreateImage( cvGetSize(img), 8,1 );
img2 = cvCloneImage(img);

cvErode(img2, img2, 0, 2);

cvShowImage("Image:",img);
cvShowImage("Eroded", img2);

cvWaitKey();
cvReleaseImage(&img);
cvReleaseImage(&img2);

return 0;
}



picture 0_eroding_images

Part B. Dilating the Image
Dilating the image is the opposite of Erode image.


#include "cv.h"
#include "cxcore.h"
#include "highgui.h"

int main(int argc, char* argv[])
{
IplImage *img = cvLoadImage("Step_forward.jpg");
IplImage *img2= cvCreateImage( cvGetSize(img), 8,1 );
img2 = cvCloneImage(img);

cvDilate(img2, img2, 0, 2);

cvShowImage("Image:",img);
cvShowImage("Dilate", img2);

cvWaitKey();
cvReleaseImage(&img);
cvReleaseImage(&img2);

return 0;
}


picture 1_dilating_images

Part C. BrightnessFor brightness, we use the cvAddS function. This function adds a scalar to each element of the matrix. There also exists a cvAdd function, but that function adds two matrices (not a matrix and a value).


#include "cv.h"
#include "cxcore.h"
#include "highgui.h"

int main(int argc, char* argv[])
{
IplImage *img = cvLoadImage("Step_forward.jpg");
IplImage *img2= cvCreateImage( cvGetSize(img), 8,1 );
img2 = cvCloneImage(img);

cvAddS(img2, cvScalar(50,50,50), img2);

cvShowImage("Image:",img);
cvShowImage("Brigth", img2);

cvWaitKey();
cvReleaseImage(&img);
cvReleaseImage(&img2);

return 0;
}


picture 2_brightness_image

Part D. Contrast
For brightness, you added values. For contrast, you multiply.
For adding scalars (values), you use cvAddS. For multiplying, you use cvScale

#include "cv.h"
#include "cxcore.h"
#include "highgui.h"

int main(int argc, char* argv[])
{
IplImage *img = cvLoadImage("Step_forward.jpg");
IplImage *img2= cvCreateImage( cvGetSize(img), 8,1 );
img2 = cvCloneImage(img);

cvScale(img2, img2, 2);

cvShowImage("Image:",img);
cvShowImage("Contrast", img2);

cvWaitKey();
cvReleaseImage(&img);
cvReleaseImage(&img2);

return 0;
}


picture 3_contrast_image

Part E. NegativeInverting the image such as giving a NOT logic operation on each element matrix.
#include "cv.h"
#include "cxcore.h"
#include "highgui.h"

int main(int argc, char* argv[])
{
IplImage *img = cvLoadImage("Step_forward.jpg");
IplImage *img2= cvCreateImage( cvGetSize(img), 8,1 );
img2 = cvCloneImage(img);

cvNot(img2, img2);

cvShowImage("Image:",img);
cvShowImage("Negative", img2);

cvWaitKey();
cvReleaseImage(&img);
cvReleaseImage(&img2);

return 0;
}


picture 4_negative_images

Part F. GrayColorFor further image processing, is generally used to change the image in the form of a binary image so its can facilitate the morpological operation such as threshold, laplace, Canny, and others.

#include "cv.h"
#include "cxcore.h"
#include "highgui.h"

int main(int argc, char* argv[])
{
IplImage *img = cvLoadImage("Step_forward.jpg");
IplImage *img2= cvCreateImage( cvGetSize(img), 8,1 );

cvCvtColor(img,img2,CV_BGR2GRAY);

cvShowImage("Image:",img);
cvShowImage("Gray", img2);

cvWaitKey();
cvReleaseImage(&img);
cvReleaseImage(&img2);

return 0;
}
picture 5_graycolor

Okay these are just some of the existing filtering process in OpenCV. I will continue it at any time.

Referensi for Beginner