1 / 41

Introduction to IPL and OpenCV libraries

Introduction to IPL and OpenCV libraries. Bogdan Raducanu Centre de Visiò per Computador E-mail: bogdan@cvc.uab.es. Cover Story. OpenCV was of key use in the vision system of "Stanley". What is IPL?.

malana
Download Presentation

Introduction to IPL and OpenCV libraries

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction to IPL and OpenCV libraries Bogdan Raducanu Centre de Visiò per Computador E-mail: bogdan@cvc.uab.es

  2. Cover Story OpenCV was of key use in the vision system of "Stanley"

  3. What is IPL? - The IPL (Image Processing Library ) is a collection of functions implementing several image processing algorithms. It was developed by INTEL. - Is optimized for MMX and different processor types (there is a DLL for each type of INTEL processor) - The images are stored in a specific data structure. In order to work with the image, we need to know the information contained in the structure header (contains image specific characteristics)

  4. Image structure IplImage * IplImage Data • Width • Height • Bits per pixel • Channel sequence • … • pointer to image data

  5. The “IplImage” structure includes a header containing image information and attributes: • -nChannels: number of channels • (1 for grayscale, 3 for RGB, 4 for CMYK, ...) • depth: number of bits/pixel and data type • IPL_DEPTH_1U (1-bit) • IPL_DEPTH_8U (8-bit unsigned) • IPL_DEPTH_8S (8-bit signed) • IPL_DEPTH_16U (16-bit unsigned) • IPL_DEPTH_16S (16-bit signed) • IPL_DEPTH_32S (32-bit signed) • IPL_DEPTH_32F (32-bit float)

  6. colorModel: “GRAY”, “RGB”, “CMYK”, etc. • channelSeq: “GRAY”, “BGR”, “BGRA”, “RGB”, “RGBA”, “HSV”, “YUV”, etc. • dataOrder: RGBRGBRGB... or RRR...GGG...BBB • origin: Top-Left or Bottom-Left (IPL_ORIGIN_TLó IPL_ORIGIN_BL) • scanline alignment: DWORD or QWORD • width (in pixels) • height (in pixels) • ROI: (could be NULL) • maskROI: (could be NULL) • imageSize: image size (in bytes) • - imageData: pointer to pixel data

  7. Function categories in IPL: • - create/destroy an image and access its content • arithmetical/logical operations • filtering • - morphological operations • color space conversion • histogram • linear/geometrical transformations • image statistics

  8. Create/destroy an image and access its content: • - iplCreateImageHeader • iplAllocateImage • iplCreateROI • iplSetROI • iplCopy • iplClone • iplDeallocateImage • iplPutPixel • iplGetPixel

  9. Example (create/destroy an image): #include ”ipl.h” … IplImage*img = iplCreateImageHeader( 3, 0, IPL_DEPTH_8U, “RGB”, “BGR”, IPL_DATA_ORDER_PIXEL, IPL_ORIGIN_TL, IPL_ALIGN_QWORD, 150, 100, NULL, NULL, NULL, NULL ); iplAllocateImage(img, 0, 0); /////// Use the image //////// iplDeallocate(img, IPL_IMAGE_ALL);

  10. We created an image of 150x100 pixels with 3 channels. The color model is RGB and data type is 8 bits/pixel, unsigned. The channel order is BGRBGRBGR… starting from the upper row. The data are aligned in memory as QWORD (64 bits). There is no ROI defined • We allocated memory for data, but without initialize it. • With the IPL_IMAGE_ALL parameter, we freed the header of the structure, the data and the existing ROIs (if any)

  11. - The content of an IplImage can be accessed in several ways: • - using the functions GetPixel and PutPixel: • Inconvenience: slow access • - go directly to the memory address corresponding to the pixel al pixel: • - Inconvenience: • this operation is ‘complex’ because we have to compute the memory address beforehand • careful with the data types • - Advantage: • is faster to access a big chunk of data

  12. - Example: Let’s assume we have a RGB image with 8 bits/pixel, unsigned: #include ”ipl.h” … IplImage*img = iplCreateImageHeader( 3, 0, IPL_DEPTH_8U, “RGB”, “BGR”, IPL_DATA_ORDER_PIXEL, IPL_ORIGIN_TL, IPL_ALIGN_QWORD, 150, 100, NULL, NULL, NULL, NULL ); iplAllocateImage(img, 0, 0); unsigned char *R,*G,*B; B = (unsigned char *) img->imageData; G = B+1; R = G+1; for (int i=0; i<15000; i++, R+=3, G+=3, B+=3) { ///// here we can use/modify the pixel data } iplDeallocate(img, IPL_IMAGE_ALL);

  13. Remarks: • in order to be able to use the IPL functions, you must include the header file “ipl.h” in your source code • include the “ipl.lib” in the project settings • the structures’ names use the “Ipl” (“I” uppercase) prefix, meanwhile the functions’ names use the “ipl” (“i” lowercase) prefix • the remaining function categories will be presented in the section dedicated to OpenCV library • for more information, consult the IPL manual: • C:\Archivos de Programa\Intel\plsuite\doc\iplman.pdf

  14. OpenCV library • Why OpenCV? • IPL is a “low-level” library (it allows basic operations) • OpenCV is a library which contains more complex data structures and “high-level” functions: optical flow, pattern recognition 2D-3D real-time tracking, camera calibration, etc. • it comes with some extensions that allow: • accessing a camera or working with AVI files • graphical user interface (“HighGUI”) allowing a faster and easier way • to interact/visualize the images

  15. OpenCV is in general compatible with the IPL library. It is based also on the “IplImage” structure. But it must be employed taking into account the following restrictions: • - the image statistics functions require that “IplImage” be defined either with a single channel or three channels of the following data types: IPL_DEPTH_8U, IPL_DEPTH_8S or IPL_DEPTH_32F. • OpenCV supports only interleaved images • the following attributes: colorModel, channelSeq, BorderMode, and BorderConst are ignored • the attibutes maskROI and tileInfo must be set to 0. • the ROIs of the input and output image must be the same. • Remark: the structures’ name uses the “Cv” (“C” uppercase) prefix, meanwhile the functions’ name uses the “cv” (“c” lowercase) prefix.

  16. Create/Destroy an image and access its content • cvCreateImage • cvCreateImageHeader • cvReleaseImageHeader • cvReleaseImage • cvCreateImageData • cvReleaseImageData • cvSetImageROI • cvCopyImage • cvCloneImage • cvGetImageRawData

  17. #include ”ipl.h” #include “cv.h” #include “cxcore.h” … IplImage*img = cvCreateImage(cvSize(150, 100), IPL_DEPTH_8U, 3); /////// Use the image //////// cvReleaseImage(&img); Remark: In some cases (when we work with images captured from the camera), would be more convenient to initialize the IplImage structure using the IPL functions.

  18. Arithmetical/Logical operations • Arithmetical operations: • unary; cvAddS, cvSubS, ... • binary: cvAdd, cvMul, cvSub, cvMatMulAdd, cvInvert, ... • Logical operations • unary: cvAndS, cvOrS, cvXorS, ... • binary: cvAnd, cvOr, cvXor, ...

  19. Remark: Most of the OpenCV functions are defined to support both the IplImage and CvMat data types. That’s possible because of the definition of CvArr data type: typedef void CvArr; The OpenCV functions look-up for the first integer of the structure being passed, in order to distinguish between the two data types. In the case of IplImage, this integer is equal to the size of IplImage structure, meanwhile it is equal to 0x4224xxxx in the case of CvMat.

  20. Image filtering • Based on convolution with fixed kernel: • cvLaplace, cvSobel, cvSmooth • Morphological operations • cvErode, cvDilate • cvMorphologyEx (advanced operations: opening, closing, Top-Hat, etc.) • user-defined structuring elements: • cvCreateStructuringElementEx • cvReleaseStructuringElement

  21. Example: application of the erosion function

  22. Color space conversion • cvCvtColor allows the following color-space conversions: • CV_RGB2GRAY • CV_RGB2HSV • CV_RGB2YCrCb • Histogram • cvCreateHist, cvReleaseHist • cvCalcHist, cvCopyHist • cvCompareHist, cvThreshHist • cvGetMinMaxHistValue, cvNormalizeHist

  23. Linear/geometrical transformations • cvFFT (Fast Fourier Transform) • cvDCT (Discrete Cosine Transform) • cvResize, cvMirror, cvConvertScale • Feature extraction • cvCanny (border detection) • cvHoughLines (line detection) • cvFindCornerSubPix (corner detection)

  24. Example of border detection

  25. Image statistics • cvNorm (C- , L1- and L2-norm) • cvMoments (spatial and central moments) • cvMinMaxLoc (find the min/max values) • Drawing functions • cvLine, cvRectangle • cvCircle, cvEllipse • cvPolyLine, cvFillPoly • cvInitFont, cvPutText

  26. Motion analysis • estimators: Kalman, Condensation • cvKalmanXX, cvCondensXX • movement patterns • cvCalcMotionGradient, cvMotionHistoryUpdate • optical flow • cvCalcOpticalFlowPyrLK (implements the Lucas-Kanade method based on pyramidal decomposition) • tracking • cvMeanShift, cvCamShift, cvSnakeImage • background substraction

  27. 3D reconstruction • - camera calibration • cvCalibrateCamera, cvFindExtrinsecCameraParams, cvUnDistort • hand detection • cvFindHandRegion • pose estimation • cvPOSIT • finding pixel correspondence in a pair of stereo images • cvFindStereoCorrespondence

  28. Example of distorsion correction

  29. Object detection (faces) The object detection algorithm implemented in OpenCV is based on the following papers: - Paul Viola and Michael J. Jones. Rapid Object Detection using a Boosted Cascade of Simple Features. IEEE CVPR, 2001 and - Rainer Lienhart and Jochen Maydt. An Extended Set of Haar-like Features for Rapid Object Detection. IEEE ICIP 2002, Vol. 1, pp. 900-903, Sep. 2002.

  30. Idea A classifier (namely a cascade of boosted classifiers working with haar-like features) is trained with a few hundreds of sample views of a particular object (i.e., a face or a car), called positive examples, that are scaled to the same size (say, 20x20), and negative examples - arbitrary images of the same size.

  31. The weak classifier outputs a "1" if the region is likely to show the object of interest (i.e., face/car), and "0" otherwise • - The classifier is designed so that it can be easily "resized" in order to be able to find the objects of interest at different sizes, which is more efficient than resizing the image itself. • - The word "cascade" in the classifier name means that the resultant classifier consists of several simpler classifiers (stages) that are applied subsequently to a region of interest until at some stage the candidate is rejected or all the stages are passed.

  32. Data structures and functions implemented in OpenCV for face detection • CvHaarClassifierCascade (structure representing a cascade of classifiers) • cvLoadHaarClassifierCascade (reads from a file a cascade of classifiers and stores it in the CvHaarClassifierCascade structure). The cascade is stored in a XML file format: • C:\Archivos de programa\Intel\OpenCV\data\haarcascades\...) • cvHaarDetectObjects (detects the objects in the image) • cvReleaseHaarClassifierCascade (frees the memory occupied by the CvHaarClassifierCascade structure)

  33. HighGUI library • Allows a fast and easy interaction/visualization with/of images • - Reading an image from a file • cvLoadImage (const char* filename, int iscolor CV_DEFAULT(1)); • Writing an image to a file • cvSaveImage (const char* filename, const CvArr* image); • It supports several formats: BMP, GIF, JPG, TIFF, etc. • Open a window for visualization • - cvNamedWindow (const char* name, int flags); • Visualize the image • cvShowImage (const char* name, const CvArr* image);

  34. The structure of OpenCV • The ‘cv.h’ contains: • general image processing functions: filter, color conversion, morphological operators, structural analysis, motion analysis, pattern recognition (object detection), camera calibration and 3D reconstruction • The ‘cxcore.h’ contains: • basic structures, arithmetical/logical operators (copy, transformation), dynamic structures (sets, graphs, trees), drawing functions, error handling and system functions • The ‘cvaux.h’ • - stereo correspondence, texture descriptors, 2D-3D trackers, background segmentation, morphing, etc. • The ‘highgui.h’ contains: • user interface

  35. Installation of IPL/OpenCV libraries and environment settings for MSVC++ • In order to keep track with the following settings, OpenCV folder and IPL folder (named ‘plsuite’) must be installed in C:\Archivos de Programa\Intel\ • in the ‘c’ source file add the header files needed: ‘ipl.h’, ‘cv.h’, ‘cxcore.h’, ‘highgui.h’, ‘cvaux.h’, ‘cvhaartraining.h’ • from the ‘Project’ menu, choose ‘Settings’ and click on the ‘Link Tab’. In the field: ‘Object/library modules’ add the following: • ipl.lib cv.lib cxcore.lib highgui.lib cvaux.lib cvhaartraining.lib (if you use the face detector functions)

  36. from the menu ‘Tools’ choose ‘Options’ and then click on the ‘Directories’ tab. In the field ‘Show directories for’ choose ‘Include files’, then edit the following paths: • C:\ARCHIVOS DE PROGRAMA\INTEL\PLSUITE\INCLUDE • C:\ARCHIVOS DE PROGRAMA\INTEL\OPENCV\CV\INCLUDE • C:\ARCHIVOS DE PROGRAMA\INTEL\OPENCV\OTHERLIBS\HIGHGUI • C:\ARCHIVOS DE PROGRAMA\INTEL\OPENCV\CXCORE\INCLUDE • C:\ ARCHIVOS DE PROGRAMA\INTEL\OPENCV\apps\HaarTraining\include • (if you are using the face detector)

  37. - from the menu ‘Tools’ choose ‘Options’ and then click on the ‘Directories’ tab. In the • field ‘Show directories for’ choose ‘Library files’, then edit the following paths: • C:\ARCHIVOS DE PROGRAMA\INTEL\PLSUITE\LIB\MSVC • C:\ARCHIVOS DE PROGRAMA\INTEL\OPENCV\LIB

  38. the last step consists in adding the paths for DLLs in the ‘Environment Variables’ • from ‘Control Panel’, choose ‘System’, then ‘Advanced’ and finally click on ‘Environment Variable’. From dialog box that appears, in the ‘System Variables’ section, click and edit the ‘PATH’ item • you have to add the following route: • C:\Archivos de programa\Intel\OpenCV\bin • C:\Archivos de programa\Intel\plsuite\bin

  39. Examples • Three examples are included (provided in separate files): • first is about getting a live stream from a webcam (you must make sure you have DirectX library installed • second is about detecting faces in an image by using the detector which comes with the OpenCV library • third is about image convolution • More information... • http://www.cvc.uab.es/~bogdan/CV/cv.html • IPL/OpenCV online documentation (html, pdf files) • C:\Archivos de programa\Intel\OpenCV\samples • http://www.site.uottawa.ca/~laganier/tutorial/opencv+directshow/cvision.htm • http://groups.yahoo.com/group/OpenCV/ (you have to register)

More Related