In 1999 Intel Research decided to develop a software library to
encourage the development of CPU-intensive applications. Image
processing and computer vision tasks certainly make intensive use of
the CPU.
- to advance research by providing a set of optimized base functions;
- to encourage the production of readable and reusable software;
- to encourage the production of commercial software that uses an open and free infrastructure (through proper licensing).
The library is written in C language, for maximal efficiency and portability, but there are wrappers for C++ (openFrameworks), C#, Pyton, Java (JavaCV), and Processing (OpenCV and JavacvPro). It is available under several operating systems, including Linux, Windows, Mac OS, Android, and iOS. After several beta releases the first 1.0 version was released in 2006, and the 2.0 was released in october 2009, with a new C++ interface. At the moment of writing OpenCV 2.4 has been released and is available through the
developer web site.
Within the Processing language and environment, a small set of OpenCV
functions has been available for years through the OpenCV "hypermedia"
library. The available functions were real-time capture, video file
import, basic image treatment, face and body identification, and blob
detection. A quick introduction to the use of this library can be
found in
Andy Best's tutorial or in Programming
Interactivity by Joshua Noble [link]. This library, no
longer updated, is now superseeded by the JavacvPro
library, which has a much larger set of functions.
The JavacvPro library makes its operations on a buffer, and it uses two auxiliary buffers where images can be stored and recalled with remember() and restore() operations. This allows, for example, to compute the difference between two neighboring frames.
import codeanticode.gsvideo.*;
import monclubelec.javacvPro.*;
OpenCV opencv; //declare an OpenCV object
GSCapture cam; //declare a camera object
void setup(){
size(320,240);
opencv = new OpenCV(this); //objectcv instance
opencv.allocate(320,240);
cam = new GSCapture(this,320,240); //prepare for reading a video stream
cam.start();
}
void draw(){
if (cam.available()){
cam.read(); //frame grabbing
opencv.copy(cam.get());
opencv.absDiff(); //difference of frames: result in Memory2
image(opencv.getMemory2(),0,0); //display the difference image
}
}
void keyPressed() {
opencv.remember(); //store the frame in the memory buffer
println("remember");
}
// stop processing
public void stop(){
cam.delete();
super.stop();
}
By addition of
opencv.remember() as the last line of the
internal block of the
draw() function, the difference is
computed between two contiguous frames, thus obtaining a visualization
of motion.
If, instead of the
image(opencv.getMemory2(),0,0), we put the instructions
opencv.restore2();
opencv.gray();
opencv.threshold(0.5);
image(opencv.getBuffer(),0,0);
the difference is reduced to gray levels and then thresholded at 0.5 level.
Within a frame, a blob is a contiguous area represented in a data
structure. A blob has a contour that represent a contrasting edge with
neighboring regions.
Tracking an object means to follow in time its corresponding blob that
is moving across the screen. Previous-frame blobs are compared with
the current-frame blobs to identify those blobs that are likely to
keep representing the same object.
The following code has been derived from Programming
Interactivity by Joshua Noble [link], adapted to use the
JavacvPro library.
import codeanticode.gsvideo.*;
import monclubelec.javacvPro.*;
int IMG_WIDTH = 320;
int IMG_HEIGHT = 240;
int BLURAMOUNT = 7;
float THRESHOLD = 0.8;
OpenCV opencv;
GSCapture cam;
void setup(){
size(IMG_WIDTH*2, IMG_HEIGHT*2);
cam = new GSCapture(this, IMG_WIDTH, IMG_HEIGHT);
opencv = new OpenCV(this);
opencv.allocate(IMG_WIDTH, IMG_HEIGHT);
cam.start();
}
void draw(){
if (cam.available() == true) {
cam.read();
opencv.copy(cam.get());
opencv.remember();
opencv.blur(BLURAMOUNT);
image(opencv.getBuffer(),IMG_WIDTH,0);
opencv.restore();
opencv.invert();
image(opencv.getBuffer(),0,IMG_HEIGHT);
opencv.restore();
opencv.gray();
opencv.threshold(THRESHOLD);
image(opencv.getBuffer(),0,0);
}
// Args: minAreaIn, maxAreaIn, maxBlobIn, findHolesIn, maxVerticesIn, debug
Blob[] blobs = opencv.blobs(opencv.area()/256, opencv.area()/2, 20, false, 100, false);
for (int i=0; i < blobs.length; i++){
noFill();
rect(blobs[i].rectangle.x, blobs[i].rectangle.y, blobs[i].rectangle.width, blobs[i].rectangle.height);
stroke(0,0,255);
line(blobs[i].centroid.x-5, blobs[i].centroid.y, blobs[i].centroid.x+5, blobs[i].centroid.y);
line(blobs[i].centroid.x, blobs[i].centroid.y-5, blobs[i].centroid.x, blobs[i].centroid.y+5);
fill(255,0,255,64); stroke(0,255,0);
if (blobs[i].points.length > 0){
beginShape();
for (int j=0; j < blobs[i].points.length; j++){
vertex(blobs[i].points[j].x, blobs[i].points[j].y);
}
endShape(CLOSE);
}
}
}
With OpenCV, it is possible to load the configuration of a classifier
that has been trained to recognize a specific shape, such as a human
face or body.
import codeanticode.gsvideo.*;
import monclubelec.javacvPro.*;
import java.awt.*; // for classes Point, Rectangle, ...
OpenCV opencv; //declare an OpenCV object
GSCapture cam; //declare a camera object
void setup(){
size(320,240);
opencv = new OpenCV(this); //objectcv instance
opencv.allocate(320,240);
cam = new GSCapture(this,320,240); //prepare for reading a video stream
cam.start();
opencv.cascade("/usr/local/share/OpenCV/haarcascades/","haarcascade_frontalface_alt.xml"); // load the classifier configuration
}
void draw(){
if (cam.available()){
cam.read(); //frame grabbing
opencv.copy(cam.get());
image(opencv.getBuffer(),0,0);
Rectangle[] faces = opencv.detect(); //look for faces
noFill();
stroke(255,0,0);
for (int i=0; i < faces.length; i++)
rect(faces[i].x, faces[i].y, faces[i].width, faces[i].height);
}
}