In Processing, the representation of graphic objects is based on a cartesian 3D coordinate system, as displayed in Figure 1.

Figure 1: 3D coordinate system used in Processing

Coordinate system

In Processing, an image can be assigned to an object of the class PImage. The function loadImage("myImage") takes a file (gif or jpg) myImage, containing the pixel coding of an image, and gives back the content of such image, which can be assigned to a variable of type PImage. The file myImage must be loaded in the data folder of the directory having the same name as the Processing sketch we are working at.

	  size(400,300);
	  PImage b;
	  b = loadImage("gondoliers.jpg");
	  println("width=" + b.width + " height=" + b.height);
	  image(b, 0, 0, 400, 300); // position (0,0); width=400; height=300;
	  image(b, 20, 10, 100, 80); // position (20,10); width=100; height=80;
	

Since our color receptors (cones), each tuned to a wavelength region, are of three kinds, color models are always referred to a three-dimensional space. In additive color models, each of three axes correspond to a base color, and by mixing three colored light beams one can obtain all colors within a gamut volume in the space defined by the three axes. The three base colors can be chosen arbitrarily or, more often, based on the application domain (e.g., color of three phosphors or laser beams). In printing processes, subtractive color models are used, where the starting point is the white surface and primary ink colors are used to subtract color from white.

In processing color is a primitive type used to specify colors. It is realized by a 32-bit number, where the first byte specifies the alpha value, and the other bytes specify a triple either in the RGB or in the HSB model. The choice of one model or the other is made by the colorMode() function. With three bytes, a number of 256×256×256=16777216 256 256 256 16777216 are representable.

HSB Model

Colors are represented by a triple of numbers, the first number giving the Hue, the second giving Saturation, and the third giving the Brightness.

Note:

Often the model is called HSV, where V stands for Value.

The hue takes values in degrees between 0 0 (red) and 360 360, being the various hues arranged along a circumference and being red positioned at 0 ˚ 0˚ . Saturation and brightness vary between 0 0 and 100 100. The saturation is the degree of purity of color. If a pure color is added with a white light its degree of purity decreases until the color eventually sits on a gray scale when saturation is zero. In physical terms, the brightness is proportional to the signal power spectrum. Intuitively, the brightness is increased when the light intensity increases. The three-dimensional HSB space is well represented by a cylinder, with the hue (nominal scale) arranged along the circumference, the saturation (ratio scale) arranged along the radius, and the brightness (interval scale) arranged along the longitudinal axis. Alternatively, such three-dimensional space can be collapsed into two dimensions, as in the color chooser of the image-processing program Gimp, displayed in Figure 2. Along the circumference, the three primary colors (red, green, and blue) are visible, 120 ˚ 120˚ apart from each other, separated from the secondary colors (magenta, cyan, yellow). Each secondary color is complementary to the primary color in front of it in the circumference. For instance, if we take the green component out of a white light, we obtain a magenta light. The triangle inscribed in the circumference has a vertex pointing to a selected hue. The opposite side contains the gray scale, thus representing colors with null saturation and variable brightness. Going from the reference vertex to the opposite side we have a gradual decrease in saturation.

Figure 2: Color chooser of the software Gimp

Gimp color chooser

Example 2: Loading and visualizing an image with transparency

Table 1

size(400,300); PImage b = loadImage("gondoliers.jpg"); PImage a = loadImage("gondoliers.jpg"); float ramp = 0; for (int j = 0; j < b.height; j++) for (int i = 0; i < b.width; i++) { b.set(i, j, b.get(i,j) + color(0,0,0, 255 - (int)((1-ramp)*255)) ); ramp = ramp + 1/(float)(b.width * b.height); } a.blend(b, 0, 0, b.width, b.height, 80, 10, 450, 250, BLEND); image(a, 0, 0, 400, 300);

In Processing, it is possible to assign a color to a variable of type color by means of the function color(), and the model can be previously set with colorMode(). The functions red(), green(), blue(), hue(), saturation(), and brightness() allow to move from one model to the other.

	  colorMode(RGB);
	  color c1 = color(102, 30,29);
	  colorMode(HSB);
	  color c2 = color(hue(c1), saturation(c1), brightness(c1));
	  colorMode(RGB);
	  color c3 = color(red(c2), green(c2), blue(c2));
	  // the variables c1, c2, and c3 contain the coding of the same color
	

Representing Points and Vectors

In computer graphics, points and vectors are represented with the

Definition 1: homogeneous coordinates

quadruples of numbers, where the first triple is to be read in the X-Y-Z space, while the fourth number indicates a vector if it takes value 0, or a point if it takes value 1.

Translations

The function translate() moves an object in the image window. It takes two or three parameters, being the displacements along the directions x x, y y (and z z), respectively.

Rotations

In two dimensions, the function rotate() is used to rotate objects in the image window. This is obtained by (left) multiplying the coordinates of each pixel of the object by a rotation matrix. Rotations are always specified around the top left corner of the window ( 00 0 0 coordinate). Translations can be used to move the rotation axis to other points. Rotation angles are specified in radians. Recall that 2 ⁢π rad = 360 ˚ 2 rad 360 ˚ . For example, insert the rotation rotate(PI/3) before the second image() command in Example 1. In three dimensions, we can use elementary rotations around the coordinate axes rotateX(), rotateY(), e rotateZ().

Scale Transformations

The function scale() allows to expand or contract an object by multiplication of its point coordinates by a constant. When it is invoked with two or three parameters, different scalings can be applied to the three axes.

Every tool or language for media manipulation gives the opportunity to work with written words and with their fundamental visual elements: typographic characters.

The aspect of a type has two main components: font and size.

Processing has the class PFont and the methods loadFont() (to load a font and assign it to an object of the PFont class) and textFont() (to activate a font with a specific size). In order to load a font, this has to be pre-loaded into the directory data of the current sketch. The tool Create Font, accessible from the Tools menu in Processing, creates the bitmaps of the characters that the programmer intends to use. The file with the bitmaps is put in the data directory. After these preliminary operations, the font can be used to write some text, using the function text(). With this function, a string of characters can be put in the 2D or 3D space, possibly inserting it within a rectangular box. The alignment of characters in the box is governed by the function textAlign(). In the default configuration, the written text can be spatially transformed like any other object. The color of characters can be set with the usual fill(), like for any other graphic object.

Example 3: Overlapped text

Table 2

PFont fonte; /*The font have been previously created in the data folder*/ fonte = loadFont("HoeflerText-Black-48.vlw"); textFont(fonte, 12); fill(10, 20, 250, 80); textAlign(RIGHT); text("pippo pippo non lo sa", 10, 14, 35, 70); textFont(fonte, 24); fill(200, 0, 0, 100); text("ppnls", 25, 5, 50, 90);

Processing allows a tight control of the spatial occupation of characters and of the distance between contiguous characters (see Figure 3). The function textWidth() computes the horizontal extension of a character or a string. It can be used, together with the exact coordinates passed to text(), to control the kerning and the tracking between characters. The textSize() allows to redefine the size of characters. The textLeading() re-defines the distance in pixels between adjacent text lines. This distance is measured between the baselines of the strings of characters. Letters such as "p" or "q" extend below the baseline for a number of pixels that can be obtained with the textDescent(). Instead, the textAscent() gives back the maximum extension above the baseline (typically, the height of the letter "d").

Figure 3: Typeface metrics

Typeface metrics

Untill version beta 112, Processing gave the possibility to program several audio functionalities by means of some core primitives. In those older versions only two basic primitives are available to playback and load .wav files. In more recent versions, Processing delegate sound management and processing functionalities to external libraries. The most used libraries are Ess, Sonia, and Minim. Only the latter is included in the base installation of Processing. Ess and Sonia need an explicit installation process. Recently, a well-structured and documented Java library called Beads has also been introduced. It is well suited to the construction of audio-processing algorithms based on chains of base objects. As in the case of images, in order to process and playback sounds the source files have to be stored in the data folder of the current sketch. The library Sonia is the most complex one. With its functions, one can do sample playback, realtime Fourier-based spectral analysis, .wav file saving. In order to use the Sonia library, the programmer has to download the .zip file from Sonia. Once decompressed, the directory Sonia_?_? has to be copied into the directory Processing/libraries. Finally, the command import has to be inserted into the code by selecting it from the menu item Sketch / Import Library / Sonia_?_?.

In this section, we first use then analyze an application for the exploration of timbres, similar in conception to the Color Chooser of Figure 2, and here called Sound Chooser. For the moment, let us think about a sound timbre in analogy with color in images. For example, the various instruments of the orchestra have different and characterizing timbres (colors). Later on, we will define the physical and perceptual aspects of timbre more accurately. In the Sound Chooser applet, four sounds with different timbres can be played by clicking onto one of the marked radii. Each radius corresponds to a musical instrument (timbre/color). By changing position along the radius it is possible to hear how the brightness is changed. More precisely, as long as we proceed toward the centre, the sounds gets poorer.

Let us analyze the Processing code that implements the Sound Chooser in its salient aspects. The Sonia.start(this) command is necessary to activate the Sonia audio engine. The line Sample mySample1 declares a variable aimed at containing audio samples. Several methods can be applied to such variable. Among these, the play methods plays the sound sample back. In the draw() code section the graphic aspect of the applet is defined. Finally, by the function mouseReleased(), we detect when the mouse is released after being pressed, and where it has been released. At this point a sequenceo of if conditions finds what instrument/timbre has been selected according to the clicking point. Moreover, within the function mouseReleased() the function filtra(float[] DATAF, float[] DATA, float RO, float WC) is invoked. This function, which is implemented in the last segment of the code listing, performs a sound filtering. More precisely, it is a low-pass filter, thus a filter that leaves the low frequencies unaltered and reduces the intensity of the high frequencies. According to the radial position of the mouse click, the filtering effect changes, being more dramatic (that is the sound becomes darker) as the mouse is released closer and closer to the centre. A lighter realization of the Sound Chooser by means of the library Minim is proposed in problem Exercise 4. The problem Exercise 5 explores the recent library Beads.

                 
import pitaru.sonia_v2_9.*;

Sample mySample1, mySample2, mySample3, mySample4;
Sample mySample1F, mySample2F, mySample3F, mySample4F;

float[] data1, data2, data3, data4;
float[] data1F, data2F, data3F, data4F;

int sr = 11025;  // sampling rate


void setup()
{
  size(200, 200);
  colorMode(HSB, 360, height, height);
  Sonia.start(this);

  mySample1 = new Sample("flauto.aif");
    mySample2 = new Sample("oboe.wav");
      mySample3 = new Sample("tromba.wav");
        mySample4 = new Sample("violino.wav");

  mySample1F = new Sample("flauto.aif");
 // ... OMISSIS ...

  data1  = new float[mySample1.getNumFrames()]; 
	  // creates new arrays the length of the sample
	  // for the original sound
 // ... OMISSIS ...

  data1F  = new float[mySample1.getNumFrames()]; 
	  // creates new arrays the length of the sample
	  // for the filtered sound
// ... OMISSIS ...


  mySample1.read(data1);
// ... OMISSIS ...

}

void draw()
{
// ... OMISSIS ...
}

void mouseReleased()
{

float ro;
float roLin;
float wc;


  // FLAUTO
  if ((mouseX > 95) && (mouseX < 105)&& (mouseY > 50)&& (mouseY < 90)) {

    roLin = (mouseY-49.99)/41;
    ro = pow(roLin,.33);
    wc = 298*(TWO_PI/sr);
    filtra(data1F, data1, wc, ro);

    mySample1F.write(data1F);
    mySample1F.play();
  }
// ... OMISSIS ...

}

//filtra = new function
void filtra(float[] DATAF, float[] DATA, float WC, float RO) {

  float G;
  float RO2;
  RO2 = pow(RO, 2);
  G = (1-RO)*sqrt(1-2*RO*cos(2*WC)+RO2)*4; // (*4) is for having it louder
 
  for(int i = 3; i < DATA.length; i++){
    DATAF[i] = G*DATA[i]+2*RO*cos(WC)*DATAF[i-1]-RO2*DATAF[i-2];
	  //recursive filtering 
  }
}

// safely stop the Sonia engine upon shutdown.
public void stop(){
  Sonia.stop();
  super.stop();
  }