Textures in Processing

Color Interpolation As seen in Graphic Composition in Processing, one can obtain surfaces as collections of polygons by means of the definition of a vertex within the couple beginShape() - endShape(). It is possible to assign a color to one or more vertices, in order to make the color variations continuous (gradient). For example, you can try to run the code

 
	  size(200,200,P3D);
          beginShape(TRIANGLE_STRIP);
          fill(240,   0, 0); vertex(20,31, 33);
          fill(240, 150, 0); vertex(80, 40, 38);
          fill(250, 250, 0); vertex(75, 88, 50);
                             vertex(49, 85, 74);
          endShape();

in order to obtain a continuous nuance from red to yellow in the strip of two triangles.

Bilinear Interpolation The graphical system performs an interpolation of color values assigned to the vertices. This type of bilinear interpolation is defined in the following way: For each polygon of the collection For each side of the polygon one assigns to each point on the segment the color obtained by means of linear interpolation of the colors of the vertices i e j that define the polygon: C ij α 1 α C i α C j A scan line scans the polygon (or, better, its projection on the image window) intersecting at each step two sides in two points l ed r whose colors have already been identified as C l e C r . In each point of the scan line the color is determined by linear interpolation C lr β 1 β C l β C r A significative example of interpolation of colors associated to the vertices of a cube can be found in examples of Processing, in the code RGB Cube.

Texture When modeling a complex scene by means of a composition of simple graphical elements one cannot go beyond a certain threshold of complexity. Let us think about the example of a modelization of a natural scene, where one has to represent each single vegetal element, including the grass of a meadow. It is unconceivable to do this manually. It would be possible to set and control the grass elements by means of some algorithms. This is an approach taken, for example, in rendering the hair and skin of characters of the most sophisticated animation movies (see for example, the Incredibles). Otherwise, especially in case of interactive graphics, one has to resort to using textures. In other words, one employs images that represent the visual texture of the surfaces and map them on the polygons that model the objects of the scene. In order to have a qualitative rendering of the surfaces it is necessary to limit the detail level to fragments not smaller than one pixel and, thus, the texture mapping is inserted in the rendering chain at the rastering level of the graphic primitives, i.e. where one passes from a 3D geometric description to the illumination of the pixels on the display. It is at this level that the removal of the hidden surfaces takes place, since we are interested only in the visible fragments. In Processing, a texture is defined within a block beginShape() - endShape() by means of the function texture() that has as unique parameter a variable of type PImage. The following calls to vertex() can contain, as last couple of parameters, the point of the texture corresponding to the vertex. In fact, each texture image is parameterized by means of two variables u and v , that can be referred directly to the line and column of a texel (pixel of a texture) or, alternatively, normalized between 0 and 1 , in such a way that one can ignore the dimension as well as the width and height of the texture itself. The meaning of the parameters u and v is established by the command textureMode() with parameter IMAGE or NORMALIZED. In the code that follows the image representing a broken glass is employed as texture and followed by a color interpolation and the default illumination. The shading of the surfaces, produced by means of the illumination and the colors, is modulated in a multiplicative way by the colors of the texture.


            size(400,400,P3D);
            PImage a = loadImage("vetro.jpg");
            lights();
            textureMode(NORMALIZED);
            beginShape(TRIANGLE_STRIP);
            texture(a);
            fill(240,   0, 0); vertex(40,61, 63, 0, 0);
            fill(240, 150, 0); vertex(340, 80, 76, 0, 1);
            fill(250, 250, 0); vertex(150, 176, 100, 1, 1);
                               vertex(110, 170, 180, 1, 0);
            endShape();

Texture mapping It is evident that the mapping operations from a texture image to an object surface, of arbitrary shape, implies some form of interpolation. Similarly to what happens for colors, only the vertices that delimit the surface are mapped onto exact points of the texture image. What happens for the internal points has to be established in some way. Actually, Processing and OpenGL behave according to what illustrated in , i.e. by bilinear interpolation: a first linear interpolation over each boundary segment is cascaded by a linear interpolation on a scan line. If u and v exceed the limits of the texture image, the system (Processing) can assume that this is repeated periodically and fix it to the values at the border. A problem that occurs is that a pixel on a display does not necessarly correspond exactly to a texel. One can map more than one texel on a pixel or, viceversa, a texel can be mapped on several pixels. The first case corresponds to a downsampling that, as seen in Sampling and Quantization, can produce aliasing. The effect of aliasing can be attenuated by means of low pass filtering of the texture image. The second case corresponds to upsampling, that in the frequency domain can be interpreted as increasing the distance between spectral images.

Texture Generation Textures are not necessarely imported from images, but they can also be generated in an algorithmic fashion. This is particularly recommended when one wants to generate regular or pseudo-random patterns. For example, the pattern of a chess-board can be generated by means of the code

  PImage textureImg =
loadImage("vetro.jpg"); // dummy image colorMode(RGB,1);

int biro = 0;
int bbiro = 0;
int scacco = 5;
for (int i=0; i<textureImg.width; i+=scacco) {
   bbiro = (bbiro + 1)%2; biro = bbiro;
   for (int j=0; j<textureImg.height; j+=scacco) {
     for (int r=0; r<scacco; r++)
       for (int s=0; s<scacco; s++)
        textureImg.set(i+r,j+s, color(biro));
    biro = (biro + 1)%2; 
  }
}
image(textureImg, 0, 0);

The use of the function random, combined with filters of various type, allows a wide flexibility in the production of textures. For example, the pattern represented in was obtained from a modification of the code generating the chess-board. In particular, we added the line scacco=floor(2+random(5)); within the outer for, and applied an averaging filter.

How could one modify the code in order to make the breaks in the glass more evident? It is sufficient to consider only a piece of the texture, with calls of the type

vertex(150, 176, 0.3,
	  0.3);

The excercise consists in modifying the code of the generator of the chess-board in in order to generate the texture . This exercise consists in running and analyzing the following code. Try then to vary the dimensions of the small squares and the filtering type.


          

size(200, 100, P3D);

PImage textureImg = loadImage("vetro.jpg"); // dummy image
colorMode(RGB,1);

int biro = 0;
int bbiro = 0;
int scacco = 5;
for (int i=0; i<textureImg.width; i+=scacco) {
   // scacco=floor(2+random(5));
   bbiro = (bbiro + 1)%2; biro = bbiro;
   for (int j=0; j<textureImg.height; j+=scacco) {
     for (int r=0; r<scacco; r++)
       for (int s=0; s<scacco; s++)
        textureImg.set(i+r,j+s, color(biro));
    biro = (biro + 1)%2; 
  }
}
image(textureImg, 0, 0);
textureMode(NORMALIZED);
beginShape(QUADS);
  texture(textureImg);
  vertex(20, 20, 0, 0);
  vertex(80, 25, 0, 0.5);
  vertex(90, 90, 0.5, 0.5);
  vertex(20, 80, 0.5, 0);
endShape();

// ------ filtering -------

PImage tImg = loadImage("vetro.jpg"); // dummy image
float val = 1.0/9.0;
float[][] kernel = { {val, val, val}, 
                     {val, val, val}, 
                     {val, val, val} }; 
int n2 = 1; 
int m2 = 1;
colorMode(RGB,255);
// Convolve the image 
for(int y=0; y<textureImg.height; y++) { 
  for(int x=0; x<textureImg.width/2; x++) { 
    float sum = 0; 
    for(int k=-n2; k<=n2; k++) { 
      for(int j=-m2; j<=m2; j++) { 
        // Reflect x-j to not exceed array boundary 
        int xp = x-j; 
        int yp = y-k; 
        if (xp < 0) { 
          xp = xp + textureImg.width; 
        } else if (x-j >= textureImg.width) { 
          xp = xp - textureImg.width; 
        } 
        // Reflect y-k to not exceed array boundary 
        if (yp < 0) { 
          yp = yp + textureImg.height; 
        } else if (yp >= textureImg.height) { 
          yp = yp - textureImg.height; 
        } 
        sum = sum + kernel[j+m2][k+n2] * red(textureImg.get(xp, yp)); 
      } 
    } 
    tImg.set(x,y, color(int(sum)));
  } 
} 
 
translate(100, 0);
beginShape(QUADS);
  texture(tImg);
  vertex(20, 20, 0, 0);
  vertex(80, 25, 0, 0.5);
  vertex(90, 90, 0.5, 0.5);
  vertex(20, 80, 0.5, 0);
endShape();