m12986 Graphic Composition in Processing 1.9 2005/08/14 07:19:40 GMT-5 2009/05/20 06:47:27.271 GMT-5 Davide Rocchesso Davide Rocchesso roc@iuav.it Davide Rocchesso Davide Rocchesso roc@iuav.it Davide Rocchesso Davide Rocchesso roc@iuav.it m12665 Graphic Composition in Processing Davide Rocchesso Davide Rocchesso roc@iuav.it Davide Rocchesso Davide Rocchesso roc@iuav.it 2D graphics 3D graphics curves Science and Technology Elements of 2D and 3D graphic composition (including curves) for the Processing language and environment. en

In Processing, we can arrange points, lines, surfaces, and volumes (objects in 0, 1, 2, and 3 dimensions, respectively) in a 3D space or, where this makes sense in the 2D space of the image window. The number of parameters of the object primitives will determine if these objects have to be positioned in the X-Y or in the X-Y-Z space.

The point() is the simplest of the graphic primitives. When invoked with two coordinate parameters, it positions a point in the X-Y space. When invoked with three coordinate parameters, it positions a point in the X-Y-Z space. A point, in geometric sense, does not have dimension, but it can be assigned an occupation in pixels and a color, by the functions strokeWeight() and stroke(), respectively. For example, the simple Processing sketch stroke(180,0,0); strokeWeight(10); point(60,60); draws a dot in the image window. A set of points can be grouped into a single object (cloud of points) by the delimiters beginShape(POINTS) and endShape(). Between them each point has to be specified with the vertex() function. Transformations of rotation, translation, and scaling do not apply to the inside of composite objects described with beginShape() and endShape(), but they can precede the definition of a composite object and apply to the whole.

The line() draws a line segment between two points in the plane or the 3D space, with width and color that can be set with strokeWeight() and stroke(), respectively. A set of segments can be defined, as we saw for points, by the delimiters beginShape() and endShape(). Between them, vertices are listed by calls to the function vertex(). Using the invocation beginShape(LINES) the vertices are taken in couples, each identifying a segment. With the argument-free invocation beginShape() the vertices, taken one after the other, define a polygonal line. With the closure endShape(CLOSE) the line is closed on itself by linking the first and last vertices. The color of such polygon can be set by using the fill() function or, conversely, left equal to the background color with the noFill(). The function curve(), when called with eight parameters, draws a curve on the image plane, with initial and final points determined, respectively, by the second and third couple of coordinates passed as arguments. The first and last couple of coordinates define two control points for the curve, which is an interpolating spline, thus passing for all four points. In Processing, however, only the curve segment between the intermediate points is visualized. Spline Piecewise-polynomial curve, with polynomials connected with continuity at the knotsSee Introduction to Splines and, for an introduction to the specific kind of splines (Catmull-Rom) used in Processing, the term spline in Wikipedia. In order to have an arbitrary number of control points we must use the function curveVertex() to specify each point in the block delimited by beginShape() and endShape(). As opposed to the curve(), in the bezier() function call the two control points specified by the four middle parameters are not points touched by the curve. They only serve to define the shape of the approximating Bézier curve, which has the following interesting properties: it is entirely contained in the convex hull defined by the extremal points and the control points; transformations of translation, rotation, or scaling, appied to the extremal and control points determine a similar transformation of the curve. As we can see by running the code stroke(255, 0, 0); line(93, 40, 10, 10); line(90, 90, 15, 80); stroke(0, 0, 0); noFill(); bezier(93, 40, 10, 10, 90, 90, 15, 80); the control points lay on the tangent passing by the extremal points. In order to have an arbitrary number of control points one must use the bezierVertex() to specify each point within a block delimited by beginShape() and endShape(). In this way, an arbitrarily involute curve can be traced in the 3D space. In 2D, the function bezierVertex() has six parameters that correspond to the coordinates of two control points and one anchor point. The first invocation of bezierVertex() has to be preceded by a call to vertex() which fixes the first anchor point of the curve. There are other methods that allow to read the coordinates or the slope of the tangent to an arbitrary point of a Bézier or spline curve. Such point can be specified by a parameter t that can go from 0 (first extreme) to 1 (second extreme). It is also possible to set the precision of approximating or interpolating curves in 3D. For details see the Processing reference manual . The Processing sketch in table shows the difference between the spline interpolating curve and the Bézier curve.See the term Bézier curve in Wikipedia. applet that compares the Bézier curve (red) and the interpolating spline (black) void setup() { c1x = 120; c1y = 110; c2x = 50; c2y = 70; background(200); stroke(0,0,0); size(200, 200); } int D1, D2; int X, Y; int c1x, c1y, c2x, c2y; void draw() { if (mousePressed == true) { X = mouseX; Y = mouseY; // selection of the point that is modified D1 = (X - c1x)*(X - c1x) + (Y - c1y)*(Y - c1y); D2 = (X - c2x)*(X - c2x) + (Y - c2y)*(Y - c2y); if (D1 < D2) { c1x = X; c1y = Y; } else { c2x = X; c2y = Y; } } background(200); stroke(0,0,0); strokeWeight(1); noFill(); beginShape(); curveVertex(10, 10); curveVertex(10, 10); curveVertex(c2x, c2y); curveVertex(c1x, c1y); curveVertex(190, 190); curveVertex(190, 190); endShape(); stroke(255,30,0); bezier(10,10,c2x,c2y,c1x,c1y,190,190); strokeWeight(4); point(c1x,c1y); point(c2x,c2y); }

Objects in two or three dimensions take a color that can be determined by the illumination, as explained in , or established by the method fill(), which also gives the possibility to set the degree of transparency. The triangle is the fundamental construction element for 3D graphics. In fact, by juxtaposition of triangles one can approximate any continuous surface. In Processing, however, the triangles are specified in 2D by the primitive triangle(), whose six parameters correspond to the coordinates of the vertices in the image window. Even though each triangle is defined in 2D, it can be rotated and translated in the 3D space, as it happens in the Processing sketch void setup(){ size(200, 200, P3D); fill(210, 20, 20); } float angle = 0; void draw(){ background(200); // clear image stroke(0,0,0); angle += 0.005; rotateX(angle); triangle(10,10,30,70,80,80); } A set of triangles can be defined, similarly to what we did for points and segments, by the delimiters beginShape() and endShape(). Between them, the vertices of the triangles are listed by calls to the function vertex(). By the invocation beginShape(TRIANGLES) the vertices are taken in triples, each defining a triangle, while the invocation beginShape(TRIANGLE_STRIP) takes the vertices one after the other to define a strip mad of triangular facets. If the vertex() has three arguments, the vertices are located in the 3D space and the corresponding triangles identify planar surfaces in space. Rectangles are defined, in Processing, by the function rect() of four parameters, where the first couple specifies, by default, the position in the the 2D plane of the top-left corner, and the third and fourth parameters specify the width and height, respectively. The meaning of the first couple of parameters can be changed with the function rectMode(): rectMode(CORNER) gives the default positioning; rectMode(CENTER) gives the positioning of the center of the rectangle at the specified point; with the rectMode(CORNERS) the four parameters are interpreted as the coordinates of the top-left and bottom-right vertices, respectively. A generic quadrilateral is defined by the coordinates of its four vertices, passed as parameters to the function quad(). It is important to notice that in 3D, while a triangle stays planar in any case, a quadruple of points does not necessarily lay on a plane. Viceversa, the quadrilaterals that are defined by 3D roto-translations of quadruples of 2D vertices, remain planar. Processing allows only eight parameters to be passed to quad(), thus forcing the definition of a quadrilateral as a quadruple of vertices in 2D. A set of quadrilaterals can be defined, similarly to what we saw for triangles, by the delimiters beginShape() and endShape(). Between them, vertices are listed by calls to the function vertex(). By using the invocation beginShape(QUADS) the vertices are taken in quadruples, each identifying a quadrilateral, while the invocation beginShape(QUAD_STRIP) takes the vertices one after the other to define a strip mad of quadrilateral facets. If the vertex() have three parameters, the planarity of the resulting faces is not ensured, and the resulting rendering can be misleading. For instance, by running the code size(200,200,P3D); lights(); beginShape(QUADS); vertex(20,31, 33); vertex(80, 40, 38); vertex(75, 88, 50); vertex(49, 85, 74); endShape(); we realize that the quadrilateral is rendered as the juxtaposition of two triangles belonging to different planes. A generic polygon is defined as a set of vertices, and it has a surface that can be colored. In Processing the vertices are listed within a couple beginShape(POLYGON); - endShape(); Actually, the polygon has to be intended in a generalized sense, as it is possible to use the bezierVertex() and curveVertex() to specify curved profiles. For instance, the reader may try to draw the moon: fill(246, 168, 20); beginShape(POLYGON); vertex(30, 20); bezierVertex(80, 10, 80, 75, 30, 75); bezierVertex(50, 70, 60, 25, 30, 20); endShape(); The function ellipse() draws an ellipse in the 2D plane. Its four parameters are interpreted, as in the case of rect(), as position followed by width and height. The position can be set in different ways according to the ellipseMode(), whose parameter can take values CORNER, CORNERS, CENTER, CENTER_RADIUS. The first couple of these possible values have to be referred to the rectangle that is circumscribed to the ellipse.

Processing offers a very limited repertoire of 3D-object primitives, essentially only balls and boxes. The function box() produces a cube when invoked with a single parameter (edge), a parallelepiped when invoked with three parameters (width, height, depth). The function sphere() produces, by an approximating polyhedron, a sphere whose radius is specified as a parameter.The function sphereDetail() can be used to specify the number of vertices of the polyhedron that approximates the ideal sphere.

A rotation or a translation can be imagined as operations that rotate or translate the Cartesian reference system. In other terms, after a rotate() or a translate() the following positioning operations of the objects will have a new coordinate system. When various objects are positioned in different ways in space, it is useful to keep trace of the coordinate systems that are set, one after the other. The data structure that is suited for containing such systems is the stack of transformations (matrix stack). With the function pushMatrix() the current coordinate system is put on top of the stack. On the other hand, to revert to the coordinate system before the last transformation, we have to call a popMatrix(). Actually, the stack contains the affine transformation matrices, according to what is dictated by OpenGL and described in . In this example two objects are positioned in the 3D space: a planar square and a cube. The first pushMatrix() saves the coordinate system onto the stack, then some transformations are applied, and finally the square is drawn. To go back to the previous coordinate system and apply new transformations to position the cube, we apply a popMatrix(). Essentially, the pushMatrix() and popMatrix() determine the scope for the geometric positioning of an object. float angle; void setup(){ size(100, 100, P3D); int angle = 0; } void draw(){ background(200); angle += 0.003; pushMatrix(); translate(25,50); rotateZ(angle); rotateY(angle); rectMode(CENTER); rect(0,0,20,20); popMatrix(); translate(75,50,-25); rotateX(angle); box(20); }

The Processing lighting model echoes the model used in OpenGL, that is the Phong shading. Such model is not physically justified, but it is particularly efficient. OpenGL considers as illuminated each polygon whose normal forms an acute angle with the direction of incoming light. This happens regardless of any masking objects. Therefore, shadows are not cast. OpenGL is said to use a local illumination model, since multiple reflections among surfaces and cast shadows are not automatically rendered. An environmental light is available, which is not coming from any particular direction, and whose color is specified by the parameters of the activation call ambientLight(). A directional light source is set with the directionalLight(), whose parameters specify color and incoming direction. The method lights() activates a default combination of gray ambient light and directional light, the latter also gray, coming from the frontal direction. It is possible to set a point light source in a given point of space by the call pointLight(). Finally, the method spotLight() activates a light beam which can be controlled in its color, position, direction, aperture, and concentration around the axis. The exponent e tunes the falloff around the axis: φ e When hitting a planar surface, a directional light produces reflected light along several directions, depending on the surface properties. In the case of perfectly-diffusive (or Lambertian) surface, the light radiates evenly from the surface along all directions, with an intensity that is larger for incident directions closer to the surface normal. Vice versa, if the surface is perfectly reflecting, light is only reflected along the direction that is specularly symmetric (about the surface normal) to the incident direction. In OpenGL, to have some flexibility in defining the illumination, each source has the three illumination components: ambient, diffuse, and specular. These three components are separately defined and interact with the respective components that define the surface properties of objects. The colors defined in the methods directionalLight(), pointLight(), and spotLight() define the Lambertian component of illumination. The lightSpecular() specifies the color of the component of incoming light that is subject to specular reflection. In Processing, the properties of surfaces are controlled by the methods ambient() (acting on the ambient component of incoming lights) and specular() (acting on the specular component). The function shininess() controls the concentration of the specularly-reflected beam, by a coefficient that acts similarly to the exponent of . The represented objects can also be considered as sources of light, and they can be assigned an emission light by the emmissive() call. However, the sources defined in this way do not illuminate the other objects on the scene. In OpenGL the point, spot, and ambient lights are attenuated with increasing distance, according to the model attenuation 1 a b d c d 2 The method ligthFalloff() allows to specify the parameters a , b , and c . Here, a cube and a QUAD_STRIP are positioned in space and illuminated by a rotating source. Moreover, a soft fixed light is set. Notice the absence of shadows and the apparent planarity of surfaces in the QUAD_STRIP. float r; float lightX, lightY, lightZ; void setup() { size(400, 400, P3D); r = 0; ambient(180, 90, 0); specular(0, 0, 240); lightSpecular(200, 200, 200); shininess(5); } void draw() { lightX = 100*sin(r/3) + width/2; lightY = 100*cos(r/3) + height/2; lightZ = 100*cos(r); background(0,0,0); noStroke(); ambientLight(153, 102, 0); lightSpecular(0, 100, 200); pointLight(100, 180, 180, lightX, lightY, lightZ); pushMatrix(); translate(lightX, lightY, lightZ); emissive(100, 180, 180); sphere(4); //Put a little sphere where the light is emissive(0,0,0); popMatrix(); pushMatrix(); translate(width/2, height/2, 0); rotateX(PI/4); rotateY(PI/4); box(100); popMatrix(); pushMatrix(); translate(width/4, height/2, 0); beginShape(QUAD_STRIP); vertex(10,13,8); vertex(13,90,13); vertex(65,76,44); vertex(95,106,44); vertex(97,20,70); vertex(109,70,80); endShape(); popMatrix(); r+=0.05; }

A perspective projection is defined by a center of projection and a plane of projection. The projector rays connect the points in the scene with the center of projection, thus highlighting the corresponding points in the plane of projection. The shows a section where the plane of projection produces a straight line whose abscissa is d , and the center of projection is in the origin.

By similarity of two triangles it is easy to realize that the point having ordinate y gets projected onto the plane in the point having ordinate y p y d z . In general, the projection of a point having homogeneous coordinates x y z 1 onto a plane orthogonal to the z axis and intersecting such axis in position d is obtained, in homogeneous coordinates, by multiplication with the matrix 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 d 0 . The projected point becomes x y z z d , which can be normalized by multiplication of all its element by d z . As a result, we obtain x d z y d z d 1

Parallel views are obtained by taking the center of projection back to infinity ( ). In this way, the projector rays are all parallel.

The orthographic projection produces a class of parallel views by casting projection rays orthogonal to the plane of projection. If such plane is positioned orthogonally to the z axis and passing by the origin, the projection matrix turns out to be particolarly simple: 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 . Among orthographic projections, the axonometric projections are based on the possibility to measure the object along three orthogonal axes, and on the orientation of the plane of projection with respect to these axes. In particular, in the isometric projection the projections of the axes form angles of 120 ° . The isometric projection has the property that equal segments on the three axes remain equal when they are projected onto the plane. In order to obtain the isometric projection of an object whose main axes are parallel to the coordinate axes, we can first rotate the object by 45 ° about the y axis, and then rotate by 1 2 35.264 ° about the x axis.

We can talk about oblique projection every time the projector rays are oblique (non-orthogonal) to the projection plane. In order to deviate the projector rays from the normal direction by the angles θ and φ we must use a projection matrix 1 0 θ 0 0 1 φ 0 0 0 0 0 0 0 0 1

As we have seen, Processing has a local illumination model, thus being impossible to cast shadows directly. However, by manipulating the affine transformation matrices we can cast shadows onto planes. The method is called flashing in the eye, thus meaning that the optical center of the scene is moved to the point where the light source is positioned, and then a perspective transformation is made, with a plane of projection that coincides with the plane where we want to cast the shadow on. The following program projects on the floor the shadow produced by a light source positioned on the y axis. The result is shown in

size(200, 200, P3D); float centro = 100; float yp = 70; //floor (plane of projection) distance from center float yl = 40; //height of light (center of projection) from center translate(centro, centro, 0); //center the world on the cube noFill(); box(yp*2); //draw of the room pushMatrix(); fill(250); noStroke(); translate(0, -yl, 0); // move the virtual light bulb higher sphere(4); //draw of the light bulb stroke(10); popMatrix(); pushMatrix(); //draw of the wireframe cube noFill(); rotateY(PI/4); rotateX(PI/3); box(20); popMatrix(); // SHADOW PROJECTION BY COMPOSITION // OF THREE TRANSFORMATIONS (the first one in // the code is the last one to be applied) translate(0, -yl, 0); // shift of the light source and the floor back // to their place (see the translation below) applyMatrix(1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1/(yp+yl), 0, 0); // projection on the floor // moved down by yl translate(0, yl, 0); // shift of the light source to center // and of the floor down by yl pushMatrix(); // draw of the cube that generate the shadow fill(120, 50); // by means of the above transformations noStroke(); rotateY(PI/4); rotateX(PI/3); box(20); popMatrix();

OpenGL is a set of functions that allow the programmer to access the graphic system. Technically speaking, it is an Application Programming Interface (API). Its main scope is the graphic rendering of a scene populated by 3D objects and lights, from a given viewpoint. As far as the programmer is concerned, OpenGL allows to describe geometric objects and some of their properties, and to decide how such objects have to be illuminated and seen. As far as the implementation is concerned, OpenGL is based on the graphic pipeline , made of modules as reported in . An excellent book on interactive graphics in OpenGL was written by Angel.

In Processing (and in OpenGL), the programmer specifies the objects by means of world coordinates (standard coordinates). The model-view matrix is the transformation matrix used to go from standard coordinates to a space associated with the camera. This allows to change the camera viewpoint and orientation dynamically. In OpenGL this is done with the function gluLookAt(), which is reproduced in Processing by the camera(). The first triple of parameters identifies the position, in world coordinates, of the optical center of the camera (eye point). The second triple of parameters identifies a point where the camera is looking at (center of the scene). The third triple of coordinates identifies a vector aimed at specifying the viewing vertical. For example, the program void setup() { size(100, 100, P3D); noFill(); frameRate(20); } void draw() { background(204); camera(70.0, 35.0, 120.0, 50.0, 50.0, 0.0, (float)mouseX /width, (float)mouseY /height, 0.0); translate(50, 50, 0); rotateX(-PI/6); rotateY(PI/3); box(45); } draws the wireframe of a cube and enables the dynamic rotation of the camera. The projection matrix is responsible for the projection on the viewing window, and this projection can be either parallel (orthographic) or perspective. The orthographic projection can be activated with the call ortho(). The perspective projection is the default one, but it can be explicitly activated with the call perspective(). Particular projections, such as the oblique ones, can be obtained by distortion of objects by application of the applyMatrix(). There is also the texture matrix, but textures are treated in another module. For each type of matrix, OpenGL keeps a stack, the current matrix being on top. The stack data structure allows to save the state (by the pushMatrix()) before performing new transformations, or to remove the current state and activate previous states (by the popMatrix()). This is reflected in the Processing operations described in . In OpenGL, the transformations are applied according to the sequence Push on the stack; Apply all desired transformations by multiplying by the stack-top matrix; Draw the object (affected by transformations); Pop from the stack. A viewport is a rectangular area of the display window. To go from the perspective projection plane to the viewport two steps are taken: (i) transformation into a 2 x 2 window centered in the origin ( normalized device coordinates ) (ii) mapping the normalized window onto the viewport. Using the normalized device coordinates, the clipping operation, that is the elimination of objects or parts of objects that are not visible through the window, becomes trivial. screenX(), screenY(), and screenZ() gives the X-Y coordinates produced by the viewport transformation and by the previous operators in the chain of . The viewing frustum is the solid angle that encompasses the perspective projection, as shown in . The objects (or their parts) belonging to the viewing volume are visualized, the remaining parts are subject to clipping. In Processing (and in OpenGL) the frustum can be defined by positioning the six planes that define it (frustum()), or by specification of the vertical angle, the, aspect ratio, and the positions of the front and back planes (perspective()). One may ask how the system removes the hidden faces, i.e., those faces that are masked by other faces in the viewing volume. OpenGL uses the z-buffer algorithm, which is supported by the graphic accelerators. The board memory stores a 2D memory area (the z-buffer) corresponding to the pixels of the viewing window, and containing depth values. Before a polygon gets projected on the viewing window the board checks if the pixels affected by such polygon have a depth value smaller than the polygon being drawn. If this is the case, it means that there is an object that masks the polygon.

Sophisticated geometric transformations are possible by direct manipulation of the projection and model-view matrices. This is possible, in Processing, starting from the unit matrix, loaded with resetMatrix(), and proceeding by matrix multiplies done with the applyMatrix(). Run and analyze the Processing code size(200, 200, P3D); println("Default matrix:"); printMatrix(); noFill(); ortho(-width/2, width/2, -height/2, height/2, -100, 100); translate(100, 100, 0); println("After translation:"); printMatrix(); rotateX(atan(1/sqrt(2))); println("After about-X rotation:"); printMatrix(); rotateY(PI/4); println("After about-Y rotation:"); printMatrix(); box(100); What is visualized and what it the kind of projection used? How do you interpret the matrices printed out on the console? Can one invert the order of rotations? The wireframe of a cube is visualized in isometric projection. The latter three matrices represent, one after the other, the three operations of translation (to center the cube to the window), rotation about the x axis, and rotation about the y axis. A sequence of two rotations correspond to the product of two rotation matrices, and the outcome is not order independent (product is not commutative). The product of two rotation matrices R x R y correspond to performing the rotation about y first, and then the rotation about x . Write a Processing program that performs the oblique projection of a cube. For example: size(200, 200, P3D); float theta = PI/6; float phi = PI/12; noFill(); ortho(-width/2, width/2, -height/2, height/2, -100, 100); translate(100, 100, 0); applyMatrix(1, 0, - tan(theta), 0, 0, 1, - tan(phi), 0, 0, 0, 0, 0, 0, 0, 0, 1); box(100); Visualize a cube that projects its shadow on the floor, assuming that the light source is at infinite distance (as it is the case, in practice, for the sun). We do it similarly to , but the transformation is orthographic: size(200, 200, P3D); noFill(); translate(100, 100, 0); pushMatrix(); rotateY(PI/4); rotateX(PI/3); box(30); popMatrix(); translate(0, 60, 0); //cast a shadow from infinity (sun) applyMatrix(1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1); fill(150); pushMatrix(); noStroke(); rotateY(PI/4); rotateX(PI/3); box(30); popMatrix();

Edward Angel Interactive Computer Graphics: A Top-Down Approach With OPENGL primer package-2nd Edition Prentice-Hall, Inc. 2001