Processing Processing is a language and development environment. It is oriented towards interaction design . The Processing project was initiated and is currently pursued by Ben Fry and Casey Reas. In the course Media Processing in Processing (MPP) , processing is one of the main tools that we use to introduce the basics of image, graphic, and sound processing. Processing is an extension of Java and it supports many of Java structures with a simplified syntax. Processing can be used in three Programming Modes Basic Sequence of commands (graphic primitives) Intermediate Procedural programming Advanced Object-Oriented Programming Examples

Media Representation Visual objects populate a 3D (or 2D) space. Images, curves, surfaces, and typographic elements can populate the space. Support for sound in Processing is very limited. Guide to Media Representation in Processing.

Graphic Composition Processing provides primitives for drawing, coloring, lighting, and shading visual elements in the 3D (and 2D) space. It can rely on OpenGL and hardware acceleration for locally-run applications. Guide to Graphic Composition in Processing.

Repertoire The complete reference manual The libraries

Sound in Processing

Core support The PSound class

Libraries The Sonia library (by Amit Pitaru) is based on JSyn and "provides advanced audio capabilities such as multiple sample playback, realtime sound synthesis, realtime FFT (frequency) analysis of the microphone input, and writing .wav files from samples". JSyn is Phil Burk's "Audio Software Synthesis API and Plugins for Java". "JSyn is based on the traditional model of unit generators which can be connected together to form complex sounds". Phil Burk's presentation at ICMC98. Ess is Krister Olsson's "sound library that allows sound data to be loaded (AIFF, WAVE, AU, MP3), generated (sine, square, triangle and sawtooth waves, white and pink noise), manipulated (raw or via built-in effects), saved (AIFF, WAVE), analyzed (FFT) or simply played back. Ess is built on JavaSound and as such requires no additional plug-ins, just Java 1.3 or better". It was announced in the discourse section of Processing site on May 9th, 2005, 12:50pm.

Examples Audio-Visual Quantization In this applet, the number of bits for representing the primary colors and the audio samples can be changed by hitting the keyboard arrows. Sound Chooser A sound chooser, similar in spirit to the color chooser of image-processing applications. JSyn in Processing JSyn can be used directly in Processing, or side by side with Sonia. In this example, enveloped sine waves are generated by calling JSyn methods from within Processing code. Pan and Volume control in Ess Example by Krister Olsson. Audio Aliasing visualized Visualizing (and hearing) aliasing through a spectral representation, in Sonia. An audio-visual wallpaper based on that.

Interaction for the Masses

Talking with the Universe The Net library allows the creation of clients and servers. This is especially useful to talk easily with Pure Data. See how a Processing client sends frequency and amplitude controls to a PD patch.

Talking with artefacts The Serial library allows "for easily reading and writing data to and from external machines".

Seeing the World The Video library allows "to display QuickTime video files and grab video data from a camera". Here we compute and display the histogram of camera-grabbed frames dynamically.