Teaching Media Processing in Processing
Summary: Skeleton of the presentation given in Copenhagen at the Sound in Interactive Media Workshop, may 12 2005
-
Links
-
Supplemental links
Elaborazione di Media in Processing (italian version)Processing
Processing is a language and development environment. It is oriented towards interaction design . The Processing project was initiated and is currently pursued by Ben Fry and Casey Reas. In the course Media Processing in Processing (MPP) , processing is one of the main tools that we use to introduce the basics of image, graphic, and sound processing. Processing is an extension of Java and it supports many of Java structures with a simplified syntax.
Processing can be used in three
Programming Modes
- Basic - Sequence of commands (graphic primitives)
- Intermediate - Procedural programming
- Advanced - Object-Oriented Programming
Media Representation
Visual objects populate a 3D (or 2D) space. Images, curves, surfaces, and typographic elements can populate the space. Support for sound in Processing is very limited. Guide to Media Representation in Processing.
Graphic Composition
Processing provides primitives for drawing, coloring, lighting, and shading visual elements in the 3D (and 2D) space. It can rely on OpenGL and hardware acceleration for locally-run applications. Guide to Graphic Composition in Processing.
Sound in Processing
Libraries
The Sonia library (by Amit Pitaru) is based on JSyn and "provides advanced audio capabilities such as multiple sample playback, realtime sound synthesis, realtime FFT (frequency) analysis of the microphone input, and writing .wav files from samples".
JSyn is Phil Burk's "Audio Software Synthesis API and Plugins for Java". "JSyn is based on the traditional model of unit generators which can be connected together to form complex sounds". Phil Burk's presentation at ICMC98.
Ess is Krister Olsson's "sound library that allows sound data to be loaded (AIFF, WAVE, AU, MP3), generated (sine, square, triangle and sawtooth waves, white and pink noise), manipulated (raw or via built-in effects), saved (AIFF, WAVE), analyzed (FFT) or simply played back. Ess is built on JavaSound and as such requires no additional plug-ins, just Java 1.3 or better". It was announced in the discourse section of Processing site on May 9th, 2005, 12:50pm.
Examples
Example 1: Audio-Visual Quantization
In this applet, the number of bits for representing the primary colors and the audio samples can be changed by hitting the keyboard arrows.
Example 2: Sound Chooser
A sound chooser, similar in spirit to the color chooser of image-processing applications.
Example 3: JSyn in Processing
JSyn can be used directly in Processing, or side by side with Sonia. In this example, enveloped sine waves are generated by calling JSyn methods from within Processing code.
Example 4: Pan and Volume control in Ess
Example 5: Audio Aliasing visualized
Visualizing (and hearing) aliasing through a spectral representation, in Sonia. An audio-visual wallpaper based on that.
Interaction for the Masses
Talking with the Universe
The Net library allows the creation of clients and servers. This is especially useful to talk easily with Pure Data. See how a Processing client sends frequency and amplitude controls to a PD patch.
Talking with artefacts
The Serial library allows "for easily reading and writing data to and from external machines".
Comments, questions, feedback, criticisms?
Send feedback
- E-mail the authors of the module, Teaching Media Processing in Processing
Footer
More about this content: Metadata | Version History
This work is licensed by Davide Rocchesso and Pietro Polotti under a Creative Commons Attribution License (CC-BY 2.0).
Last edited by Davide Rocchesso on Jan 22, 2007 10:50 am US/Central.