Teaching Media Processing in Processing
Summary: Skeleton of the presentation given in Copenhagen at the Sound in Interactive Media Workshop, may 12 2005
Note: You are viewing an old version of this document. The latest version is available here.
Processing
Processing is a
language and development environment. It is oriented towards
interaction design . The Processing project was
initiated and is currently pursued by Ben Fry and Casey Reas. In the course
Media Processing in Processing (MPP) , processing is
one of the main tools that we use to introduce the basics of
image, graphic, and sound processing. Processing is an
extension of Java and it supports many of Java structures with
a simplified syntax.
Processing can be used in three
Examples
| Programming Modes | ||
| Basic | - | Sequence of commands (graphic primitives) |
| Intermediate | - | Procedural programming |
| Advanced | - | Object-Oriented Programming |
Media Representation
Visual objects populate a 3D (or 2D)
space. Images, curves, surfaces, and typographic elements can
populate the space. Support for sound in Processing is very
limited. Guide to Media Representation
in Processing.
Graphic Composition
Processing provides primitives for drawing,
coloring, lighting, and shading visual elements in the 3D (and
2D) space. It can rely on OpenGL and hardware acceleration for
locally-run applications. Guide to
Graphic Composition in Processing.
Sound in Processing
Core support
Libraries
The
Sonia library (by Amit Pitaru) is based on JSyn and "provides advanced
audio capabilities such as multiple sample playback, realtime
sound synthesis, realtime FFT (frequency) analysis of the
microphone input, and writing .wav files from
samples".
JSyn is Phil
Burk's "Audio Software Synthesis API and Plugins for
Java". "JSyn is based on the traditional model
of unit generators which can be connected together to form
complex sounds". Phil Burk's
presentation at ICMC98.
Ess is Krister
Olsson's "sound library that allows sound data to be
loaded (AIFF, WAVE, AU, MP3), generated (sine, square,
triangle and sawtooth waves, white and pink noise),
manipulated (raw or via built-in effects), saved (AIFF, WAVE),
analyzed (FFT) or simply played back. Ess is built on
JavaSound and as such requires no additional plug-ins, just
Java 1.3 or better". It was announced in the discourse
section of Processing site on May 9th, 2005, 12:50pm.
Examples
Example 1: Audio-Visual Quantization
In
this applet,
the number of bits for representing the primary colors and the
audio samples can be changed by hitting the keyboard
arrows.
Example 2: Sound Chooser
A sound
chooser, similar in spirit to the color chooser of
image-processing applications.
Example 3: JSyn in Processing
JSyn can
be used directly in Processing, or side by side with Sonia. In
this example, enveloped sine
waves are generated by calling JSyn methods from within
Processing code.
Example 5: Audio Aliasing visualized
Visualizing (and hearing) aliasing through
a spectral representation, in Sonia. An audio-visual
wallpaper based on that.
Interaction for the Masses
Talking with the Universe
Talking with artefacts
The
Serial
library allows "for easily reading and writing data to
and from external machines".
More about this content: Metadata | Version History | Cite This Content
This work is licensed by Davide Rocchesso and Pietro Polotti. See the Creative Commons License about permission to reuse this material.
Last edited by Davide Rocchesso on Jul 5, 2005 7:35 am GMT-5.