Skip to content Skip to navigation

Connexions

You are here: Home » Content » Teaching Media Processing in Processing

Navigation

Recently Viewed

This feature requires Javascript to be enabled.
 

Teaching Media Processing in Processing

Module by: Davide Rocchesso, Pietro Polotti. E-mail the authors

Summary: Skeleton of the presentation given in Copenhagen at the Sound in Interactive Media Workshop, may 12 2005

Processing

Processing is a language and development environment. It is oriented towards interaction design . The Processing project was initiated and is currently pursued by Ben Fry and Casey Reas. In the course Media Processing in Processing (MPP) , processing is one of the main tools that we use to introduce the basics of image, graphic, and sound processing. Processing is an extension of Java and it supports many of Java structures with a simplified syntax.

Processing can be used in three

Programming Modes

  • Basic: Sequence of commands (graphic primitives)
  • Intermediate: Procedural programming
  • Advanced: Object-Oriented Programming
Examples

Media Representation

Visual objects populate a 3D (or 2D) space. Images, curves, surfaces, and typographic elements can populate the space. Support for sound in Processing is very limited. Guide to Media Representation in Processing.

Graphic Composition

Processing provides primitives for drawing, coloring, lighting, and shading visual elements in the 3D (and 2D) space. It can rely on OpenGL and hardware acceleration for locally-run applications. Guide to Graphic Composition in Processing.

Repertoire

The complete reference manual

The libraries

Sound in Processing

Libraries

The Sonia library (by Amit Pitaru) is based on JSyn and "provides advanced audio capabilities such as multiple sample playback, realtime sound synthesis, realtime FFT (frequency) analysis of the microphone input, and writing .wav files from samples".

JSyn is Phil Burk's "Audio Software Synthesis API and Plugins for Java". "JSyn is based on the traditional model of unit generators which can be connected together to form complex sounds". Phil Burk's presentation at ICMC98.

Ess is Krister Olsson's "sound library that allows sound data to be loaded (AIFF, WAVE, AU, MP3), generated (sine, square, triangle and sawtooth waves, white and pink noise), manipulated (raw or via built-in effects), saved (AIFF, WAVE), analyzed (FFT) or simply played back. Ess is built on JavaSound and as such requires no additional plug-ins, just Java 1.3 or better". It was announced in the discourse section of Processing site on May 9th, 2005, 12:50pm.

Examples

Example 1: Audio-Visual Quantization

In this applet, the number of bits for representing the primary colors and the audio samples can be changed by hitting the keyboard arrows.

Example 2: Sound Chooser

A sound chooser, similar in spirit to the color chooser of image-processing applications.

Example 3: JSyn in Processing

JSyn can be used directly in Processing, or side by side with Sonia. In this example, enveloped sine waves are generated by calling JSyn methods from within Processing code.

Example 4: Pan and Volume control in Ess

Example by Krister Olsson.

Example 5: Audio Aliasing visualized

Visualizing (and hearing) aliasing through a spectral representation, in Sonia. An audio-visual wallpaper based on that.

Interaction for the Masses

Talking with the Universe

The Net library allows the creation of clients and servers. This is especially useful to talk easily with Pure Data. See how a Processing client sends frequency and amplitude controls to a PD patch.

Talking with artefacts

The Serial library allows "for easily reading and writing data to and from external machines".

Seeing the World

The Video library allows "to display QuickTime video files and grab video data from a camera". Here we compute and display the histogram of camera-grabbed frames dynamically.

Content actions

Download module as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Add module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks