Introduction

A basic understanding of human perception of sound is vital if you wish to design music synthesis algorithms to achieve your goals. Human hearing and other senses operate quite well in a relative sense. That is, people perceive properties of sound such as pitch and intensity and make relative comparisons. Moreover, people make these comparisons over an enormous dynamic range: they can listen to two people whispering in a quiet auditorium and determine which person is whispering the loudest. During a rock concert in the same auditorium, attendees can determine which vocalist is singing the loudest. However, once the rock concert is in progress, they can no longer hear someone whispering! Senses can adapt to a wide range of conditions, but can make relative comparisons only over a fairly narrow range.

In this module you will learn about pitch and frequency, intensity and amplitude, harmonics and overtones, and tuning systems. The treatment of these concepts is oriented to creating music synthesis algorithms. Connexions offers many excellent modules authored by Catherine Schmidt-Jones that treat these concepts in a music theory context, and some of these documents are referenced in the discussion below. For example, Acoustics for Music Theory describes acoustics in a musical setting, and is a good refresher on audio signals.

Pitch and Frequency

Pitch is the human perception of frequency. Often the terms are used interchangeably, but they are actually distinct concepts. Musicians normally refer to the pitch of a signal rather than its frequency; see Pitch: Sharp, Flat, and Natural Notes and The Circle of Fifths.

Perception of frequency is logarithmic in nature. For example, a change in frequency from 400 Hz to 600 Hz will not sound the same as a change from 200 Hz to 400 Hz, even though the difference between each of these frequency pairs is 200 Hz. Instead, you perceive changes in pitch based on the ratio of the two frequencies; in the previous example, the ratios are 1.5 and 2.0, respectively, and the latter pitch pair would sound like a greater change in frequency. Musical Intervals, Frequency, and Ratio offers additional insights.

Often it is desirable to synthesize an audio signal so that its perceived pitch follows a specific trajectory. For example, suppose that the pitch should begin at a low frequency, gradually increase to a high frequency, and then gradually decrease back to the original. Furthermore, suppose that you should perceive a uniform rate of change in the frequency.

The screencast video of Figure 1 illustrates two different approaches to this problem, and demonstrates the perceptual effects that result from treating pitch perception as linear instead of logarithmic.

Figure 1: [video] Two approaches to the design of a frequency trajectory: one linear, and the other logarithmic

Intensity and Amplitude

Perception of sound intensity also logarithmic. When you judge one sound to be twice as loud as another, you actually perceive the ratio of the two sound intensities. For example, consider the case of two people talking with one another. You may decide that one person talks twice as loud as the other, and then measure the acoustic power emanating from each person; call these two measurements T 1 T 1 and T 2 T 2 . Next, suppose that you are near an airport runway, and decide that the engine noise of one aircraft is twice the intensity of another aircraft (you also measure these intensities as A 1 A 1 and A 2 A 2 ). In terms of acoustic intensity, the difference between the talkers T 2 − T 1 T 2 − T 1 is negligible compared to the enormous difference in acoustic intensity A 2 − A 1 A 2 − A 1 . However, the ratios T 2 / T 1 T 2 / T 1 and A 2 / A 1 A 2 / A 1 would be identical.

The decibel (abbreviated dB) is normally used to describe ratios of acoustic intensity. The decibel is defined in Equation 1:

where I 1 I 1 and I 2 I 2 represent two acoustic intensities to be compared, and R dB R dB denotes the ratio of the two intensities.

Acoustic intensity measures power per unit area, with a unit of watts per square meter. The operative word here is power. When designing or manipulating audio signals, you normally think in terms of amplitude, however. The power of a signal is proportional to the square of its amplitude. Therefore, when considering the ratios of two amplitudes A 1 A 1 and A 2 A 2 , the ratio in decibels is defined as in Equation 2:

Can you explain why "10" becomes "20"? Recall that log⁡( a b )=blog⁡(a) log⁡( a b )=blog⁡(a) .

Often it is desirable to synthesize an audio signal so that its perceived intensity will follow a specific trajectory. For example, suppose that the intensity should begin at silence, gradually increase to a maximum value, and then gradually decrease back to silence. Furthermore, suppose that you should perceive a uniform rate of change in intensity.

The screencast video of Figure 2 illustrates two different approaches to this problem, and demonstrates the perceptual effects that result from treating intensity perception as linear instead of logarithmic.

Figure 2: [video] Two approaches to the design of an intensity trajectory: one linear, and the other logarithmic

Harmonics and Overtones

Musical instruments produce sound composed of a fundamental frequency and harmonics or overtones. The relative strength and number of harmonics produced by an instrument is called timbre, a property that allows the listener to distinguish between a violin, an oboe, and a trumpet that all sound the same pitch. See Timbre: The Color of Music for further discussion.

You perhaps have studied the concept of Fourier series, which states that any periodic signal can be expressed as a sum of sinusoids, where each sinusoid is an exact integer multiple of the fundamental frequency; refer to Equation 3:

where f 0 f 0 is the fundamental frequency (in Hz), n n denotes the harmonic number, and a 0 a 0 is the DC (constant) offset.

When an instrument produces overtones whose frequencies are essentially integer multiples of the fundamental, you do not perceive all of the overtones as distinct frequencies. Instead, you perceive a single tone; the harmonics fuse together into a single sound. When the overtones follow some other arrangement, you perceive multiple tones. Consider the screencast video in Figure 3 which explains why physical instruments tend to produce overtones at approximately integer multiples of a fundamental frequency.

Figure 3: [video] Why musical instruments produce overtones at approximately integer multiples of a fundamental frequency

Musicians broadly categorize combinations of tones as either harmonious (also called consonant) or inharmonious (also called dissonant). Harmonious combinations seem to "fit well" together, while inharmonious combinations can sound "rough" and exhibit beating. The screencast video in Figure 4 demonstrates these concepts using sinusoidal tones played by a synthesizer.

Figure 4: [video] Illustration of harmonious and inharmonious sounds using sinusoidal tones

Please refer to the documents Consonance and Dissonance and Harmony for more information.

Tuning Systems

A tuning system defines a relatively small number of pitches that can be combined into a wide variety of harmonic combinations; see Tuning Systems for an excellent treatment of this subject.

The vast majority of Western music is based on the tuning system called equal temperament in which the octave interval (a 2:1 ratio in frequency) is equally subdivided into 12 subintervals called semitones.

Consider the 88-key piano keyboard below. Each adjacent pair of keys is one semitone apart (you perhaps are more familiar with the equivalent term half step). Select some pitches and octave numbers and view the corresponding frequency. In particular, try pitches that are an octave apart (e.g., A3, A4, and A5) and note how the frequency doubles as you go towards the higher-frequency side of the keyboard. Also try some single semitone intervals like A0 and A#0, and A7 and A#7.

The frequency values themselves may seem rather mysterious. For example, "middle C" (C4) is 261.6 Hz. Why "261.6" exactly? Would "262" work just as well? Humans can actually perceive differences in the sub-Hz range, so 0.6 Hz is actually noticeable. Fortunately an elegantly simple equation exists to calculate any frequency you like. The screencast video of Figure 5 explains how to derive this equation that you can use in your own music synthesis algorithms. Watch the video, then try the exercises to confirm that you understand how to use the equation.

Figure 5: [video] Derivation of equation to convert semitones to frequencies