The EM Poll

All Together Now

Jan 1, 2007 12:00 PM, By Jim Aikin

           

In the late 1980s, sample-based synthesis was the premier technology that keyboardists used to produce a wide range of realistic sounds. Today, the synthesist has many more choices, including physical modeling for realistic sounds, additive synthesis, and older but still versatile options like modeled analog and FM. But sample playback remains a vital musical resource.

Last month I discussed how samplers produce sound (see “Square One: Get Real” in the December 2006 issue of EM; the article is also available online at www.emusician.com). To recap, a digital recording, or sample, is assigned to a particular key on the sampler's keyboard. Depending on which key is played, the original sample may have to be transposed up or down by some number of half steps in order to allow the performance of conventional musical passages. But when a sample is transposed too far, it no longer sounds natural. (What is considered to be “too far” depends on the type of sound that has been sampled.) Keyboardists use a technique called multisampling to get a full range of notes while avoiding the problems associated with sample transposition.

Teamwork

With multisampling, separate samples of the same instrument (or some other sound source) are recorded at different pitches. For instance, you might record an acoustic guitar playing every E, G#, and C over a range of three octaves. Each of these samples can then be assigned to a range of MIDI keys called a key zone (see Fig. 1). If you play a low E or F, you'll trigger one sample; if you play a G# or A, you'll trigger a different sample. Because each sample is confined to a narrow key zone, the sample needs to be transposed up or down by only a few half steps from its original pitch, so it won't suffer from the type of sonic degradation caused by being transposed too far. The keyboardist can play a musical part that covers a wide range without having to worry that transposition will make the sound unrealistic.

FIG. 1: Shown here are the key and Velocity zones in the Fazioli Grand preset in the -DirectWave sampler in Image-Line FL Studio 6. The root key (original pitch) of the selected sample is shown on the keyboard in red. This multisample is a 3-way Velocity cross-switch.

But this solution creates a new problem. Let's say you assign one guitar sample (the one recorded at a pitch of E) to the keys Eb through F# and another (recorded at G#) to the keys G through Bb. If you play the notes F# and G one after another, you'll trigger two different samples. If there's a noticeable difference in tone color or volume between the two samples, the two notes won't sound alike (see Web Clip 1).

The juncture in this example between F# and G is called a multisample split point. If you assign a multisample to the keyboard and then play a chromatic scale up or down the keyboard, you'll most likely hear the multisample split points. They may be subtle or obvious. If the mismatch is too glaring, the multisample won't sound realistic, especially when used to play a scale, even though each sample may have been recorded flawlessly.

Developing multisamples in which the multisample split points are unobtrusive is an exacting challenge. That is one reason why sampled-instrument libraries are expensive.

Trade-offs

The larger the key zones are, the more obvious the multisample split points will be. That is because at the split point, the sample in the lower zone will have been transposed up by a few half steps (or a lot of half steps) and the sample in the upper zone will have been transposed down by a similar amount. Because transposing a sample causes sonic changes, the guitar multisample mentioned above will sound too bright and brittle on the F# but too dull and thick on the G.

An obvious solution to the problem is to avoid transposing altogether by recording the source material (the guitar, electric piano, or whatever) at every half step. Each key zone then becomes one key wide. If each sample is recorded with reasonable care, the multisample split points will be less obvious because none of the samples will need to be transposed.

But that also creates new problems. For one thing, the more samples the developer of the sound library has to record, the higher the cost of the library. In addition, the more samples you assign to the keyboard, the more memory the sampler needs. In the early days of sampling, memory was expensive and assigning a separate sample to every key wasn't practical. Today, memory is much less expensive, but computer-based samplers still require large amounts of memory.

How much memory? If each 16-bit, 44.1 kHz mono sample is 5 seconds long, you'll need about 27 MB to assign a separate sample to each key of a 61-note keyboard. That's not much compared with the memory most newer computers have. The memory requirements of a modern sampler, however, can be considerably greater.

Timbre

When an acoustic instrument is played loudly, it isn't just the amplitude of the sound that changes; there are also changes in the timbre (tone color). Typically, a loud note on an instrument is brighter (has more high partials). To a limited extent, we can mimic the effect of playing an acoustic instrument more loudly or softly by sampling a note played loudly, and then using a lowpass filter to remove some of the high partials when a MIDI note with a low Velocity is received. On plucked and struck instruments such as guitar, piano, and percussion, softly played notes also die away sooner. We can mimic that effect by modulating the length of the envelope decay and release parameters using Velocity.

But that approach is imperfect at best. It works adequately with plucked and struck instruments but poorly with bowed and wind instruments. To create a more realistic multisample, record your source instrument playing various notes at various volume levels. You can then assign each sample to a different Velocity zone within the multisample. When the MIDI keyboard is played lightly (outputting a Velocity of, say, less than 64), a sample of a softly played note can be triggered. When the MIDI keyboard is played harder (with a Velocity of 65 or greater), a different sample, one of a loudly played note, can be triggered. You can use this technique, called Velocity cross-switching, to create a multisample that more closely approximates what a performance would sound like on the original instrument.

FIG. 2: In the sequence data used to -create Web Clip 2, the MIDI Velocity values increase smoothly, as seen in the lower pane. But when the Velocity reaches 40 and then 95, the loudness and brightness of the multisample suddenly increases in an unnatural way.

But now we've introduced a new transition point between samples. If the player is attempting to play at a medium dynamic level, some of the notes played might have Velocities between 60 and 64, and others might have Velocities between 65 and 70. Because the Velocities are all between 60 and 70, the notes will sound similar. But if the Velocity cross-switch point is between 64 and 65, they'll sound quite different (see Fig. 2), and the changes in tone color from note to note will be difficult to control from the keyboard (see Web Clip 2).

This problem occurs with many of the sample-playback keyboards and soft synths being built today; to some extent, it's unavoidable. Remember, we want the low-Velocity sample to have a different timbre than the high-Velocity sample. That's what Velocity cross-switching is for. But it doesn't always sound good when the multisample is used to play real music.

The solution is to record more samples at a variety of playing strengths and to assign them to relatively shallow Velocity zones. Many synthesizers today use 4-way Velocity cross-switching, which isn't sufficient; I'd prefer to see 8-way Velocity switching become the norm. But that would add considerably to the cost of the instrument. Creating a multisample that uses 8-way Velocity cross-switching is time-consuming, so purchasing it is expensive. And instead of just 27 MB of memory, you'll need 275 MB for a 61-note multisample that has 8-way cross-switching (for stereo samples, make that 550 MB). And that's starting to push the system requirements of the average computer.

In practice, instrument designers and sound-library developers try to find effective compromises. They try to balance development costs and memory requirements against the musical needs of their customers. Creating good multisamples requires science, but it requires art, too.

Jim Aikin writes about music technology for various magazines and Web sites. To get a preview of his new novel, visit him online at www.musicwords.net.