Customizing QuickTime MIDI

May 1, 2001 12:00 PM, By Peter Drescher

In 1991, after finishing my last tour with bluesman Joe Louis Walker and the BossTalkers, I sent résumés to every music-software developer I could find, hoping to get into a new line of work. Steve Hales had the foresight and generosity to offer this burned-out road dog and blues-piano player a job as studio director of his new company, Halestorm.

While at Halestorm, I provided content for SoundMusicSys, a software sample-playback synthesizer developed by Hales and his partner Jim Nitchals. They formed Halestorm to license the SoundMusicSys technology to game developers and other creators of multimedia products, and business was brisk. In fact, seed money to get the company growing came from the first big licensee, Apple Computer.

The idea behind SoundMusicSys was simple: use MIDI data to fire off instrument samples at the correct times and pitches. The files could be small, the code would be efficient, and the resulting music would sound as intended on any computer. Apple wanted to incorporate the technology into its QuickTime system extension to render Standard MIDI Files, and it licensed a GM sample library from Roland Corporation to provide the instrument sounds. Apple released its version of the software synthesizer, QuickTime Music Architecture (QTMA), in 1995. (To find out more about the SoundMusicSys family tree, see the sidebar “A Rose by Any Other Name.”)

QTMA has continued to evolve. With a bigger, better sample library and a host of new features and audio-compression algorithms, PCs and Macs can playback Standard MIDI Files with an audio quality rivaling that of hardware sound cards. Of course, QuickTime also has the added advantage of providing soundtracks that are locked to picture. That is a common requirement in the film and TV worlds, but it is often difficult to manage in the chaotic multimedia environment.

HIDDEN TREASURE

The greatest strength of QTMA's software synthesizer is that it can create custom instruments. General MIDI (GM) is, by definition, a finite palette of sounds, and multimedia musicians typically prefer to be limited only by their imaginations and engineering skills. With QTMA, however, you can include your own samples and trigger them at the appropriate times and pitches, which lets you write for instruments outside the GM specification. Vocals, guitar solos, drum loops, horn-section riffs, explosions, bird calls — whatever kind of audio you want — can be bundled with the MIDI data and transmitted over the Internet.

This feature has been hidden in QTMA for a long time, but the latest version of QuickTime Player Pro (4.1.2 as of this writing) provides access to the Instrument parameters. I put together a QuickTime movie that consisted of a Standard MIDI File enhanced with custom samples. Putting all the pieces together was a little tricky; it required careful planning and a lot of trial and error.

PUZZLE PIECES

I began with a Pro Tools session (mainly drum loops and horn riffs) created by house-music producers Agent K and Division 6. Next I cut up the digital-audio tracks into short sections starting on the beat. I converted the audio into 16-bit, 22 kHz System 7 format sounds with SoundApp, a versatile audio-conversion utility. That produced a series of small SFIL Mac files with the audio data stored in an SND resource. The Mac OS can play this native format when you simply double-click on the audio file's icon, so it's easily recognized and imported by QuickTime.

I then created a Standard MIDI File containing one track for each sample, with a middle-C trigger note at each place I wanted the sample to play. For example, the “beat1” sample is one bar long starting on the first beat, so I placed a C3 whole note in each bar, starting at bar 10, to keep the beat in sync (see Fig. 1). Other tracks played the additional beats or fired off the vocal and horn riffs at the correct times. I also included the GM piano, organ, and percussion tracks from the original Pro Tools session.

I imported the MIDI file into QuickTime Player Pro and saved it as a movie (MOV) file. (You can purchase QuickTime Player Pro for $29.99 at www.apple.com/quicktime.) When I selected the Get Info command from the Movie menu, a dialog box appeared, offering access to the Music Track section and its associated Instruments list. The tracks I laid out in my sequencer were displayed in the same order, so I knew which tracks went with which samples. I then dragged and dropped the appropriate System 7 files onto the correct tracks and hit Play (see Fig. 2). Voilà! (You can download the results at www.twittering.com/K6.)

LOCK, STOCK, AND BARREL

The Instruments list also provides access to the full range of Roland's GS sound set. This GM specification extension contains lots of great sounds, including nine drum kits, myriad synths, sound effects, and ethnic percussion, as well as some nice variations on the more standard instruments. I chose the patch for the MIDI bass line by double-clicking on the track and selecting Synth101 from the GS Piano and Chromatic Percussion category (see Fig. 3). You can also audition instrument sounds on the little piano keyboard at the bottom of the dialog box.

When you drop a sample onto a track, QuickTime Player Pro automatically creates an Instrument definition consisting of preset envelope, LFO, and effects data. The default volume envelope has a fast decay, which is unfortunate if you just want to fire off your sample. The sound fades to silence quickly, so I had to open the envelope editor by Option — double-clicking on the track name.

That caused a new Edit button to appear on the New Instrument for Part dialog box, which provided access to a basic set of Instrument parameters (see Fig. 4). I put the screen into expert mode by clicking on the lock icon; then I opened the Volume parameters (see Fig. 5). There, I found the culprit. The Sustain Level was set to 50 percent, and the Sustain Time was set to 5,000 ms. This meant that the sound would fade to half volume within five seconds. Well, I didn't want my sound to fade, so I set the Sustain Level to 100 percent and checked the Infinite Sustain box. I also used the Overall Volume slider to adjust the instruments' mix levels.

Expert mode offers a large number of parameters to play with, and many can radically alter an instrument's behavior and sound. For example, resetting the default Release Time to the maximum value effectively creates an instrument that plays the entire sample regardless of the triggering MIDI note's length. This is good for percussion, but don't use it on a looped sample — the loop will never stop playing.

Other parameters create a wide range of effects. The Pitch and Volume LFO parameters can be set to add delayed vibrato to a flute or tremolo to an electric piano. You can set stereo placement defaults, play around with the filter effects, or transpose the instrument to a different key. You can also modify the instruments' settings from the Roland GS bank to create wild new sounds.

TRY, TRY AGAIN

After I loaded my samples and set the envelopes, I simply played the movie to hear the MIDI file with the triggered samples. Of course, I didn't get it right on the first try; some samples were out of place, the bass wasn't loud enough, the hi-hat was too loud, and so forth. All I had to do was modify the MIDI data and reimport the file.

QuickTime Player Pro's editing limitations, however, made this process tedious: I had to reload the samples and reset the envelopes each time I imported a new version of the MIDI file. When I finally mixed everything correctly, I selected the Save As Self-Contained Movie command, which compacted the file to about 1.6 MB. That was a bit larger than I hoped it would be, so I downsampled the sound files to 8-bit, 22 kHz, which is the resolution the Roland sound set uses. After importing the audio and resetting the envelopes for the umpteenth time, I ended up with a file size of 770 KB for my three-minute song.

Whereas creating an entire song this way pushes the technology's limits — and my patience — a simpler approach can also prove to be quite effective. Say you want to use your own kick and snare sounds to enhance a mediocre-sounding GM rhythm track. You can easily break out the kick and snare trigger notes into separate tracks, import the MIDI and audio into QuickTime Player Pro, and then save it as a QuickTime movie. Adding even a couple of sound effects, vocal riffs, or guitar lines in that manner can liven up an otherwise ordinary MIDI composition.

LOOKING AHEAD

Despite the difficulties, developing this kind of content for QTMA has advantages over other Internet audio systems. More than 100 million machines have QuickTime, which means your music has a huge potential audience. You can also use the system to create high-quality, low-bandwidth audio-enhanced MIDI scores for QuickTime videos, and the audio will lock to picture just like with a real movie.

On the other hand, QuickTime has its limitations. QTMA does not support compression when working with individual samples, even though QuickTime audio tracks can be compressed with the excellent QDesign algorithm. If I could have used compressed samples within my MIDI tracks, the three-minute song could have been reduced to about 100 KB. There are other stumbling blocks as well. You can't create multisampled instruments; you can't select custom instruments in your tracks using MIDI Program Change messages; and you can't play your custom instruments from your sequencer (but you can gain access to the internal Roland bank through the Open Music System).

Fortunately, assistance is on the way. QuickTime 5 is already in beta release (the final version may be available by the time you read this article), and it includes a major upgrade and overhaul of the MIDI architecture. DLS and SoundFont banks will be supported, which will make the entire content-development process easier because of the many tools available for creating audio in those formats. QuickTime is such a practical Internet streaming-media technology that a widespread cross-platform installed base is guaranteed.

In any case, I will be watching my favorite software synthesizer's evolution. The newest incarnation appears to be emerging as an audio engine for cell phones and other wireless devices. Who knows where it will pop up next?

---

Peter Drescher is a composer, piano player, and owner of Twittering Machine, a project studio in San Francisco. He maintains his Web site at www.twittering.com .

A ROSE BY ANY OTHER NAME

During the mid-1990s the SoundMusicSys technology that gave rise to Apple's QuickTime Music Architecture (QTMA) underwent a number of incarnations. Steve Hales and Jim Nitchals started a new company called Igor's Software Laboratories to develop their technology for Internet applications. By that time, I was independent but still providing content for the system; I worked on sample libraries for Be and WebTV, which had already licensed versions of the synthesizer for their operating systems.

That's when Thomas Dolby got involved. He had a company called Headspace and ideas about implementing interactive audio on the Internet. He discovered the Igor synth while writing music for the WebTV box; he liked it so much, he bought the company. Renamed the Headspace Audio Engine, the system went through further changes and eventually emerged as Beatnik, the premiere interactive audio plug-in for “sonified” Web sites.

The fact that SoundMusicSys, QTMA, and Beatnik share a common ancestry explains why QTMA and Beatnik have so many similar features and parameter options. I had a definite sense of déjà vu the first time I opened the Instruments editor in QuickTime Player Pro. The parameters had familiar labels because they were based on the same set of data fields that I tweaked for years while designing instruments for Beatnik.