Picture Window

May 1, 2007 12:00 PM, By Mike Levine and Dennis Miller

Scoring to picture — the composition and placement of music to fit with video — has moved beyond the worlds of film, TV, and industrials to the mainstream. Thanks in large part to the stunning growth of YouTube and other Web-based video, the opportunities to put music to picture have never been greater.

Virtually every major digital audio sequencer offers video support of some kind. Import a video file and hit play in your audio app, and the video plays in sync with your sequence. But beyond that basic implementation, how does your application stack up as a scoring environment? To answer that question, we took a look at 11 of the most popular digital audio sequencers, covering both Windows and Mac. Though the recording, editing, and mixing capabilities that are their primary focus are all first rate, we were surprised at the differences in their abilities to work with video.

We'll start with an overview of the key video and scoring features that are common to most of the programs. Then we'll look at the highlights of each individual program.

We didn't cover multitrack audio editors because they don't use tempo as a timing reference, and they have little or no MIDI functionality. However, such programs are useful for postproduction mixing or for simply adding sound effects (see the sidebar “Mix to Pix”).

The programs covered in this story are Ableton Live 6, Apple GarageBand 3, Apple Logic Pro 7.2, Cakewalk Sonar 6 Producer Edition, Digidesign Pro Tools HD 7.3 and LE 7.3, Mackie Tracktion 3, MOTU Digital Performer 5.11, Sony Acid Pro 6, Steinberg Cubase 4, and Steinberg Nuendo 3. Many of these programs have lower-priced, reduced-feature versions. You may want to check manufacturer Web sites to be sure that those other versions have the features you want (see the sidebar “Manufacturer Contacts” online at www.emusician.com).

Start Me Up

If you sync up almost any piece of music against any video, some of the important moments of the video will likely be “hit” by important beats of the music, just by chance. But if you're doing a scoring job, chance won't cut it. You need to control which moments get those musical accents.

Taking into account the usable tempo range for the type of music you plan to write for a particular segment of the video (aka a cue), you typically want to come up with a tempo setting or tempo map that allows you to hit as many of the key points, within a frame or two, as possible with strong musical beats. Even if there aren't specific hit points that you need to accent, the tempo you choose should help the music “feel good” against the picture.

Besides tempo, another helpful variable when scoring is the start time of the music relative to the video. Changing how many “frames in” your music starts can impact whether a particular tempo will work. You typically finesse both the tempo and the start time to find the best combination. You can also experiment with the meter of the music, throwing in an occasional measure in a different meter (for instance, a 3/4 bar in a 4/4 composition) to change rhythmic emphasis.

Make Your Mark

When you're spotting your video (scanning through it and deciding what to emphasize), it's critical to be able to drop markers on the fly, and all of the programs covered here let you do that. Once you've got markers on the work space, the programs all make it easy (to varying degrees) to name and number them. With the exception of Live and GarageBand, they all let you lock those markers to absolute time values so that while you're experimenting with different tempos, preexisting markers stay at the same point relative to the picture.

Moving an audio clip to a specific SMPTE time is also a handy option. Sonar's Clip Properties menu, for example, lets you type in the exact location where you want your audio clip to start. Pro Tools' Spot mode makes it a snap to drop an audio clip at a specific SMPTE time. Logic Pro and Digital Performer have event lists that let you do the same thing. In Cubase and Nuendo, as you drag an event backward or forward in a track, its SMPTE position updates in real time, making placement easy.

Notable Additions

Some composers, especially those who work in the orchestral realm, prefer to work in standard notation when they write to picture. Several of the sequencers, including Logic Pro, Digital Performer, Cubase, and Nuendo, offer robust notation features. Sonar offers limited notation support.

Digital Performer (DP), Logic Pro, Cubase, and Sonar all show their markers in their notation windows. DP goes a step further with its QuickScribe Film Cues view, which shows your markers above the staves, with the beat they occur on clearly marked.

Snip Snip

When you're working on a scoring job, there are times when the ability to actually edit the video can be useful. Say you're given an entire reel from a project but have to write music only for a segment in the middle. If your sequencer lets you cut and paste the video, you can get rid of all but the portion you need to work on. Or, if your video has preroll before the actual material you're scoring, you can simplify your work flow by cutting it out so that the video starts right from frame 1 of the material you're scoring.

Cubase, Nuendo, Pro Tools, and Live offer basic cut, copy, and paste video editing. While these features are much more limited than what you'd get in a full-fledged video editor, they're helpful nonetheless.

Say Cheese

A video thumbnail track, which shows you the individual frames of a video file (depending on your zoom settings), is important for locating cue points. Having a dedicated, movable video window is also useful. All the programs except GarageBand (which has a stationary video window) let you drag your video window onto a second computer monitor to give yourself a separate display area.

The ability to preview your video on an external RGB monitor comes in handy too. Depending on the hardware you're using, this can enhance playback of the video, and it will also give you a more accurate image. In addition, it can help free up space for your sequencing windows on your primary monitor, though you can also accomplish that with a second video monitor. You'll want to output to an external device if, for instance, you are printing your final audio and video to a DV camcorder or other recording device.

Normally, your software passes the picture to the external monitor using a FireWire port, which means you'll need an external FireWire-to-RGB converter such as the Canopus TwinPact. All the programs except Tracktion, Live, and GarageBand can output video through FireWire.

In high-level TV and film work, surround sound is often called for. Most of the programs we looked at, including Acid, Pro Tools, Logic, Sonar, Digital Performer, Cubase, and Nuendo, are surround capable.

Ins and Outs

With all the video formats that exist, it's handy to have an audio app that handles different video-file formats. On the Mac, QuickTime (which uses a .mov or .qt file extension) is the most common type of digital video format. On the PC, QT, AVI, and Windows Media Video (WMV) are the most frequently found. But professionals at the high end and Web video users at the low end use numerous other formats. For example, many professionals require support for Avid OMF or Sony MXF formats (both of which are supported in Pro Tools HD), while work intended for the Web might require SWF (Flash), RMV (Real Media Video), MPG (MPEG-1 or MPEG-2), and other formats. (See the table “Video-Scoring Features Compared” for a list of formats supported by each program.)

Many of these programs can import QT and AVI, and some of the Windows programs support Windows Media Video. But beyond that, things vary dramatically. To handle some formats, you may need a dedicated video-editing program or format converter. RMV shows up only in Acid, which is also the only program to support Flash files.

Moreover, although all of these programs let you import various types of files, you can't assume that you can export a video (along with your newly recorded soundtrack), much less to a different format than you imported. For example, Tracktion and Live have no support for exporting video, and Sonar lets you import but not export MPG files. Convenient as it is to import QuickTime movies directly into your sequencer, remember that running a movie from your sequencer will add to the drain on your processor.

Fortunately, the ability to import a video file into your sequencer or audio application has obviated, for the most part, the sometimes dicey process of syncing to an external video deck. But if you do need external sync, all the programs but GarageBand sync to external timecode (with the appropriate peripheral hardware).

Click here for more of this story

	Fill in the form below and click Order Now! to get two years (26 issues) for just $23.97 - the regular price of one year. But HURRY - this offer won't last forever! (U.S. orders only please)
First Name: Last Name: Address: City: State: Zip: Email:
This data will be sent directly to Electronic Musician Magazine  and will not be used for any other purposes.