Say It With Pictures

Sep 1, 2008 12:00 PM, By Dennis Miller

           
 

EIGHT PROGRAMS THAT CONVERT IMAGES TO MUSIC

BONUS MATERIAL
The UPIC System

It's Not How Long You Make It

You can set the duration of the new file to any arbitrary length (with limits in some cases) in AudioPaint, Coagula, FL Studio, HighC, and MetaSynth, then once the program works its magic, all except AudioPaint, Coagula, and HighC allow you to manipulate the new audio file in various ways. As a full-featured sound-design “workstation,” MetaSynth offers an especially large number of effects and processes for that purpose, while Cameleon and Poseidon provide the professional processing and performance tools you'd expect from high-quality modern soft synths. Audition also has a wide range of options for working with your new audio file and is the only program in this group to provide a traditional multitrack-audio interface.

All of the programs allow you to export your new audio file, with 16-bit WAV being the most commonly supported format. And of the programs that can show a sonogram display of the converted image file, all but Poseidon let you export the display as a graphic image, perhaps for external processing and reimporting (FL Studio lets you copy the image and paste it into an external image editor).

Documentation varies widely among these programs, with Audition (which supplies the only printed documentation) and MetaSynth offering the best in the class. HighC provides some useful getting-started tutorials; AudioPaint, a short getting-started PDF; and Coagula, a thorough online help system (you can open the HLP file directly on the desktop). You'll also find users forums for a number of the programs, some of which are more active than others.

Keep in mind that in most cases, you won't automatically get musically useful results from any random image you choose to convert, regardless of which program you're using. I found that a bit of parameter tweaking, a fair amount of “postprocessing” (reverb, pitch-shifting, and the like), and, above all, a lot of trial and error were often needed for me to get something I could use. Also, the programs that allow you to draw gestures that will control musical parameters tend to be far more useful and, ultimately, satisfying. That's probably no surprise given the long tradition of using graphic symbols to specify musical parameters in Western music.

Adobe Audition 3 (Win, $349)

Audition is a full-featured stereo and multitrack audio editor, and given its family tree, which includes video and graphics apps Photoshop, Premiere Pro, and After Effects, it's no surprise that it boasts some innovative approaches to working with images. Audition's Spectral Frequency Display is where you work with spectra you generate both by analyzing an audio file and by importing graphic images, and the features it offers in both cases are identical.

By default, the program maps an imported image using increments of either 100 Hz per pixel (in linear mode) or 100 cents per pixel (in log mode) and lets you specify a different increment only if you already have an analysis open and a region selected. It has three additional options that determine how individual lines of pixels generate new frequency components: Pure Tones, which uses a 1-pixel-per-partial mapping and tends to produce rather static, harsh additive sounds; Random Noise Bands, which adds a bit of randomness to the partials' frequencies; and Track Frequency Spectrum, which is not clearly explained but produced the most interesting results in my tests regardless of the source files (see Web Clip 1).

One of the interesting options Audition provides is the ability to mask (filter) the spectrum of one file with that of another. For example, you could save the display of one sound's spectrum as a BMP file, then analyze a second sound and view its spectrum in the Spectral Frequency Display. You could then import the graphic image of the first sound's spectrum and use it to filter out some of the components of the second sound. In most cases, a good bit of trial and error is required, but the technique has a lot of potential for things like cross-synthesis and cloning the resonant qualities of one sound onto another.

Once you've got a graphic image converted into a spectral display, you can use Audition's powerful spectral-editing tools to modify the spectrum before resynthesizing it. For instance, you can isolate a small segment of the spectrum — all frequencies from 500 to 1,500 Hz between 3 and 6 seconds, for example — and then process just that region with reverb, EQ, or any other effect. And of course, with its robust multitrack mixing and editing options, Audition will make your newly generated audio file feel right at home when combined with any other audio files in your project.

Thomas Baudel's HighC 2.2 (Mac/Win, about $46 [MSRP])

FIG. 3: This is the score for Rob Arnold’s piece Study After Xenakis as shown in HighC. You can hear the resulting music in Web Clip 2.

HighC has strong ties to the original Xenakis UPIC system and in many ways is an enhancement to that system. The program's interface looks more like a traditional piano roll than the blank canvas found in most of the other programs, and many of the tools are optimized for drawing curves, lines, and gestures (see Fig. 3 and Web Clip 2). A typical session would start by using the Paint tool to draw some strokes on the canvas, picking a waveform and envelope to use for the sound file the image will generate, and then rendering the sound into audio. You can quickly build up echo, chorusing, or dense cluster effects by copying and pasting strokes, and easily adjust the total duration of the new sound regardless of the size of the original image you drew.