Web Audio Action
Sep 1, 2001 12:00 PM, By Spencer Critchley
advertisement
|
CURRENT NEWSSTAND ISSUERead the full Table of Contents for the issue on sale now! Click here Subscribe for only $1.84 an issue! Please tell us about yourself so we can better serve you. Click here to take our user survey. |
| |
 |
Life in the Fast Lane This collection of St.CroixÕs columns was assembled during the two years following his death of cancer in May 2006. Included are many of his most-read columns, as well as personal notes, drawings and photographs. Click for more books |
 Listen to these latest podcasts and more: |
|
eDeals Newsletter for Discounts on GearGet First Dibs on Hot Gear Discounts, Manufacturer Close-Outs and Job Opportunities when you sign up to receive eDeals E-newsletter, sent twice a month. Check out an issue get advertising info or subscribe |
|
There may be lots of audio activity on the Web (such as streaming and downloading), but not much of it is actually integrated into the Web. Just as sound is synched to picture in films, Internet audio must sync to onscreen elements before music and sound effects are truly integrated into a Web site's overall design.
In this article, I'll explore the ways to incorporate sound into a Web site and to make it seem like it belongs there. That means going past the limitations of HTML, which by nature is page oriented, to understand the role of helper applications, scripting, and programming.
PLAYING THE LINKS
Playing audio from a Web page depends on an audio-player helper application, such as a Netscape Navigator plug-in or an ActiveX control added to Microsoft Internet Explorer. The helper application might be the browser's default audio player or one of several third-party players, including RealPlayer, QuickTime Player, or Windows Media Player. The most basic way to play an audio file is through a simple hyperlink, such as this bit of HTML:
<html>
<head></head>
<body>
Here is a link to an audio file:
<a href=“myAudioFile.au”> Click here for a sound.</a>
</body>
</html>
At www.spencercritchley.com/em/webaudio, you can try that as well as the other examples. As you will see, when you click on a link such as that, the helper application is supposed to launch and play the file; see example 1 on my Web site for more information. The behavior of the audio file when it is played varies widely depending on which helper application is installed, though. From a design point of view, that behavior is not very integrated, because you don't have a lot of control over what happens on the user's machine.
EMBEDDING AUDIO
The first step beyond just linking from a page to an audio file is embedding the audio in the page. Actually, what you're embedding is an instance of a helper application, which plays the audio. (An instance of an application is created whenever an application is launched and loaded into memory. Frequently, you can have more than one instance of the same application running.)
From an interactivity standpoint, embedding has several advantages. The audio can reside within the page along with the text and images. Typically, the audio file and the rest of the page are loaded into memory so that once the page has finished loading, the audio is ready for playback without a noticeable delay. That responsiveness is crucial for successful interactive design. Depending on the helper application, you could also have a fair amount of control over what kind of playback interface, if any, is presented to the user.
Fig. 1 shows how an embedded audio file might appear on a Web page (see online example 2). Any browser installed or updated in the past few years (Navigator 3.0 and later or Internet Explorer 4.0 and later) should be able to play an embedded audio file if the audio is in a standard format (see the sidebar “Web Audio Formats”).
Embedding can also have some shortcomings. Because the file is typically loaded into memory, embedding is not a good choice if you work with large audio files or many small ones. Transferring a lot of audio data across the Internet slows down the page load and may take up more memory than is available on the user's computer. Loading the entire file isn't always necessary; many embeddable players allow streaming, so the audio file plays as it arrives from the Web server.
Without the effort of extra scripting, interactive control is typically limited to playing, stopping, cueing, and setting the volume. Managing layers of audio without extra scripting is also difficult. You can embed multiple audio files, but any attempts to synchronize them or control their volumes will likely end in frustration.
FORMAT FINESSE
Your audio format choice should be based on several factors, including its compatibility with various browser versions. The most widely adopted helper-application formats are Windows Media, RealAudio, QuickTime, Shockwave, and Flash.
Take into account each format's audio quality at various bandwidths; some formats do better than others at certain bandwidths. The type of playback interface and the depth of interactivity you want are also important considerations.
For many formats, encoding doesn't cost anything. Your audio-editing software might support Web encoding, or the Web site of the company that makes your player software may offer a free, downloadable content-creation tool (see the sidebar “Useful Resources”). The encoding process is usually well explained in each company's documentation, and wizards are often available to guide you through the process. Some content tools, such as RealSystem Producer, convert the audio and generate the HTML to embed the file in a Web page.
IN THE CODE
The next example shows the HTML for embedding an audio file; go to online example 2 for more information. I use QuickTime because its code is relatively straightforward; you can do the same kind of thing with the other helper applications. The
<p><embed src = “myAudioFile.mov” height = “16“ width = “240” pluginspage=“http://www.apple.com/quicktime”>
</embed>
I used the
The interface can have different dimensions as appropriate, or you can make it invisible, which might be an option for background music, spoken greetings, or ambient sound effects. You might also want the audio to start playing automatically. For QuickTime, that code looks like the following example (see online example 3).
<p><embed src = “myAudioFile.mov” height = “2” width = “2” hidden = “True” autoplay = “True” pluginspage=“http://www.apple.com/ quicktime”></embed>
The
Keep in mind that, from a design standpoint, it's risky to start audio automatically without giving the user a way to stop it or control its volume. You might therefore want to seriously consider making the interface visible and think carefully before using
If you do not want to write code by hand, visual Web-authoring tools such as Macromedia Dreamweaver can often write it for you. Dreamweaver makes it easy to embed Flash and Shockwave files, because they are Macromedia formats. With free extensions, downloadable from the Macromedia Exchange, Dreamweaver can automatically embed RealAudio, QuickTime, Beatnik, and other formats. Other Web-authoring programs, such as Adobe GoLive, offer similar capabilities.
SYNC ME
It's surprisingly tricky to synchronize events on a Web page, because browsers load and display page elements as they become available and do so at varying rates depending on the momentary state of the connection. Embedded helper applications make achieving sync possible, at least within a file if not across the page. Many player formats handle various media, so they can play sound and pictures together. For example, in most video-editing or animation programs, such as Macromedia Flash and Adobe LiveMotion, you can add a linear soundtrack; you can encode and embed the resulting file in an appropriate Web format. Online example 4 has a musical sting synched to a splash-screen animation.
In many cases, linear sync is not enough. True interactivity means responding to user gestures, such as rollovers and clicks. To achieve that kind of interactivity, you can go further with helper applications by one of two routes: from the outside in or the inside out. Outside in means controlling the behavior of a helper application with a scripting language, which can be a code-heavy approach. Inside out means using the extensive capabilities already built into the helper applications.
INSIDE OUT
Some helper applications are powerful enough that you can use them to create complete multimedia Web sites. In online example 5, I reproduced my home page using Flash, and I added rollover sounds to the menu.
For interactive audio purposes, Flash is the best mainstream choice because of its combination of good audio interactivity and near-universal adoption. That doesn't mean you should ignore the other formats; several have fascinating capabilities and may be better choices for individual projects. However, the Flash player has emerged as the VHS of interactive formats. I'll point out the highlights of using Flash with audio; study the manual for the details, because Flash is quite deep.
Flash takes a timeline approach to content creation. A virtual playback head progresses along a series of frames, playing any images, text, video, sounds, and actions (scripting instructions) it encounters (see Fig. 2). In a Flash movie, the location of the playback head (and the execution of actions) can be controlled by onscreen buttons to provide interactivity.
The simplest way to incorporate audio in Flash is by importing one or more audio files and specifying at which frames you want them to start playing. You can also adjust volume, fade in or out, set looping behavior, and add simple effects. When you export a Flash movie, you can specify the type and quality of audio-data compression (typically MP3). Flash also generates the HTML needed to embed the movie in a Web page.
The next step might be to associate sounds with buttons. A button can look like an actual button, or it can be any object on the screen that you define as a button; a tree might produce the sound of leaves rustling, for example. The button can be activated by a rollover or a click. In Fig. 3, I added a sound to a button that is triggered by a click.
Next, try importing short audio snippets that can be placed on multiple layers and played in different orders. In that way, it's possible to create evolving ambient sound environments or remix music in response to user actions. If you want to keep going, you can learn to program complex behaviors using ActionScript, Flash's scripting language, which is similar to JavaScript.
OUTSIDE IN
It's also possible to control helper applications from the outside in, using a scripting language, usually JavaScript. JavaScript integrates closely with HTML, and because it can control the browser and the helper application, it allows the addition of detailed interactivity to a Web page.
Becoming expert in JavaScript takes a lot of effort, especially if you don't have much programming experience. In the past, that effort was seldom worth the trouble for an audio person unless he or she was working with Beatnik, because the way JavaScript communicates with helper applications varies greatly between Navigator and Internet Explorer. Consequently, Web pages have to be authored and tested for both browsers.
Except in the case of Beatnik, JavaScript isn't able to communicate with multimedia helper applications under Internet Explorer on a Macintosh because Microsoft has not implemented the capability on that platform. Beatnik's work-around for the problem is its MusicObject JavaScript library, which hides browser differences and enables deep interactivity. In addition, Beatnik has an Actionset extension for Dreamweaver, which makes JavaScript sonification of Web pages easy.
Unfortunately, Beatnik doesn't have a Flash-size installed base. However, some clever people with expertise in Flash, JavaScript, and audio figured out how to leverage Flash to provide some of the power of Beatnik's approach while keeping the benefit of the huge Flash user population. Working through the interactive-audio community, Sonify.org developer Hayden Porter has released the free FlashSound JavaScript library. Using it still requires knowledge of Flash and JavaScript, but it's a great solution if you're willing to invest the effort. It doesn't always make sense to author entirely in Flash, and FlashSound lets you build a sonified site based on HTML. What's more, users who don't have Flash installed see the same site; it's just silent. I used FlashSound for the menu rollovers in online example 6.
PROGRAMMING POTENTIAL
As I said, scripting languages such as JavaScript allow the control of other applications, and programming languages enable the development of applications themselves. Java (similar to JavaScript in name only) is a programming language that has good Web-audio functionality. One advantage of working with Java is that it lets you deliver interactive audio that doesn't require plug-ins. A small Java applet can be embedded in a Web page, providing its own user interface and any set of behaviors the programmer decides to provide.
On the other hand, Java has until recently been limited to the 8-bit AU file format, so sound quality has not been great. That changed with the recent Java 2, which uses a version of the more flexible Beatnik Audio Engine, though most browsers have not yet implemented it. (For more about working with Java, see “Java Jive” in the November 2000 issue.)
Shockwave is a format that bridges scripting and programming. Using the Lingo scripting language in Macromedia's authoring program Director, you can create deeply interactive Shockwave applications, including games that incorporate several kinds of audio.
I covered a lot of information here, and if you haven't already created some Web content, the number of new concepts, techniques, and tools involved might intimidate you. I won't pretend that this stuff is always easy, but if you have experience with MIDI or digital audio, you're probably up to the challenge. As you know from those more familiar fields, the way to learn is step-by-step. Good luck!
Spencer Critchley provides creative consulting and production services for BeVocal and other companies. He's the cofounder, with Tony DeYoung, of Sonify.org.
WEB AUDIO FORMATS
Standard Web-audio formats include AU, MP3, MIDI, WAV, and AIFF. Popular proprietary formats include Flash, RealAudio, QuickTime, Windows Media, and Shockwave. Proprietary player applications can often play several standard formats, and in some cases, they can play proprietary formats from other companies.
The AU file extension refers to the Sun Microsystems audio format, which is the original Internet-audio format and therefore the most widely compatible. Because the format is limited to 8-bit samples, it's not the best sounding, though µ-Law or a-Law data compression improves its performance.
WAV and AIFF are also widely compatible and capable of the highest quality, but there's little reason to use them on the Web anymore because compressed formats such as MP3 sound good and are much smaller. MIDI files are tiny, but unless you're using a MIDI-enabled player such as QuickTime Player or Beatnik Player, MIDI files are played with whatever MIDI sound set is installed on the user's computer, with widely varying results. (For more about QuickTime's MIDI capabilities, see “Desktop Musician: Customizing QuickTime MIDI” in the May 2001 issue.)
Less-standard formats. Liquid Audio matches high-quality data compression with flexible security and is oriented more toward distribution than interactivity. Ogg Vorbis is an open-source, free-license alternative to MP3. Beatnik's Rich Music Format (RMF) is a hybrid of MIDI data and sample-based sound sets. (The QuickTime Musical Instruments set is a branch of the same family tree.) RMF yields very small files and high interactivity. (For more about Beatnik's capabilities, see “Desktop Musician: The Beatnik Player” in the December 2000 issue.) Sseyo's Koan also yields small, highly interactive files, though it's geared more toward generative composition.
RMF and Koan are examples of structured formats, in which descriptive information about the data remains visible to outside applications that can act on the data in real time. The difference between structured and unstructured formats (such as WAV and MP3) is like the difference between a fax and an e-mail. A fax is fixed on the page; you can't do much to change it beyond physically cutting up the paper. An e-mail remains infinitely editable. Some formats, such as QuickTime, can incorporate unstructured data and some structured data such as MIDI and animation sprites. Generally, structure is the trend. MPEG-4 aims to be the ultraformat, offering a structured approach to every form of media, all in the same file.
USEFUL RESOURCES
WEB SITES
http://java.sun.com/products/java-media/sound
This site offers information about the Java Sound API (application programming interface) for programming freestanding audio applications.
http://sonify.org
This nonprofit community site focuses on interactive audio for the Web. It provides many links to other sites.
www.apple.com/quicktime
This is the source for Apple's QuickTime Player and for QuickTime Player Pro, which adds important content-creation features.
www.beatnik.com
Founded by musician Thomas Dolby, Beatnik, Inc. provides the Beatnik Player, content tools, and the MusicObject JavaScript library.
www.builder.com
Replete with information on a variety of topics, this CNET site offers “solutions for site builders.”
www.macromedia.com
Macromedia offers a variety of content tools and players and is the home of Flash and Shockwave.
www.real.com
RealNetworks makes RealPlayer and the associated content tools, including RealSystem Producer.
www.sseyo.com
This is the home of the Koan format used by Brian Eno and others to create generative music.
www.webmonkey.com
Webmonkey: The Web Developer's Resource provides Web-authoring tools for beginners, builders, and masters.
www.webreference.com
This site is an excellent place to find information and tutorials about all aspects of Web authoring.
www.windowsmedia.com
The home of Microsoft's Windows Media Player, this site includes many links, downloads, content-creation tools, and information.
www.xiph.org/ogg/vorbis
This site features the license-free, open-source Ogg Vorbis MP3 alternative.
BOOKS
Designing Web Audio
By Josh Beggs and Dylan Thede This book concentrates on RealAudio, Flash, Shockwave, MP3, and Beatnik. It also looks at a few other formats.
Digital Audio with Java
By Craig A. Lindley This book explains the digital signal processing (DSP) theory behind audio effects and shows how to build two sample applications: a phrase sampler and a guitar-and-bass tuner. It assumes familiarity with Java.
Want to use this article? Click here for options!
© 2010 Penton Media, Inc.
Acceptable Use Policy blog comments powered by Disqus
