The EM Poll

This is not a scientific poll but a tabulation of readers responses and is purely just for fun!

See Past Poll Results

XML for Music

Dec 1, 2003 12:00 PM, By Darin Stewart

Desktop musicians have an embarrassment of riches when it comes to software tools for making music. The proliferation of applications and plug-ins should allow you to piece together a suite of tools tailor-made for a variety of creative and commercial projects. Reality, however, doesn't always work out quite so neatly, even with the great power and versatility of MIDI.

The problem is that MIDI was designed to be a mechanism for sending and receiving musical event information, especially the explicit, quantifiable attributes of a performance. It is less than optimal for the richer set of attributes that many applications, such as notation and analysis programs, depend on. As a result, it's often difficult to get software from different vendors to play together nicely.

For example, a musician might select a best-of-breed sequencer to capture the nuances of a performance from a MIDI-equipped instrument. She wants to expand the arrangement, so she imports the MIDI file into her favorite notation program and manually fleshes out the score. So far, so good.

Next, she needs to send the file to her writing partner, but he has a different notation program. Because MIDI doesn't have any sense of beams, stems, ties, or other staples of music notation, it can't be used to move the score from one application to another. An adapter to convert the proprietary format of the original program to that of the second program must be used. Each time the collaborators exchange the file, this translation occurs and most likely introduces errors with each pass.

The day of the session arrives, and the final score must again be converted for the studio's software, and some parts must be translated back into MIDI for the sequencers. When the session players arrive, several are toting digital music displays. With a sigh, our songwriter fires up a Web browser and starts hunting for a plug-in that will let her properly communicate with everyone. The moral of the story? Without a sufficiently rich standard for representing music, each application needs a specific translator for every other program that it wants to talk to.

Several attempts to address this unfortunate situation have been made, chief among which are the Notation Interchange File Format (NIFF) and the Standard Music Description Language (SMDL). NIFF was completed in 1995 and was designed specifically as a music-notation interchange standard. Even though it has been adopted by several notation and music-scanning programs, its graphical approach to representing music information limits its usefulness for performance applications. While SMDL has reached the status of official standard (ISO/IEC Draft International Standard 10743), it suffers from the opposite extreme of trying to be all things to all people.

The ISO standard defines SMDL as “an architecture for the representation of music information, either alone, or in conjunction with text, graphics, or other information needed for publishing or business purposes. Multimedia time-sequencing information is also supported.” In short, SMDL set out to become an all-encompassing standard for representing all music-related information. While SMDL has largely succeeded in its ambitious goal, its approach is so abstract, unwieldy, and even obscure (a note is called a cantus event) that a commercial implementation has yet to emerge after 15 years of availability.

MARKUP FOR MUSIC

This problem is not unique to music applications. Demand for interoperability among products from competing vendors has been the bane of the computer industry since its birth. Fortunately, a solution has arisen from the world of Internet applications: the Extensible Markup Language (XML). XML has emerged as the accepted standard for data representation and interchange among commercial software packages. Readers of EM are probably familiar with XML as it relates to Web pages (see “Mark My Words” in the November 2001 issue). But readers may not realize that XML has capabilities that reach far beyond just souping up home pages.

XML is a markup language for defining markup languages. It allows you to define a grammar for describing the data associated with any group or class of computer applications. When two programs want to share data, they exchange a highly structured text document that conforms to the syntax rules of XML and the grammatical rules that the two programs have agreed upon.

XML requires you to define your grammar in the form of another XML document called a schema or a Document Type Definition (DTD). By following XML's rules for creating a valid DTD, you don't need to worry about how your grammar will be interpreted by a program. There are standard tools for that, and that frees you to focus on how to organize and represent the data you want to share. This balance of flexibility and consistency is the core strength of XML as a vehicle for data interchange.

XML is particularly well suited to the representation of music. First, there are many ways to represent a musical structure ranging from a single, indivisible monolith to a finely grained, ordered collection of notes and inflections. Most music has internal structure that falls somewhere between those two extremes. The different levels of structure imply hierarchy, and that is how XML structures information. In addition, XML lets you identify and isolate various types of musical components.

This object-oriented approach to music representation allows you to define how a certain operation behaves depending on the nature of the musical element it is applied to. It also ensures that a given type of musical element is treated consistently regardless of where it occurs in a piece of music or even across completely unrelated works. Most importantly, XML enables you to move beyond the structure of music notation to represent its semantics.

FIG. 1: The above shows a one-note song encoded in MusicXML.

To demonstrate this concept, look at the example in Fig. 1. It shows a one-note song in 4/4 time represented in MusicXML. MusicXML is one current implementation of XML for music produced by Recordare (reh-cor-DAH-ray). The first four lines of the example consist of header information that shows the file is an XML document. The lines also show what the XML version number is and which character set is used. Most importantly, they specify which DTD or grammar should be used to validate the document called “score-partwise” and where it can be found (www.musicxml.org/dtds/partwise.dtd).

The rest of an XML document is composed primarily of elements indicated by angle-bracketed tags such as and . Similar to HTML, this open- and close-tag pair indicates the beginning and end of a particular element — in this case, the part list. It also implies that elements may be nested within one another according to rules that are spelled out in the DTD.

In the MusicXML example, a is made up of that consist of a , and so on. An element defines the information that is needed to interpret the rest of the marked-up song. That includes things such as key and time signatures and the base time unit as divisions of a quarter note. This is where semantic markup and the strength of XML really come into play.

As mentioned, MIDI has no concept of a note; it just has Note On and Note Off events. Rests don't exist but are inferred from the empty spaces between Note Off and Note On. This leads to ambiguities that can destroy the fidelity of a notation program. Likewise, MIDI makes no distinction between enharmonic equivalents; the F-sharp and G-flat above middle C are both MIDI Note Number 66. That leaves a notation program to make its best guess, often with mixed results. XML solves this problem by letting you explicitly indicate what you are encoding. A note (and all of its components) is described as follows:

<note>
<pitch>
<step>C</step>
<octave>4</octave>
</pitch>
<duration>4</duration>
<type>whole</type>
</note>

This fragment defines a single note in the XML-encoded song. The note has a pitch that is defined by a of C in the current key signature (defined elsewhere in the example) and an of 4 indicating that the note is middle C. The note value may be altered by the key signature or by an tag indicating a sharp or a flat. Every note also has a duration that is based on divisions of a quarter note. The previous example, in 4/4 time, does not divide a quarter note at all so our whole note has a of 4 (4 quarter notes). Just to make sure that it's notated correctly we give it a of “whole.”

Including separate tags for duration and note type may seem overly verbose. At a surface level, the entire example may seem bloated, but that's part of the price paid for flexibility. For example, the way a song is notated may be very different from how it is performed. You may want to score a series of notes as straight eighth notes but want to give the notes more of a laid-back feel when performed. You could accomplish that by tagging the notes with a of “eighth” and then playing with the value.

The potential verbosity of an XML score is alleviated somewhat by the use of attributes. Attributes can modify or extend the nature of any given element. For example, a well-placed attribute can help you know exactly where you are in a given work. The tag <measure number=“1”>, where “number” is the attribute, indicates that the chunk of XML we are about to look at describes a measure, and it happens to be the first one in the song.

This emphasizes one of the goals of XML encoding and how it bypasses the pitfalls of some past efforts: it is intended to be readable. The comprehensive tagging makes an XML-encoded score much more comprehensible to the average person than a shorter document crammed with acronyms and cryptic indicators. If disk space is an issue, compression can reduce XML files to the size of MIDI files.

Another benefit of XML is that it's free — something that might well ensure its adoption. All of the currently defined music DTDs and schemas are available under royalty-free licenses. As a result, they can be incorporated into existing and emerging products without paying a dime to their authors, and that's very appealing to software developers.

WHO'S DOING WHAT?

For XML to be truly useful as a music representation and an interchange mechanism, a standard must emerge. The ability to specify which DTD a program wants to see would seem to make that unnecessary, but in fact it makes the situation worse. Defining music-related DTDs seems to be the rage among both music and markup enthusiasts. A quick search of the Internet identifies more than 20 music-related DTDs, along with assorted schemas. Most are purely academic exercises or hobbyist tinkering, but a few are gaining momentum toward commercial viability.

Without significant consolidation of the field, each application must map its own capabilities to each markup scheme (in the form of a DTD) that comes its way. That eliminates most of the benefits of adopting XML in the first place. Fortunately, some leaders have started to emerge; the most ambitious is the Music Markup Language (MML) being developed by Jacques Steyn (www.musicmarkup.info).

MML is more of a comprehensive framework for music markup than a single DTD. Steyn has identified 12 modules for structuring and representing music; the Time and Frequency modules form the essential core. All music objects and events are represented within these two modules, with additional modules being layered in as necessary. According to Steyn, MML is “the only XML-based attempt to describe a very large scope of the domain of music.”

While this approach runs the risk of turning MML into the next SMDL — too complex to be of any practical use — Steyn seems to be adopting more of a philosophical goal than striving for a product implementation. His intention is to frame and structure the discussion of music markup in order to develop a standard within the music community rather than within corporate labs. His efforts seem to be bearing fruit.

The Music Encoding Initiative (MEI), lead by Perry Roland of the University of Virginia, is emerging as a contender XML standard (www.people.virginia.edu/~pdr4h/mei). The MEI is a stepchild of the Text Encoding Initiative (TEI), a mature standards effort revolving around the encoding and transmission of text. Like the TEI, the Music Encoding Initiative is a noncommercial endeavor, which Roland says eliminates “the pressure to rush to shipment; instead we can slow down and develop a DTD that not only works right now but has some longevity.”

Roland describes the MEI as being in the “testing and tweaking” phase. The MEI strives to meet the needs of a broad range of music applications while focusing on a smaller set of features than the full MML framework. It also avoids the obscurities of SMDL. (In the MEI, a note is called a note.) In addition, the MEI takes a more scholarly bent than many of its rivals by supporting features most commonly associated with historical or critical editions of music, such as commentary analysis.

While the MEI is “way past the proof-of-concept phase” in Roland's words, it has yet to move from the academic realm into a commercial product. MusicXML from Recordare (www.recordare.com) is one XML-for-music implementation that has made the leap.

MusicXML is without question the most mature effort so far in using XML to encode music. Like the MEI, MusicXML has its roots in academia as a direct descendant of MuseData and Humdrum. Company founder Michael Good took the core functionality of those venerable programs and built an XML implementation, choosing to support essential core features for music encoding rather than comprehensive coverage. That focus has resulted in an XML implementation that has found its way into more than a dozen commercial applications, including MakeMusic Finale.

Most recently, Recordare released Dolet for Sibelius Inc.'s Sibelius. The software plug-in exports Sibelius 2.1 (and later) files to MusicXML, serving as a sort of “Universal Translator” between Sibelius and any application that understands MusicXML. With the Dolet plug-in, programs like Finale, Igor Engraver, and Sibelius, which prior to MusicXML had been isolated by their proprietary formats, can exchange files much more accurately.

FIG. 2: AMuseTec's MuseBook Score relies on MusicXML for its combination of music-performance recognition and its automatic page turning capability.

MusicXML is also finding its way into hardware applications, such as the AMuseTec MuseBook Score (see Fig. 2). MusicXML is currently at version 0.8. Good plans to have it to the 1.0 level by the end of the year. At that point, Recordare intends to submit the DTD to OASIS (Organization for the Advancement of Structured Information Standards) for consideration to become a formally recognized XML standard for music.

BEYOND NOTATION

Each of the leading implementations of XML for music has focused on notation and scoring, but that's a small sample of the possible applications for XML and music. Several applications taking advantage of XML for other music-related purposes are starting to emerge, including several music-analysis tools (see Fig. 3). Other developers, MML's Steyn among them, are working on ways to “sequence” a piece of music with no software beyond a simple text editor, such as Microsoft's Notepad.

FIG. 3: This XML-based Pitch Range Analysis tool was developed by Recordare.

One potential drawback, viewed as a strength by developers, is that all of the currently available DTDs restrict themselves to Common Western Music Notation (CWMN or just CMN), with some also including tablature. CMN focuses on European music from the period between 1700 and 1950, which glosses over a large part of the world's music and its history. This is largely due to the dangers of trying to be all things to all people. Even with this limitation, XML holds remarkable promise for the representation, interchange, and analysis of music.

---

Darin Stewart is a Chapman Stick player in the Portland, Oregon, area. He is also Director of Research Information Systems for Oregon Health and Science University (stewarda@ohsu.edu).