Using Nintendo Wiimote for Music Production

Mar 1, 2009 12:00 PM, By Paul D. Lehrman

           
 

USING NINTENDO'S WIIMOTE AS A MUSICAL-INSTRUMENT CONTROLLER

FIG. 4: One of the display screens from “Imaginary Dialogues,” showing which parameters are being controlled and their current state.

A piece, “Imaginary Dialogues,” that graduate student Phil Acimovic and I recently wrote and performed for two Wiimotes, Nunchuks, and Max/MSP had 12 scenes. When I raised my left hand and shook the Nunchuk hard, it sent a message to Max (and also a visual message to Phil) to change scenes — and the actions of the controllers instantly changed completely.

The output of Max/MSP was sent to two Propellerhead Reason racks, each with a bundle of synthesis and processing modules, and different combinations of modules were used in different scenes. In one scene, pressing the B button played and held a random note, but the randomness was restricted by the middle three buttons: whichever of those was last pressed determined the scale the note would fall in — major, harsmonic minor, or pentatonic. Raising the pitch angle of the Wiimote raised the volume, while changing the yaw angle moved the sound within the stereo field, and moving the Nunchuk's joystick altered the filter envelope. When used with a slow pad from a Maelström module, it was very effective.

A test patch in Max that lets you look at the data from the Wiimote as it is being converted into MIDI.

In another scene, holding the B button spit out a string of random short notes, again restricted by the scale selection, with the yaw controlling the speed of the notes and the pitch controlling the range. The Nunchuk's joystick and motion sensors changed their character within the scene depending on which buttons were pressed: they might be used for LFO rate, filter frequency, vocoder mix, reverb wet/dry mix, distortion, or the degree of feedback in a flanger (see Fig. 4).

Phil did some very fancy programming that showed us, on two screens, exactly which parameters were being controlled at any moment and what their current values were. You can look at some of the Max patches at emusician.com (see Web Clips 1, 2, and 3) and watch our performance at tuftsemid.com.

This is just the beginning. With the economies of scale the video-game industry offers, we can look forward to devices with even more sophisticated capabilities in the future. And because Wiimotes are small, they can be attached to other objects or to people's clothing and can give them the power of motion detection and tracking. It won't be long before the technology is put to use by anyone or anything that moves, from dancers and circus performers to model-car makers and pet owners. Right now, though, these simple and inexpensive toys will let you get creative, onstage and in the studio, in ways you've probably never thought of before.

Paul D. Lehrman is the author of The Insider Audio Bathroom Reader, a collection of 11 years of his columns for EM's sister publication Mix. He is also coordinator of music technology for Tufts University and has been known to do strange things with player pianos and robots.

SOFTWARE FOR WIIMOTE CONTROL