Using Nintendo Wiimote for Music Production

Mar 1, 2009 12:00 PM, By Paul D. Lehrman

           
 

USING NINTENDO'S WIIMOTE AS A MUSICAL-INSTRUMENT CONTROLLER

FIG. 2: The Max Object called aka.wiiremote that brings Wiimote data directly into Max/MSP.

For Windows users, there's a comprehensive tool called GlovePIE (donationware), developed by Australian programmer Carl Kenner. Originally created for use with virtual reality gloves (as the name would indicate; the second part stands for “Programmable Input Emulator”), GlovePIE is a scripting language that supports a wide range of wired and wireless controllers, including the Wiimote. It's capable of working in many environments, and the downloadable package includes both OSC and MIDI commands.

Web Links

Akamatsu Aka.wiiremote

Cycling '74's Site

DarwiinRemote

IOSpirit Remote Buddy

Kenner GlovePIE

Nyko Wireless Sensor Bar

OPen Sound Control (OSC)

PureData

Troillard OSCulator

Wii Community

WiinRemote

Wiinstrument

Wiinstrument is a free, simple, open-source, cross-platform (Mac, Windows, and Linux) application from Germany that lets you generate MIDI notes and controller messages. In one mode, it provides a virtual keyboard, and in its Percussion mode, you can use the Wiimote like a drumstick. Acceleration in the vertical plane is mapped to Velocity sensing, and the buttons select various drum sounds.

If you're a Cycling '74 Max/MSP user, you can get Wiimote data into your creations with a free Max Object called aka.wiiremote (see Fig. 2). It allows you to use all the different Wiimote data streams in your Max patch and lets you send data to the Wiimote, activating its lights and rumble generator. Aka.wiiremote has a unique feature: you can throttle back the speed of the incoming data to cut down on noise and to avoid overwhelming your MIDI stream. (The Japanese developer, Masayuki Akamatsu, also has a very cool Object that uses a Macintosh laptop's Sudden Motion Sensor to tell you when you're tilting the computer in any direction. It may not be all that useful for music, but you can use it to turn your MacBook into a seismograph.)

My favorite way of getting Wiimote data into my computer is a neat little application called OSCulator, which comes from France (donationware, minimum $19). Its developer, Camille Troillard, is a power user of Symbolic Sound's Kyma, and he wrote the program so he could choose from all sorts of external controllers — graphics tablets, 3-D mice, JazzMutant's Lemur, and a variety of wireless gadgets, including Wii-Fit Balance Boards and iPhones — to play his Kyma. In the process, he put in hooks for OSC and MIDI and designed a very simple and elegant, yet highly versatile, interface (see Fig. 3). Unfortunately for Windows users, OSCulator runs only on Macs.

FIG. 3: The main screen for assigning MIDI commands to Wiimote parameters in OSCulator.

OSCulator lets you assign any Wiimote parameter to any MIDI note, controller message, or other system message as single events or toggles. A special Note w/Parameters command allows you to use continuous parameters to control the characteristics of a note. For example, when you use a button to trigger a note, you can use the vertical tilt of the Wiimote to specify the note number and the horizontal position of the Nunchuk joystick to set the Velocity. Wiimote functions can be mapped to different MIDI channels, and the software can accommodate up to four Wiimotes with their Nunchuks at the same time. You get separate smoothing functions for all the continuous sensors to cut down on noise, as well as a prominent indicator for the battery level, which goes down amazingly fast. OSCulator also does a heroic job of “discovering” multiple Wiimotes — and does so far more efficiently than Apple's Bluetooth Preference Panel.

Any of these programs will let you do some cool musical things with a Wiimote. Adding a Nunchuk gives you 12 different real-time parameters you can control, and you can split these among as many MIDI channels as you want. Imagine a virtual drum set in which you can select which drums you play by pointing at them and then flicking your wrist to make it sound; or a loop with a steep resonant filter that opens up and growls as you raise your arm; or a vocal patch that responds to one arm's position for pitch, the other arm's position for timbre, and the twist of a wrist for vibrato. The more continuous controllers a synth patch has available, the more you can mangle the sound with your physical gestures.

You can really go crazy with a Wiimote if you're willing to do a little programming in a processing language like Max/MSP or the open-source PureData (Pd). With these tools, the action of any parameter can be interdependent with the action of any other, and you can do tricks like reassign all the controls on the fly.