Mind Control

Aug 1, 2007 12:00 PM, By Scott Wilkinson

           
 

In the May 2007 issue of EM, you'll find a feature entitled “Overcoming Adversity” written by Joanna Cazden and myself. In that article, we examined some of the ways in which musicians with various disabilities use technology to make music despite their limitations.

Not long after that article appeared in print, I came across another technology that could be of great benefit to anyone with impaired mobility wishing to use computers, including musicians. Developed over the last 17 years by Andrew Junker and available from his company, Brain Actuated Technologies (www.brainfingers.com), Brainfingers is a remarkable system that opens a whole new world of computer access that is normally denied to those with limited or no use of their hands.

FIG. 1: Each colored bar represents the signal strength of a particular brainfinger, and the waveform along the bottom is a composite of all three biometric signals. In this example, the lowest-frequency alpha brain wave is the strongest, suggesting that the user was in a meditative state.

The hardware includes a headband with three sensors that make contact with the user's forehead. The sensors are connected to an interface box that amplifies, filters, and digitizes the signals and sends them to a Windows computer via USB. A software driver interprets the signals and uses them to move the cursor, click on icons, invoke macros, and perform other tasks normally accomplished with a mouse and keyboard. The product is designed to be used with a wide variety of third-party software.

The headband sensors detect three types of biometric signals: eye movement (electrooculographic, or EOG), facial-muscle movement (electromyographic, or EMG), and brain waves (electroencephalographic, or EEG), primarily alpha (8 to 12 Hz) and beta (>12 Hz). Patented algorithms separate the different types of signals and further divide the brain-wave signals into distinct frequency bands. Each signal type and frequency band is then used as a virtual finger, or “brainfinger” (see Fig. 1), that can be mapped to control a particular software trigger or continuous parameter.

Muscular and eye movements can be controlled relatively precisely. For example, Junker's colleague, Chris Berg, a composer and music professor at the University of Cincinnati, programmed the system to use the pressure of his tongue on the roof of his mouth (which involves some facial muscles) as a signal to change notes played on a MIDI synth.

However, users must practice using the system to gain proficiency and precision with it, so the software includes a training program. Once the nuances are mastered, users can often improve on their best reaction times. In a study conducted by the United States Air Force, subjects' reaction times to visual stimuli with the Brainfingers EMG “button” were 15 percent faster than with a physical button.

For musical applications, Junker and Berg have developed several ideas, such as triggering short MIDI phrases with different brainfingers and controlling the tempo of prerecorded MIDI and digital audio segments by flexing the jaw muscles. At the 1999 South by Southwest music and film festival and conference, the lead singer of a Native American group called A Band of Indians wore a Brainfingers headband and affected the playback of some MIDI synths while singing.

According to Junker, the most important thing about Brainfingers is that it measures both brain and body neural impulses, reflecting the complex and intimate interaction between volitional muscle movements and mental/emotional states. Junker claims that when applied to music, the effect can be quite profound for performers and listeners alike. The system can also be used as an advanced biofeedback device, inducing calmer mental states as soothing music is played in response to relaxing the facial muscles and lowering brain-wave frequencies. The potential of this technology is vast for both mobility-impaired and able-bodied musicians who want to explore how music, mind, and body can interact in entirely new ways.