Practice mirror goes 3-D

Margaret W. Bushee, margaret.bushee@laurin.com

For a musician, the traditional optical device is the mirror. A violinist stands in front of one while practicing to visualize whether his bow is parallel to the bridge and his instrument positioned optimally in relation to his body.

Dr. Kia Ng, a violinist and member of the school of music and engineering faculties at the University of Leeds, has developed a 3-D “practice mirror” that can record and analyze a musician’s posture, movements and sounds. The optical motion capture system, called the i-Maestro 3D Augmented Mirror, consists of cameras with an infrared light source, analysis software and reflective markers positioned on the player’s bow, instrument and body.

The cello player shown is using the 3D Augmented Mirror system to model and analyze his bowing technique. Cameras installed at various locations around the room record data in real time. The project is described in more detail at www.i-maestro.org.

The reason for dissecting the making of music is a matter of basic physics: Good technique produces a good sound. According to Ng, “In general, if the bow is held perpendicular to the string and parallel to the bridge, the minimum effort will produce the maximum result. Our system can measure this and show musicians exactly when their technique becomes less effective.”

This prototype is designed for string instruments but could be adapted for others. In a small room, six to 12 cameras are positioned near the walls in locations that circle the musician. Surrounding each camera’s lens is an infrared light source that illuminates the many reflective markers. Operating at a high speed of 100 or 200 fps, the cameras capture a high-contrast image by extracting the specific IR wavelengths reflected off the markers.

Via TCP/IP and a gigabit network connection, data is transferred to a computer, where calibration data – a synthesis of information from different cameras – is computed to determine the 3-D position of the marker. By watching an animated graphic of his instrument and bow, the musician can see, frame by frame, his performance, along with an on-screen analysis.

Optical motion capture was not the only method considered for this project. Computer vision and magnetic motion capture were evaluated, but optical motion capture was chosen for its combination of speed and accuracy.

Ng is exploring the possibility of a more portable system, along with refinements in gesture and posture analysis. He hopes that, eventually, the hardware will be less expensive than its current £5000 to £100,000 ($7,000 to $143,000).

As for the students, reportedly they are enthusiastic about the 3D Augmented Mirror. Ng said they have found it particularly helpful for achieving consistency with, for example, complex bowing techniques such as spiccato, where multiple factors such as speed, bow position and bow-bridge angle are key to producing the right sound.