Interesting audio-tactile interactions exist in different fields. Here are a few examples:
Christian Moeller’s “Fuge for Piano, op. 154” is a touch-sensitive music installation. The original sheet music of Mozart’s composition is exhibited beneath a touch-sensitive sheet of glass, fitted within a normal picture frame. By touching the glass above any given note of the music, the visitor is able to hear it played. In this way it is possible to play the composition by tracing the bars of music with one finger. The speed of movement determines the tempo of the playback.
The Theremin is an electronic instrument played by waving the hands in the vicinity of two antennas, created by Leon Theremin in 1920. In the 70’s and 80’s, Laurie Anderson used the violin and bow as an input device; outputs were audio palindromes, howling animal voices and more. Recently, the exploration of tactile control for electronic instruments has become easier due the increased availability of inexpensive movement sensors.
“The Sounding Object” (SOB) is a research project developing sound models that are responsive to physical interactions. SOB researchers developed an audio-tactile interaction using a ’pop-up’ children’s book in which carton flaps are pulled to move elements in the page. One element is a glass of lemonade that a character is drinking with a straw. Pulling the flap down decreases the level of lemonade in the glass. In the SOB implementation, a sensor is connected to the flap so that pulling it creates sound by feeding the sensor data into a sound model of ’drinking from a straw’.
The Social Mobiles project is a collection of concepts for phones created by Crispin Jones and the IDEO design firm. The Speaking SoMo is a concept for a mobile phone in which the ‘speaker’ of the phone uses a joystick to manually produce and intone expressive sounds.