Musical Mapping

A mini M5Stack, the M5StickC, is utilizied to help envision the effects of Movement Through Sound. The device was engineered for artists to draw with guitar sounds connected to the acceleration of their movements. As the users slows down so does the sound, and as they speed up, the sound increases.

Max/MSP and C++ through Arduino are used to connect to the device. A Timestretch loop alters the sound with reverb and depth perception. A simple guitar octave is the melody altered.

Skills

Max/MSP

skills.Max

Max/MSP/Jitter is a visual programming language providing the utility to edit sound, image, and most other multimedia. The oncoming accelerometer data is used to calculate the timestretch effect on the guitar track. The track has settings for 8 different energy accumulations. As the track progresses each setting will be played at different intervals.

C++

skills.Max

The Arudino IDE supports C++ to program the watch. It connects to the watch via wifi and records the Accelerometer and Gyroscope data. An M5Stack can measure very minute changes in the gyroscope. All direction is measured, meaning x, y, and z coordinates are captured. The acceleration of each direction is recorded and sent to max, where it alters the guitar file.

Video

Goal & Report

Goal

Goal.MM

The goal of Musical Mapping is to create a device that plays music as one is drawing. A wearer will be able to slow down or speed up the sound as they develop their illustration. In moments of high intensity, slow smooth strokes, the music will fade into the background to help focus while speeding up as the user generates fast erratic strokes.

Report