This is an introduction to digital interfaces, with a focus on sculptural, spiritual, ecological, and experimental approaches to instrument design and interactive objects. This module was developed through the Free Tools project for the New Media + Sound Arts program at Emily Carr University. I try to update these materials each time I teach a workshop or course.
the technical
This module uses low cost microcontrollers to sense the physical world, reading button presses, light, temperature, touch, etc. The microcontroller maps these sensor inputs to processes and events on a computer using MIDI messages. It works in the other direction too — the computer can be used to send messages that affect things in the physical world as well, like lights, speakers, relays, radios, and motors.
This site is intended to be a thorough introduction to one way of connecting sensors and actuators to a computer. It is important to note that there are a number of other platforms that allow you to do this. In addition to Arduino (which we will be using), there are many other hardware platforms like Raspberry Pi, Teensy, Bela, and Daisy, just to name a few.
If you are new to this, I recommend starting here with a Pro Micro development board for introductory explorations for a couple of reasons. First, they are relatively inexpensive (about $12-16 for the board) and second, they can be configured as class compliant midi devices, meaning that your computer will automatically recognize the board as a midi device when you plug it in. If you need more inputs, you can expand the i/o, or you can upgrade to a number of other boards that are faster and/or have more inputs and outputs.
Opinions:
- The Teensy boards are a bit more expensive and seem more complicated initially, but they are faster and more reliable than the Pro Micros. Teensy even makes an audio adapter that lets you run audio programs on the teensy for stand alone projects, no computer necessary!
- The ESP32 microcontroller chip is very popular and for good reason. I am currently working on code for the Arduino Nano 33 IoT, which would allow for wireless communication with the computer (stay tuned).
- If you are doing complex embedded audio processes and audio latency matters for your project, I highly recommend taking a look at Bela — it’s a beautiful platform made specifically for sound. The Daisy platform is also great for embedded audio applications.
MIDI (Musical Instrument Digital Interface) is a standard communication protocol created for connecting electronic musical instruments, but it is great for interactive video installations, live performances, and kinetic sculptures as well. MIDI messages can be mapped to a broad range of software — Max/MSP, Pure Data, Ableton Live, Reaper, Mad Mapper, Processing — any program with MIDI input and/or output.
Next we’ll look at the software you will need