Home   >   Open Source Projects   >   Creating a Unique Electronic Musical Instrument: The Sound Wall

Creating a Unique Electronic Musical Instrument: The Sound Wall

By on September 25, 2023
Pin It

 

We place metal foil notes on a panel and play our music by touching them, creating a melody from a truly original instrument.

Those who know a little about music or have good memories of the “music education” hours at secondary school (or secondary school, to put it in a modern way), will certainly remember that musical instruments are divided into various categories based on how the more or less erudite musician operates them to produce notes: we have wind instruments (flute, oboe, clarinet, mouth organ, etc.) percussion instruments (drums, piano, etc.), string instruments (guitar, bass, violin, but also the ancient harpsichord could fall into this category), pipe instruments and so on. We have wind instruments (flute, oboe, clarinet, mouth organ, etc.), percussion instruments (drums, piano, etc.), string instruments (guitar, bass, violin, but also the ancient harpsichord could fall into this category), pipe instruments and so on. They all have in common the fact that they work without the need for modern technology. It is not by chance that they were created many years ago when electricity or electronics were unknown, and based on very simple and primordial physical principles.

What we would like to propose to you in these pages is something a bit unusual, because it is an electronic musical instrument that we could place among those with a keyboard, but which is played by touching keys (we call them that even though they are not) which are not pressed but are simply touched or touched. As you can see in the photos of the prototype on these pages, we have conceived it as a large panel (Fig. 1) with metal plates shaped like musical notes applied to it (it is no coincidence that we have decided to call it Sound Wall…), but anyone can give the “user” interface, and therefore the keyboard, the shape, and appearance they prefer.

 

Fig. 1

 

The project

Let’s see what it’s all about, also with the help of the wiring diagram on these pages: our instrument is nothing more than an Arduino Uno board interfaced to the plates (by touching them with your fingers, the sound will be reproduced) by means of MPR121CAPTOUCH breakouts, which are contact sensors and which in the circuit are proportional in number to the notes. Every time we touch a metal plate, the breakout board to which it is electrically connected will detect the transfer of the amount of electric charge to our body and will send a signal to the Arduino Uno, which, according to the loaded firmware, will command the generation of a sound using the “collaboration” of the FT1414M board from Open Electronics, which we have already used in the project.

The breakout board touch is basically a board for the management of touch sensors, then to each pin NOTE and SEL we have connected some aluminum plates with any shape (specifically musical notes and geometric shapes) so that pressing the finger on them, can act as a button; each board reads a certain number of plates-notes and to be exact 18, of which 14 for the notes (two octaves exactly …) and four to set the mode of execution, or simulated instruments or “effects” if you prefer. These sound effects are the Celesta (or celestial metallophone with hammers), the Carillon and two standard MIDI synthesizer sounds which are the FX2 (space weapon type synthesizer) and the Sci-Fi FX8 (space sound synthesizer).

The breakout board for touch management is based on Freescale’s MPR121QR2 chip, which is a capacitive touch sensor controlled via an I²C interface. The chip can individually control up to 12 electrodes and a maximum of 8 LEDs (when the pins are not configured as electrodes), which is why you see two of them in our design. At the bottom of the PCB are 4 jumpers (pads with tracks), all of which are closed for the default setting. One of these jumpers connects the ADD pin to the ground, so the I²C-Bus address of the chip will be 0x5A. Using two breakouts, on the second this jumper should be opened, thus giving the board the address 0x5B. The jumpers also connect the SDA, SCL and IRQ pins to 10 kohm pull-up resistors. The breakout does not have a voltage regulator, so the power supply must be between 2.5 and 3.6 Vdc, which the Arduino Uno does with its built-in regulator that provides voltage via pins 3V3 and GND.

The sound generation is done by the Arduino FT1414M shield based on the Ogg Vorbis/MP3/AAC/WMA/MIDI audio decoder chip (VS1053B) produced by VLSI.

It allows a certain number of musical notes to be generated by synthesizing the sounds of a large number of musical instruments, both melodic and percussive, following commands given by a data channel; the commands accepted can be in various formats, including the popular MIDI, which has been the standard used for decades to interface music synthesizers and electronic keyboards to the PC.

 

 

For communication with the host device (the microcontroller governing the synthesizer), the VS1053B is used in this shield in serial mode.

The shield takes power from the Arduino via the 5V pin and the common GND; 5 volts is supplied by the PAM8043 amplifier module, while 3.3 volts and 1.8 volts are supplied by the LDO linear regulators (MIC5504-1.8YM5-TR and MIC5504-3.3YM5-TR).

The VS1053B integrated amplifier has a stereo audio output connected to a printed circuit jack to which headphones can be connected. In addition, the same output is connected to a low-power BF amplifier (not included in the shield) with low-impedance outputs that raises the signal level enough to drive a pair of 3-watt, 4-ohm impedance speakers.

The mini-amplifier code PAM8403POT is mounted on its own adapter board and plugged into the shield via a socket.

The electronics assembly is shown in Fig. 2 applied to the panel from the front.

 

Fig. 2

 

 

How it works

Now let’s see how the whole thing works, starting with the virtual keys: by placing your finger or hand on a sensor blade, we cause the breakout board to output a string of data that is read from the corresponding Arduino I²C bus. This event causes the Arduino to play the corresponding note, which is generated by driving the VS1053B’s internal MIDI synthesizer via serial, through the generation of a data string on tx0, containing information about the corresponding note. The data is in MIDI format and is therefore decoded by the VS1053B. The VS1053B is a multistandard codec and decoder of many formats, but it has the particularity of integrating a large-scale sound synthesizer based on VLSI’s proprietary DSP (Digital Signal Processor) (i.e., the VS_DSP). It contains code and data memory for the Ogg Vorbis, MP3, AAC, WMA and WAV PCM+ADPCM audio decoding codecs, as well as a complete standard MIDI synthesizer; it communicates with the outside world via a serial interface, which can be configured as SPI.

Inside the VS1053B is a multi-rate stereo DAC and audio preamplifier stage.

with a stereo version of the output filter. The integrated amplifier also supports PCM/ADPCM audio encoding, using either a microphone amplifier or a high-level line input with a built-in preamplifier for the input signal to be encoded; the input signal is sent to a stereo A/D converter.

Thanks to software plug-ins, the chip can also perform lossless FLAC decoding like that of a high-quality recording in Ogg Vorbis format. The integrated sound synthesiser with a standard MIDI interface has two banks of “effects” corresponding to musical instruments and called GM1 (for melodic instruments) and GM2 (for percussion instruments); in our application, we use the GM1 bank, but by modifying the

firmware you can change the instrument if you feel like it and want to try it.

The musical notes produced are 12 per octave (i.e., the 7 notes plus halftones) and the possible octaves are 11, so the VS1053B covers an extremely wide musical range, well beyond that of the piano (which has 7 octaves plus a C). Table 1 summarises the individual notes by correlating them with the value of the data received: as you can see, 128 sounds are possible for each “register” or effect, each identified by a numerical value that is passed from the Arduino to the VS1053B in serial form in binary format.

Table 1

 

The first activates instrument 1, which is the Celesta (or hammer metallophone), the second the music box, the third the space-weapon synthesizer, and the fourth the Sci-Fi FX8 synthesizer (a sound from a science fiction film). The above sounds and all those that can be developed within the GM1 bank are described in Table 2.

Table 2.

 

The midi shield

Let us now take a closer look at the heart of our musical instrument, namely the shield based on VLSI’s VS1053B integrated circuit, which is capable of generating a certain number of musical notes by synthesizing the sounds of a large number of musical instruments, both melodic (including the harp and organ) and percussion, following commands given by a data channel; the commands accepted can be of various formats, including the popular MIDI, which has been the standard used for decades to interface musical synthesizers and electronic keyboards to the PC.

For communication with the host device (the microcontroller that controls the

circuit, which in our case is Arduino) also in SPI-bus mode. In fact, the VS1053B integrates a serial interface complex that has two interfaces: a control interface (SCI=Serial Command Interface) that is used to control the operation of the chip and that in our case is the one through which Arduino gives its commands, and a data transfer interface (SDI=Serial Data Interface) thanks to which the VS1053B can transfer or acquire data streams to or from another external device.

Of the two, the one that can be configured to operate in TTL or SPI serial mode is the SCI, the control interface. The integrated sound synthesizer is based on a VLSI proprietary DSP (Digital Signal Processor), called the VS_DSP. The synthesized sounds are converted by a multi-rate stereo DAC, the output of which is a BF amplifier with a filter to cut down the conversion residues.

The integrated circuit also supports PCM/ADPCM audio coding, either by using a microphone amplifier as an input stage (for those who wish to use a microphone as a source) or by taking the signal from a line input and sending it to a stereo A/D converter. In our shield, Arduino manages the VS1053B through the serial port used as UART, which in the U1 is connected to pins 26 (RX) and 27 (TX), connected respectively to digital pins D3 and D2 of the Arduino (which are therefore reserved); the latter also manages the reset of the U1, through D4, so as to prepare it to receive commands.

 

Fig. 3

 

The three lines in question have pull-ups on the side of the VS1053 and are controlled by the Arduino pins using resistors in series; three jumpers (JRX, JTX and JRST) on the shield’s printed circuit board allow them to be disconnected, if necessary, for example if a different form of control is chosen, such as SPI.

In any case, there are three lines to allow interfacing to SPI, since one acts as a data channel, one as a reset and the other as a clock; all you need to do is write the firmware accordingly.

The shield takes power from the Arduino via the 5V pin and GND; on board it has LDO regulators to obtain 1.8 volts (MIC5504-1.8YM5-TR) and 3.3 volts (MIC5504-3.3YM5-TR) to power the various stages of the VS1053B.

The VS1053B integrated amplifier has a stereo audio output located on pins 39 (R channel) and 46 (L channel) referring to the common pin 42 (GBUF), which is not connected to the analog ground, but also coupled, like the signal lines, via R/C networks. The outputs of the U1 audio reach a stereo jack socket on the printed circuit board to which it is possible to connect headphones since they are connected to a low-power BF amplifier with low-impedance outputs.

The same R and L lines reach the inputs of the PAN8403 amplifier module, which raises the signal level enough to drive a pair of 3-watt, 4-ohm impedance loudspeakers; the U4 module is the PAM8403, based on the homonymous integrated amplifier already equipped with all the external components it needs and the volume control potentiometer.

 

The firmware

Well, now that we have explained what the hardware consists of, we can see how the sketch loaded into Arduino works to control our musical instrument, i.e., reading the status of the touch sensors (metal plates or foils) and generating the corresponding commands to the MIDI shield.

At the code level, we can see the “Play” function, which is the one that allows the note to be played if finger contact is detected on some “Touch” plate, in turn via the “noteOn” function. The “checkInterrupt1” and “checkInterrupt2” functions are the first to identify the key pressed by the NOTE breakout and the second for the SEL breakout. If a pressed key is detected I set the location “i” in the “touchStates” array for the pressed key to the value “1”, or “0” if not pressed. The sketch is structurally and conceptually simple, because after initializing the lines and defining the variables it reads the state of the plates from the data provided by the two touch-sensitive breakout boards if a contact is detected. If this happens, the firmware first of all identifies the corresponding input and distinguishes between note commands and register or instrument setting commands: in the first case, it makes the shield execute the corresponding note by sending a MIDI command string, while in the second case, it orders the VS1053B on the shield itself to set the corresponding instrument.

Whenever a new instrument is chosen from the 4, a note of the new instrument is played to identify the change.
Regarding the functions, there are basically 4 different instruments, although we only used 4 of them, but they can vary without problems by changing the number to the constant const int.
The choice of instruments is linked to the second breakout where we have indicated “SEL” in the wiring diagram:

const int INSTRUMENT1=9;const int
INSTRUMENT2=11;const
int INSTRUMENT3=98;
const int INSTRUMENT4=104.

The instruments provided and set in the sketch are, according to Table 2 which shows the program numbers of bank 0 of the midi VS1053B:

– 9 Celesta (or celestial metallophone);

– 11 Carillon;

– 98 FX2 soundtrack (space weapon-type synthesizer);

– 104 Sci-Fi FX8 (space sound synthesizer).

 

Depending on which metal plate we touch, we will then obtain notes that will reproduce the sound of the corresponding instrument.

The sketch that manages our Sound Wall is rather large to be able to report it entirely in these pages, so we show you three extracts that are the most significant: Listing 1 shows the setup for the two cards that acquire the signals related to metal contacts, i.e., the touch sensors.

Listing 1

void mpr121_setup(void){
set_register(0x5A, ELE_CFG, 0x00);
// Section A - Controls filtering when data is > baseline.
set_register(0x5A, MHD_R, 0x01);
set_register(0x5A, NHD_R, 0x01);
set_register(0x5A, NCL_R, 0x00);
set_register(0x5A, FDL_R, 0x00);
// Section B - Controls filtering when data is < baseline.
set_register(0x5A, MHD_F, 0x01);
set_register(0x5A, NHD_F, 0x01);
set_register(0x5A, NCL_F, 0xFF);
set_register(0x5A, FDL_F, 0x02);
// Section C - Sets touch and release thresholds for each electrode
set_register(0x5A, ELE0_T, TOU_THRESH);
set_register(0x5A, ELE0_R, REL_THRESH);
set_register(0x5A, ELE1_T, TOU_THRESH);
set_register(0x5A, ELE1_R, REL_THRESH);
set_register(0x5A, ELE2_T, TOU_THRESH);
set_register(0x5A, ELE2_R, REL_THRESH);
set_register(0x5A, ELE3_T, TOU_THRESH);
set_register(0x5A, ELE3_R, REL_THRESH);
set_register(0x5A, ELE4_T, TOU_THRESH);
set_register(0x5A, ELE4_R, REL_THRESH);
set_register(0x5A, ELE5_T, TOU_THRESH);
set_register(0x5A, ELE5_R, REL_THRESH);
set_register(0x5A, ELE6_T, TOU_THRESH);
set_register(0x5A, ELE6_R, REL_THRESH);
set_register(0x5A, ELE7_T, TOU_THRESH);
set_register(0x5A, ELE7_R, REL_THRESH);
set_register(0x5A, ELE8_T, TOU_THRESH);
set_register(0x5A, ELE8_R, REL_THRESH);
set_register(0x5A, ELE9_T, TOU_THRESH);
set_register(0x5A, ELE9_R, REL_THRESH);
set_register(0x5A, ELE10_T, TOU_THRESH);
set_register(0x5A, ELE10_R, REL_THRESH);
set_register(0x5A, ELE11_T, TOU_THRESH);
set_register(0x5A, ELE11_R, REL_THRESH);
// Section D
// Set the Filter Configuration
// Set ESI2
set_register(0x5A, FIL_CFG, 0x04);
// Section E
// Electrode Configuration
// Set ELE_CFG to 0x00 to return to standby mode
set_register(0x5A, ELE_CFG, 0x0C); // Enables all 12 Electrodes
set_register(0x5A, ELE_CFG, 0x0C);
//***************** seconda scheda con indirizzo 0x5B ************************
set_register(0x5B, ELE_CFG, 0x00);
// Section A - Controls filtering when data is > baseline.
set_register(0x5B, MHD_R, 0x01);
set_register(0x5B, NHD_R, 0x01);
set_register(0x5B, NCL_R, 0x00);
set_register(0x5B, FDL_R, 0x00);
// Section B - Controls filtering when data is < baseline.
set_register(0x5B, MHD_F, 0x01);
set_register(0x5B, NHD_F, 0x01);
set_register(0x5B, NCL_F, 0xFF);
set_register(0x5B, FDL_F, 0x02);
// Section C - Sets touch and release thresholds for each electrode
set_register(0x5B, ELE0_T, TOU_THRESH);
set_register(0x5B, ELE0_R, REL_THRESH);
set_register(0x5B, ELE1_T, TOU_THRESH);
set_register(0x5B, ELE1_R, REL_THRESH);
set_register(0x5B, ELE2_T, TOU_THRESH);
set_register(0x5B, ELE2_R, REL_THRESH);
set_register(0x5B, ELE3_T, TOU_THRESH);
set_register(0x5B, ELE3_R, REL_THRESH);
set_register(0x5B, ELE4_T, TOU_THRESH);
set_register(0x5B, ELE4_R, REL_THRESH);
set_register(0x5B, ELE5_T, TOU_THRESH);
set_register(0x5B, ELE5_R, REL_THRESH);
set_register(0x5B, ELE6_T, TOU_THRESH);
set_register(0x5B, ELE6_R, REL_THRESH);
set_register(0x5B, ELE7_T, TOU_THRESH);
set_register(0x5B, ELE7_R, REL_THRESH);
set_register(0x5B, ELE8_T, TOU_THRESH);
set_register(0x5B, ELE8_R, REL_THRESH);
set_register(0x5B, ELE9_T, TOU_THRESH);
set_register(0x5B, ELE9_R, REL_THRESH);
set_register(0x5B, ELE10_T, TOU_THRESH);
set_register(0x5B, ELE10_R, REL_THRESH);
set_register(0x5B, ELE11_T, TOU_THRESH);
set_register(0x5B, ELE11_R, REL_THRESH);
// Section D
// Set the Filter Configuration
// Set ESI2
set_register(0x5B, FIL_CFG, 0x04);
// Section E
// Electrode Configuration
// Set ELE_CFG to 0x00 to return to standby mode
set_register(0x5B, ELE_CFG, 0x0C); // Enables all 12 Electrodes
set_register(0x5B, ELE_CFG, 0x0C);

 

In Listing 2 you will find the portion of the sketch that deals with the actual management of the MPR121 breakout bar, i.e., the touch-sensor control. Finally, we report, in Listing 3, a last piece of code concerning the control of notes reproduction by the shield with VS1053B, in which the constant int note assumes the value of the note to be played each time.

Listing 2

void readTouchInputs()
{
if(!checkInterrupt1())
{
//read the touch state from the MPR121
Wire.requestFrom(0x5A,2);
byte LSB = Wire.read();
byte MSB = Wire.read();
uint16_t touched = ((MSB << 8) | LSB); //16bits that make up the touch states
for (int i=0; i < 12; i++)
{ // Check what electrodes were pressed
if(touched & (1<<i))
{
touchStates[i] = 1;
}
else
{
touchStates[i] = 0;
}
}
}
if(!checkInterrupt2())
{
//read the touch state from the MPR121
Wire.requestFrom(0x5B,2);
byte LSB = Wire.read();
byte MSB = Wire.read();
uint16_t touched = ((MSB << 8) | LSB); //16bits that make up the touch states
for (int i=0; i < 12; i++)
{ // Check what electrodes were pressed
if(touched & (1<<i))
{
touchStates[i+12] = 1;
}
else
{
touchStates[i+12] = 0;
}
}
}
for (int i=0; i<24; i++)
{
Serial.print(touchStates[i]);
}
Serial.println();
}

 

Listing 3

void Play()
{
int note=30;
for (int i=0; i<20; i++)
{
if ((touchStates[i]==1) && (touchPres[i]==false))
{
noteOn(0, note, 127);
touchPres[i]=true;
}
if ((touchStates[i]==0) && (touchPres[i]==true))
{
noteOff(0, note, 127);
touchPres[i]=false;
}
note++;
}
delay(100);
}

 

Conclusions

Well, we think we have explained enough to prepare our musical instrument with touch and to configure it in order to obtain the desired sounds; as explained, the project lends itself to multiple elaborations and gives you maximum freedom of expression in terms of practical implementation and choice of “keyboard”, while respecting the rules indicated.  The version proposed in the photos on these pages is a suggestion, but each of you can give free rein to your imagination by arranging the plates on the support you consider most suitable and shaping the plates as you wish.

 

OpenStore

Arduino UNO R3

Shield MIDI with VS1053B

MPR121 – Capacitive Touch Sensor Breakout

Micro amplifier 3 + 3 watt

OpenStore

About Boris Landoni

Boris Landoni is the technical manager of Open-Electronics.org. Skilled in the GSM field, embraces the Open Source philosophy and its projects are available to the community.

One Comment

Leave a Reply

Your email address will not be published. Required fields are marked *