Designing a reprogrammable instrument
by Rafaele Andrade
In a reprogrammable sound performance, the musician, its audience, collaborators (live coders), environment and the music instrument can manipulate and interact with definitions of code, music forms and blocks of code. All these agents interfere into the performance by compositional methods of live code music and hybrid sound systems, allowing new ways to connect sonic expression with digital dynamism. In 2018, I started a project aiming to build an instrument that could offer and ‘host'[1] this concept.
This reprogrammable musical instrument is Knurl, a solar-powered, reprogrammable & hybrid cello. Its 4 modes of performance (Synthesis, Detection, Programming & Analogue mode) are installed in a microcontroller in self-contained electronic circuit, which means that all the electronic devices are attached to it, including speakers, microphone, microcontroller (bela), solar panels and sensors). Its circular format leads the strings’ force between the arm and tailpiece, directing the sound into the arm’s resonant chamber.
The tuning system stores the electronic components, made by up-cycled material. At its digital settings, the modes of performance manipulate the sound through algorithmic calculations and compositional methods. Summing up those possibilities there is a variety of (71) processing definitions. As a console of these processings, a screen at the bow streams the code. Capacitive buttons (made with Bela trill Craft) are attached into the arm cavities to trigger specific conditional functions.
A reprogrammable instrument: Knurl
In November 2018, the project to build an electric cello was born through the initiative of me and more 3 classmates (Sonology(2), and art-science (2)) in the Royal conservatory of the Netherlands, which originally the intention was just to make our own electric cellos in order to play music together. My colleagues sketched an electric cello thin, with 4 strings and guitar pickups. On my first version, I aim for a cello with 12 strings, made simply of wood with a microcontroller and sensors. In the process of development, the cello evolved into its current format of recycled RPETG plastic, with 16 strings, hybrid (acoustic & electric), rechargeable and reprogrammable.
The name ‘Knurl’ (metal slippery surfaces with its tiny grippy tips) came up from the use and the apprehension of the human body movement, using friction on the knees to change the position of the instrument on the performer’s body. Likewise as the crickets, we have to find a grip in our legs in a knurling surface (kenny patterns from angle and/or grossed lines in a smooth metal surface ) to produce ‘energy’.
It can charge itself by photovoltaic solar system, being one of the pioneers electric instruments sustainably designed with energy free and self-contained circuits[3]. A solar pedal separates its acoustic functions of absorption and reflection, storing the electronics and being portable to adapt the solar panels to the light sources. By a switch, you can change the functions of the solar panels, treating it as an analogue sensor for music parameters.
The design reinvented issues and sound range of a classic violoncelo, while offering a new array of possibilities radically expanding the instrument’s range and sonic palette. Four buttons define specific triggers from a selective mode of performance (synthesis, programming, analogue & detection mode). It’s possible to play different modes at the same time by changing the settings on a capacitive switch button, made on the Bela trill Craft[2]. Those buttons can select and trigger synths or code functions that process the sound of both the cello and, potentially, the audience. This tool is geared for live electronic performances, in which the performer and audience can interact as the music unfolds in real time. An oled screen shows to the performer the code that has been reprogrammed.
On Knurl, to create chords, different multiphonics or combinations of harmonics are possible in a melodic instrument. The tuning system can be adjustable to your aesthetic purposes (without damaging the instrument). Inclusion is certainly also a matter. The bridge is designed depending on the performer’s height. The rotational angle of the instrument, which impacts the sound, is readily manipulated by the pressure of the player’s knees and feet. The principle of the traditional scordatura (a tuning system scheme) of the cello is the same:
The strings can be tuned by intervals of fifths, such a conventional cello:
I consider Knurl as a non-tuning instrument, where the search for harmonic, color and gestalt connections become part of an intuitive discovery.
Evaluating an Instrumental live code with Bela platform
The practice of live code music is defined by its flexibility and rematerialization. By building a malleable structure, a block of code can be summarized, redefined, formatted, reshaped, reposed, rewired and, therefore, creates its new definition, by changing its behavior, constitution and achievement (those terms will be further developed). But should the application of flexibility and manipulative tasks become more important than the flow of the composition itself?
Great composition teachers mentioned very often to me that the rules of your processings should guide you, not contradict you. Sonic tools have been overlapping the subjectivity of the sonic process, and that is something that also happens in live code music by applying language to define sonic relationships. The Western music culture is used to depend and be supported by symbols and numbers. In a world where technology and science dominate, It’s difficult to understand that there are cognitive dimensions in the experience of appreciation that can’t be summarized by data. Numbers, letters, objects can bother the real concept of ‘transparency’ in a musical performance. Should we aim Transparency? Which advantages does a performance have by showing the process of composition?
Regarding those challenges, I aim to design a space and performance where sonic performance still deals abstractly with the unforeseen, the mistake, the challenge, the search of flow. Those things are fundamental to build the sense of ‘spectator’, where the audience feels, supports and connects with the performer.
A programming structure is a tool, it should be simple and not the reason of your composition, otherwise it can become an exhibition method. In most of the programming performances, a great part of fear, error, development of layers is replaced by the beauty & safety that technology and science offers us[4]. The relationship between sonic layers and materials are merely delved into a subjective complex of cognitive interpretation, restricting the performance to mostly a linear development without interpolation and dialogue with previous sonic events. Even with a diversity of formats and layouts, the sonic approach is pretty similar to each other: the layers of code syntax are gradually superimposed. Time & flow offer space for typing skills, meanwhile the performer loses control during 10 secs of its sound generation.
At ICLC 2020 (Limerick), I collected some considerations from practitioners with a background in music education about the music development of live coding practices. They missed variations of density, development of layers, different approaches to repetition (usually this last one, is the one more unappreciated by them and the academics). As Mattia mentioned briefly to me in a Algorave, “[…] as a pianist, its seems to me that live coding and it’s a great fascination but also a restriction to patterns of popular music and techno dance”[5]. The evaluation of what is an ‘authentic’ idea is neglected: In a culture influenced by the open source movement, it doesn’t matter if the sound samples aren’t yours or an idea is copied, pasted or modified: what it really matter is what you do with it: “we want to see how are you listening and what are your decisions”[6]. For academic institutions, this seems to be one of the biggest issues of live coding, since sound isn’t treated as an archive or belonging. And this is what institutions and academies hope to sustain.
With this overview of issues of live coding practice, it seems more than a fruitful challenge to build an encoding system in a musical instrument. Therefore, the chosen approach should carefully design a petrification of the main purpose: the instrument has to allow the possibility of recycling its subjects.
My starting point became the danish research about MIR (Music information retrieval) in live code practice [7]. Based on this reference, I defined the application of 4 manipulative definitions in the instrument that recycle its definition of functions: repurposing, reposing, remixing and rewiring live:
- Repurposing is associated with the redefinition of purpose in a sound by analysis, synthesis or complex calculations of algorithms in an audio sample/sound streaming. You could consider as an example: the practice of management buttons and sound trackers to switch the function of systems.
- Reposing, in the case of this project, refers to the idea of using a repository or an archive not as an untouchable historic artifact, but as a changeable and reposing object inside of it. Buffer samples are replaced and recycled.
- Remixing means to provide systems that allow an analogue/digital manipulation of the material, treating the object (sound) as a material of modelling: shaping, scaling, resizing, cutting, extending and ranging by real-time analysis calculations. 4 modes of compositional methods are applied to facilitate this manipulation.
- Rewiring because it changes the direction and hierarchy of sound sample/sound generation approaching behavior, constitution and achievement delineate a new manipulation of layers.
Notice that by remodeling a block of code or a buffer sample, you are anyway redefining a material. In a sonic process where language and process are part of a live manipulation of definition (data) and result (sound), it’s also fundamental to work for a better quality of interaction. And the union of both a responsibility from the performer and its aesthetic decisions in its adaptation. An interactive system is defined by:
- Audience: The public is welcome to manipulate and compose the processing of the instrument in a digital platform.
- Environment:The light of the space changes the parameters of the Analogue mode.
- Other musicians: collaborators could also interact with the instrument by sending to specific port OSC messages.
Future investigations
The awareness and usage of different methodologies of repetition, density and sound generation seems to enhance and impulse a malleable approach but also gives back a lot of responsibility to the performer and its practice. Simplicity in your main structure doesn’t offer you less, it offers you more focus and discipline to attenuate variations and its connections.
Further developments will be taken after my project about sonic-media concerts, exploring live coding practices with public participation. A gitbook open source, called ‘digital detection’, is also coming up in the coming months. If you want to hear more from it, a CD Knurl’s solo (Un-bow: http://kck.st/3ej87TF ) has just been released on kickstarter.
In the future I would like also to optimize the detection mode turning it into a Machine listening system. I’m always investigating how to make less footprint impact, exploring more organic/ sustainable materials into the instrument.
Invitation
If you resonate with this research, the concept and the story, I would love to hear from you. My project is open to collaborations and implementations. If you live around the Netherlands I’ll be during this year grounding a smart hub space for artists and innovative practices. In this space we’ll be developing new audiences settings, different approaches to sonic art, city solutions, design and artistic projects. You’re welcome to drop by or participate in one of the artistic residences in 2021.
It has been an amazing experience to integrate multiple knowledges in order to design a single concept. However, if a reprogrammable instrument should be improved by other perspectives and developments, I can’t be its only instrument builder, performer & composer forever. I hope in the future to find fruitful collaborations between media artists and performers to share and contribute the achievements of this research globally.
More about Knurl:
www.rafaeleandrade.org/knurl
More about reprogrammable performance:
https://www.rafaeleandrade.org/
CD solo:
http://kck.st/3ej87TF
More about Knurl:
www.rafaeleandrade.org/knurl
More about reprogrammable performance:
https://www.rafaeleandrade.org/
CD solo:
http://kck.st/3ej87TF
Notes
[1] Ethernet, microcontroller, speakers, sensors, microphone are attached to its body.
[2] A capacitive sensor of bela platform launch on kickstarter 2019.
[3] All the components are inside the cello (microphone, computer, speaker,pot and sensors).
[4] By observing my live code colleagues, I notice that the fear is also present in their performance.but it is really not perceptible if the machine is filtering your emotional ‘human side’.
[5] Andrade Rafaele. Personal communication with Mattia Patterna at ICLC 2020. Limerick, Ireland. Unpublished data.
[6] McLean, Alex. Weaving lecture at ICLC 2020. Limerick, Ireland.
[7] Xambó, Anna, Lerch, Alexander; Freeman, Jason. Music information retrieval in live coding: a theoretical framework. Computer music journal, 42:4, pp.9-25, Winter, 2018 doi:10.1162/COMJ_a00484
About the author
Rafaele Andrade is a composer and performer in the field of modern/experimental music. With a background in Conducting, Composition, Cello, live coding, and Sonology, she applies those knowledges in her artworks, the reflection and practice of fair music distribution, social inclusion & sustainability. Rafaele has a passion for innovative and social initiatives, producing during her young career an orchestra of Brazilian music at 17 years old and at 22, curating a UNESCO project to promote Latin American women composers (Rádio Delas). She is a member of NLCL(Netherlands) and (Brazil) and the creator of Knurl, a reprogrammable, hybrid, and solar-powered cello.
