Chapter 6
Building It
From analysis to practice: RALF, co-regulated music, and the ethical line
I should tell you something about who is writing this.
I’m a DJ. I’ve been behind the decks for years, mostly house, mostly Chicago and Detroit lineage, mostly dark rooms where the point is not to be seen but to move. The scenes in this book are from my life, not thought experiments.
I’m a capoeirista. I train in a lineage that takes the berimbau seriously, as a governance instrument rather than a prop. Shifting the rhythm to change the game is something I have done dozens of times. It works, and it works so reliably that it stops feeling remarkable, which is exactly when you should start paying attention to why it works.
I’m also a software engineer. React, Python, real-time systems, machine learning pipelines. I build things for a living. And this combination of practitioner of the forms and builder of tools is why this book exists.
Because at some point, the question stopped being “what are these forms doing?” and became “can I build something that does it too?”
What Systems Get Wrong
The obvious first attempt is a motion-triggered sampler. Wave your hand, hear a sound. Kick your foot, trigger a drum hit. Map some joints to some parameters, throw it on stage, and call it “interactive.”
I’ve seen these systems. I’ve built early versions of them myself. They are impressive for about ninety seconds. Then you notice that the relationship between the mover and the sound only runs one way. You do something, the system reacts. Your body becomes an input device and the sound becomes an output, a fancy light switch.
That is not what happens in a roda or on a dance floor or in a jazz combo. In those systems, the sound shapes the movement and the movement shapes the sound, and the loop runs continuously. The participants establish common ground through repetition, and variation becomes expressive against that ground. You can fall out of coherence and find your way back. The sound is not just responding to you. It is shaping what you do next, which is shaping what it does next.
The loop is the point, and every form in the previous chapters is built on it. The berimbau in the roda reshapes the game it is reading. The players adjust to the music, the music adjusts to their adjustment, and the game keeps changing inside that loop. The DJ reads the floor and shapes it at the same time. The floor responds, and the response informs the next selection. The primo drummer in bomba reads the dancer’s piquete and answers it, and the answer changes what the dancer does next.
A system that misses the loop has missed what these forms are actually doing.
Musicking With a Machine
Something I keep noticing about the scenes I’ve spent my life inside.
On a dance floor, the music is not really an object being transmitted. It is something happening, between the DJ and the room and the bodies and the sound system and the hour of night. In a roda, what the berimbau plays is inseparable from what the players are doing in the circle. In a bomba, the dancer and the primo drummer are making the piece together, in real time, and neither one of them could write it down afterward.
In all of these, the music does not exist anywhere outside what is happening in the room. No one is executing a work. A set of relationships is coming alive for as long as the thing is going, and when it ends, the music ends with it.
Christopher Small has a word for this. In Musicking, he argues that we have been making a category error for about three hundred years, treating music as a noun when it is really a verb. The music is not the score. It is not the recording. It is the activity, the event, the web of relationships that comes alive when people gather to do it. He calls this musicking, deliberately awkward, to keep us from collapsing it back into object.
The Western concert tradition, in his telling, is built on the opposite instinct. Stravinsky wanted a performer who added nothing, only transmitted. Under that logic, the performer should be as invisible as possible so the composition can be revealed. The audience should sit still and receive. Everything outside the work is interference.
The forms I grew up inside never went down that road. The relationships are not in the way of the music in a roda. The relationships are the music.
This is the frame I was looking for.
Because the question with RALF is not really “how do I get a computer to respond to movement?” That problem is solvable. The harder question is: what kind of thing is happening when it works? If the dancer and the sound environment are in a real loop, shaping each other, surprising each other, developing something together over time, what do we call the event? The word for it is musicking. And if one of the participants is a generative model, driven by code and training data, that doesn’t remove the event from the category. It adds a participant.
Call it musicking with a machine.
The machine brings things a single body can’t bring alone. Aggregated taste from thousands of hours of recorded sound. Structural ideas borrowed from traditions the dancer has never studied. Combinatorial range. A kind of breadth no single lifetime affords. These are real contributions. Used well, they let a small collaboration reach for sounds a small collaboration otherwise could not reach.
The dancer brings what the machine has no access to. A present tense. Weight. Intention felt in the room. The quality of a phrase: its softness or jerkiness, whether it is loosening or compressing, what it means this time that it did not mean last time. These qualities are what a score cannot hold. They are why a great performance of a composition is not the same event as a lesser one, even when the notes are identical. They are, in Small’s sense, the relationships. And they are carried by a body that is actually there.
Chess players have a name for a human and a machine competing as a single team: a centaur. The image is worth borrowing, because a centaur is not a human giving orders to a horse; it is the two becoming one kind of animal, with capacities neither has alone. That is the design goal. The generative side of RALF is not the composer, and the dancer is not the performer executing a composition. Both are musicking. Both are shaping what the event becomes. The machine proposes; the body answers; the proposal changes based on the answer. Composition and performance, which the concert tradition spent centuries prying apart, come back into the same action.
This is why the older forms matter so much to this work. The roda, the dance floor, the bombazo: they never separated composition from performance in the first place. They stayed inside musicking the whole time. What I am trying to build is not new; it is an extension of a very old practice, into a new kind of partner.
The instinct everywhere else in technology is disruption: clear the old thing away so the new thing has room. This is the opposite motion. You reach back into a living practice precisely in order to carry it forward, into a partner it has not met yet. The tradition is not raw material for the innovation; the tradition is what the innovation is in service of.
RALF: Making Relationality Audible
I’m building a system called RALF, short for Relational Audio-visual Learning Framework. At the technical level it is a pipeline. A webcam feeds into pose detection. Pose detection feeds into gesture recognition. Gesture recognition sends control messages to Ableton Live. Camera sees body, software interprets movement, sound responds.
But the design intent is specific, and it comes directly from the forms.
The question I want RALF to answer is not “what is the body doing right now” but “what should the sound do next, given what the body just did, given what the sound just did, given the history of this exchange.” Those are different questions, and a system designed around the second one behaves very differently from a system designed around the first. The first question produces detection; the second produces conversation.
A producer creates a programmable score that will shape the encounter: a gesture vocabulary, a set of mappings, a palette of sonic possibilities meant to inspire and surprise a dancer. The dancer enters that space and moves through it, and the dancer’s movement shapes the music as much as the music shapes the dancer. Neither side controls the outcome. What emerges belongs to the exchange. And over time, something develops inside that exchange that was not present at the start: shared ground, surprise, the feeling of being heard.
The design goal is captured in a single phrase. RALF makes relationality audible. The sound is not a representation of movement and it is not background atmosphere. It is one side of a conversation whose other side is movement. When the exchange is coherent, you can hear that. When it fragments, you can hear that too. The state of the relationship lives in the sound.
There are three layers of ambition here, and I want to be honest about where each one stands.
Layer one: one mover and responsive sound. A single body in dialogue with a sound environment. The question I am trying to answer in this layer is whether the mover feels heard, in the way a good musician feels heard by a bandmate, not measured by an instrument. If the first layer does not produce that quality, nothing built on top of it will matter. This layer is working. It is rough and early, and there is a lot of refinement still to do, but the core experience of being in conversation with sound is present.
Layer two: the composed space. Someone designs the gesture vocabulary, the mappings, the sound palette. They make decisions about what movements will matter, what sounds will respond, what constraints will shape the interaction. This is the composer’s conversation with the mover: here is the space I built for you, discover what it affords. This layer exists in prototype. I have composed for it, other people have moved inside compositions, and the design language is still forming.
Layer three: multiple bodies. This is where it gets interesting, and where RALF connects most directly to the forms. Multiple people move in the same responsive environment, and each person’s movement shapes the shared sound. Coordination between movers becomes audible. When two people are in sync, the sound coheres. When they are in tension, the sound reflects that. When they are in counterpoint, doing different things that somehow fit, the sound reveals the relationship between them. This layer does not yet exist; it is the research horizon.
The design move across all three layers is the same. Dance is the intelligent system; technology is the listener. The body leads and the machine follows. This is the bomba principle brought into computation: the body as the source of the music, and the system as the primo drummer trying to keep up.
Co-Regulated Music
Here is the bigger frame.
In any room where people share space, there is a collective state. How much attention is shared, whether the energy is rising or dropping, whether people are converging or fragmenting, whether the tension is productive or corrosive. The state is real and consequential, and it is usually invisible. Someone might sense it intuitively (the room feels off) but there is no shared medium that makes it available to everyone in the room at once.
Unless there is music.
A DJ reading a dance floor and adjusting the music to the room’s energy is doing something precise. She is making the collective state audible and shaping it through sound. The DJ is in feedback with the room. The room moves, the DJ responds, the room responds to the response, and the loop runs as long as the music is playing.
A berimbau player governing a roda is doing the same thing in a different context. So is a jazz rhythm section supporting a soloist. So is a lead singer adjusting the intensity of a call based on the energy of the chorus’s response. The medium changes, the context changes, but the underlying move is the same: a person attending to the relational state of bodies in a space, and using sound to shape that state in return.
Call this co-regulated music: sound environments that listen to the relational state of bodies in a space and shape that state through what they play. Sound as a responsive medium for coordination.
The practitioners of the forms have been doing this for generations. The DJ’s art of reading a room, the rhythm section’s feel for a soloist, the berimbau’s governance of the game. These are practiced, refined models of sonic co-regulation. They have just never been formalized as transferable design knowledge.
RALF is a research platform for this larger idea. The same pipeline that senses body state, recognizes patterns, sends control messages, and shapes sound, works for one body or twenty. The difference between a solo RALF session and a co-regulated meeting room is not the technology; it is the theory layer. The model of what co-regulation means in each context, and the sound design that serves it, has to be built fresh for each application.
The applications extend beyond artistic contexts. A meeting room where the collective state is made available through ambient sound. A classroom where a teacher can hear the room’s attention without having to survey it. A rehabilitation clinic where the relationship between therapist and patient has a sonic dimension that helps both of them. I mention these to indicate the direction, not to over-promise. None of them exist yet. The principle is music as relational infrastructure, and the forms have demonstrated that the principle works. The open question is whether it can be extended into contexts where the cultural lineage is not already in place.
The Ethical Line
There is a version of this technology that I refuse to build.
Picture a system that monitors bodies in a room and reports back to management about who is engaged and who is checked out. Or a system that scores participation for someone’s quarterly metrics. Or a system that uses sensing as surveillance and optimization as control, so that the bodies in the room become objects of measurement for an observer outside it. That is the bounded-self version of co-regulated music. It would be easy to build, and it would be a betrayal of everything these forms teach.
The forms do not surveil. The berimbau does not generate a performance report on player quality. The DJ does not send metrics to the club owner about how efficiently the floor converted passive listeners into active dancers. The sound is in the room, for the room. It is a shared medium that everyone is inside, not an extraction pipeline that someone is watching from outside.
Any system I build has to hold this line. Measurement has to stay in service of flourishing rather than control, feedback has to be something felt in the room rather than filed in a report, and the sound has to belong to the people in the space rather than to an observer watching them through it.
The distinction is simple in principle and hard in practice. The question is always: who benefits from the information? If the answer is the people in the room, through a medium they can all feel, the system is participating in co-regulation. If the answer is someone outside the room, through a report they can act on, the system has slid into surveillance. The berimbau player is inside the roda. The DJ is on the dance floor with the dancers. The rhythm section is part of the band, not a measurement layer evaluating it. The governor is always inside the system being governed, subject to the same forces, transformed by the same exchange.
The forms teach that constraint is what makes freedom possible. The first constraint on this technology is that it must remain in service of the people it touches.
This is not abstract worry. The extractive version is easy to build. Imagine a rehabilitation clinic. A patient is relearning to walk. The same motion capture and responsive sound system that makes her feel heard in relationship with her therapist could be repurposed as a surveillance layer, with data tracked, metrics sent to insurance, recovery rates optimized for cost-effectiveness. The tool would not change; the intent would. The sound that made her effort audible would become a monitoring instrument, and she would feel the difference even if no one told her. The technology would be identical. What changed would be the ethics, and the change would be felt in the room.
This is why the ethical line is load-bearing rather than decorative. The forms, as the last chapter argued, are wisdom traditions that have worked out how to coordinate without coercing, hold conflict without rupture, and let difference exist within shared frame. The wisdom is inseparable from the structure of accountability the forms hold themselves to. Extract the mechanism without that accountability structure and you do not get co-regulation. You get a more sophisticated version of exactly what the forms are an alternative to.
Where This Sits
RALF is a research platform. Co-regulated music is a program of work. Neither is a finished product. What exists right now is a motion capture pipeline, a gesture recognition engine, and a set of mappings to sound. That is the simplest version of the first layer, one body in dialogue with a responsive environment.
The forms are the theory, and they have been developing it for centuries. My job is translation, taking what the roda knows and making it available as design knowledge to someone building a conference room, a classroom, a clinic. The way the arch traveled from Mesopotamia to Rome to the rest of the world: not by erasing its origin, but by understanding the principle clearly enough to build with it in new contexts.
That is the work, and the next chapter is about what it could look like.