An Altar de Muertos That Listens to the DJ
Halloween lands two days before Día de Muertos. So for the party at my place this year, the screen didn't show pumpkins. It showed an ofrenda: papel picado, calaveras, cempasúchil, candles, and petals falling across the whole thing, changing with every song the DJ played.
The altar is the visible result. The experiment underneath is something else.
The real experiment: directing, not drawing
I didn't sit down to craft these visuals by hand. I wanted to see how far I could get by orchestrating AI agents: giving direction, judging the output and steering, while the agents did the building inside TouchDesigner.
The bridge is an MCP server for TouchDesigner. A Web Server DAT inside the running project exposes it on a local port, and an AI coding agent (Claude Code, in my case) connects to it. Through that connection the agent can list and create nodes, set parameters, write GLSL and Python, run scripts inside the live project, and pull a rendered frame back to look at its own work.
That last part matters more than it sounds. The MCP only reports hard errors, not warnings, and a broken Script CHOP can fail without a word. So the loop that works is: the agent makes a change, grabs the image, looks, and fixes. I mostly stayed one level up: "more fog when the music climbs", "make the photos stand on the tiers", "one look should feel like a Posada engraving".
So what you see here is a fair sample of what today's MCPs and models deliver when a person directs them, rather than a showcase of my own shader-writing.
What the agents built
One shader, three tiers. The altar is a single GLSL TOP. There's no 3D scene and no imported models: the tiers, papel picado, sugar skulls, marigolds, candle flames, smoke and petals are all drawn in the pixel shader. Every element is a uniform, so fog, petal density, wind and flicker are sliders, and each layer (papel, calaveras, cempasúchil) fades in and out on its own.
Photos on the ofrenda. An ofrenda needs photos of the ones you remember. This one holds four: Pachita, my grandmother; Tío Jaime; Nikita, our dog; and Luigi, our turtle. Two stand on the middle tier and two on the bottom one. The shader wouldn't take a fourth wired input through the API, so the agent packed the photos into a 2×2 atlas and samples it tile by tile. It's a good example of the agent working around a limit instead of stopping.
It follows the music. The party music ran through DJBox, the DJ app I built on Pure Data for my DDJ-200 controller. TouchDesigner reads DJBox's event stream (song titles, play states, the crossfader) and an autopilot does the rest:
- Every new song picks a new look and a new scene, meaning which of papel, calaveras and cempasúchil are showing. One look is a black-and-white engraving that tips its hat to José Guadalupe Posada, the printmaker behind La Catrina. Another mixes in the Pulsar glitch.
- Intensity changes the weather. The autopilot compares a short (1.5 s) and a long (20 s) average of the master output. When the music climbs and holds for three seconds, fog, petals, wind and flicker go up with it.
- If DJBox is offline, it listens for gaps instead: at least 1.5 seconds of silence counts as a song change.
Phone control. Every patch has a Controls page that shows up in Luna, my phone app, so looks, presets and the photo fader can be changed from across the room.
The recording
Ninety seconds of one song, recorded straight from the output. The music comes off a vinyl record: we recorded the song directly from the turntable with our own software, and the altar reacts to that recording. It moves through the candlelit altar, the Posada engraving and the Pulsar glitch:
What I take from it
My job on this project was taste and direction: what the altar should feel like, what to keep and what to throw away, when something was good enough for the party. The agents wrote the shader, wired the network and worked around TouchDesigner's quirks. For a personal piece made in a few days, that split worked, and the result shows where the tools stand right now: their quality, and their rough edges.
Este altar está dedicado a Pachita, mi abuelita; al Tío Jaime; a Nikita, la perrita; y a Luigi, la tortuga.