Audioreactive Shaders

Resonon can drive GPU graphics from the same code that drives your sound. You write a fragment shader as a string, hand it to Visual(...), bind its uniforms to live signals and audio meters with the familiar << operator, and route it to the display with >>. The shader renders in a browser canvas, fed live over Resonon’s WebSocket server — so the visuals move in lockstep with the music.

The whole loop is live: edit the shader in VSCode, re-execute, and the picture hot-swaps without dropping a beat.

let frag = "#version 300 es
precision highp float;

uniform float iTime;        // transport time in seconds (built-in)
uniform vec2  iResolution;  // canvas size in pixels      (built-in)

out vec4 fragColor;

void main() {
  vec2 uv = gl_FragCoord.xy / iResolution;            // 0..1 across the canvas
  float wave = 0.5 + 0.5 * sin(iTime + uv.x * 6.2831);
  fragColor = vec4(uv.x, uv.y, wave, 1.0);
}
";

let v = Visual(frag);
v >> screen;
PLAY;

Run that and nothing appears yet — there’s no display attached. Let’s set one up, then take the example apart.

The Workflow

Visuals render in a browser, served by Resonon’s own server. So before the shader can show up, two processes need to be running alongside your editor. Open two terminals:

# Terminal 1 — start the engine + WebSocket server
resonon server

# Terminal 2 — open the browser render page (must be run after the server is up)
resonon visuals

resonon visuals checks that a server is listening (default 127.0.0.1:5555, override with --host/--port) and opens your default browser to the render page. Press f or click the canvas to go fullscreen.

With both running, the primary workflow is the same as the rest of Resonon: select the shader block in VSCode and execute it (Cmd/Ctrl+Enter). The shader and its bindings are pushed to the browser and start rendering. Tweak the shader string, select, and execute again — the picture hot-swaps live.

Note

Visual and screen are always available — no use import needed. The shader source itself is a string, so you can build it however you like (a literal, a function that returns one, etc.).

Your First Shader

Here is the minimal shape again, in isolation:

let v = Visual(frag);   // compile a fragment-shader string into a Visual
v >> screen;            // route it to the display
PLAY;                   // run the transport (drives iTime)

Visual(source) takes a GLSL-ES 3.00 fragment shader as a string and returns a Visual. The shader is compiled in the browser, not by Resonon, so a syntax error surfaces later (see Live Editing) rather than at construction.

v >> screen routes the visual to the display, exactly as track >> master routes audio. There is one screen, so routing a new visual replaces whatever was showing.

PLAY matters here: the built-in iTime uniform follows the engine transport, so without PLAY the shader sits frozen at time zero.

Built-in Uniforms

Four uniforms are supplied to every shader automatically — declare them and use them, no binding required. They come from the engine transport, so they stay beat-synced and freeze when you STOP (and resume in sync on the next PLAY).

Uniform	Type	Meaning
iTime	float	Transport time in seconds
iResolution	vec2	Canvas size in pixels (supplied by the browser)
iBeat	float	Continuous beat position
iCycle	float	Continuous cycle position

These names are reserved — trying to bind one (v.iTime << ...) is an error, because the engine already supplies it.

Binding Uniforms

Any other uniform you declare in the shader is yours to drive. Bind it with <<, just like sending a pattern or signal to a track. The value can be a constant, a signal, or a pattern:

v.brightness << 0.7;                  // a constant
v.energy     << Sine(8).range(0.2, 1.0);   // an LFO, remapped to 0.2..1.0
v.hue        << Saw(16);              // a slow ramp, 0..1 over 16 cycles

Signal periods are measured in cycles, so Sine(8) completes one oscillation every 8 cycles. Use .range(lo, hi) to remap a signal’s default range to whatever your shader expects. Re-binding the same uniform replaces the previous source.

Every bound uniform is sampled once per rendered frame, off the audio thread — binding a signal never touches the real-time path.

Going Audioreactive

The point of all this is to react to sound. Every audio track and the master bus expose two live meter feeds — .rms (average level) and .peak — and you can bind them straight to a uniform:

v.kick  << drums.rms;     // a track's level
v.level << master.peak;   // the whole mix

Here is a complete, runnable example: a CR-78 beat, two LFO-driven uniforms, and a kick uniform wired to the drum track’s live RMS so the shader flashes on every hit.

use "std/signals" { Sine, Saw };
use "std/instruments" { Sampler, Kit };

let frag = "#version 300 es
precision highp float;

uniform float iTime;        // transport seconds   (built-in)
uniform vec2  iResolution;  // canvas pixels        (built-in)
uniform float energy;       // 0..1 brightness      (LFO-driven)
uniform float hue;          // 0..1 color shift      (LFO-driven)
uniform float kick;         // live drum RMS         (audio-driven)

out vec4 fragColor;

vec3 hsv2rgb(vec3 c) {
  vec3 p = abs(fract(c.xxx + vec3(0.0, 2.0 / 3.0, 1.0 / 3.0)) * 6.0 - 3.0);
  return c.z * mix(vec3(1.0), clamp(p - 1.0, 0.0, 1.0), c.y);
}

void main() {
  vec2 uv = (gl_FragCoord.xy * 2.0 - iResolution) / min(iResolution.x, iResolution.y);
  float r = length(uv);
  float rings = 0.5 + 0.5 * sin(r * 10.0 - iTime * 2.0);
  float bright = mix(0.1, 1.0, energy) * rings + kick * 2.0;
  fragColor = vec4(hsv2rgb(vec3(hue + r * 0.2, 0.8, bright)), 1.0);
}
";

let v = Visual(frag);

// LFO-driven uniforms — periods are in cycles.
v.energy << Sine(8).range(0.2, 1.0);
v.hue    << Saw(16);

// A beat, so the transport runs and there is audio to react to.
let drums = AudioTrack("drums");
drums.load_instrument(Sampler(Kit("CR-78")));
drums << [bd [sd _ _ bd] bd sd];

// Live audio: the shader's `kick` uniform follows the drum track's RMS.
v.kick << drums.rms;

v >> screen;
PLAY;

With resonon server and resonon visuals running, executing this block plays the beat and renders concentric rings that breathe with the LFOs and flash on every kick.

Live Editing & Errors

Re-running v >> screen is the moment everything updates — it pushes the current shader source and bindings to the browser. That is what makes live editing work: tweak the shader string or change a binding, then re-execute the block to hot-swap it.

Because the browser compiles the shader, a GLSL mistake can’t crash the engine. When a shader fails to compile, Resonon reports a diagnostic on the v >> screen line with the GLSL error log, the audio keeps playing, and the last shader that compiled keeps rendering. Fix it and re-execute to recover.

Notes & Limits

• Shaders are GLSL-ES 3.00 fragment shaders rendered with WebGL2. Write #version 300 es and a precision qualifier at the top, and output to an out vec4.
• Bound uniforms are scalars (float). The built-in iResolution is the only vec2.
• This page covers writing shaders, not GLSL itself — for the shading language, see any GLSL-ES reference.

Next Steps

• Signals & Automation — the LFOs and envelopes you bind to uniforms
• Tracks & Master — where .rms / .peak meter feeds come from
• Live Workflow — the select-and-execute loop in depth