Sonifying a Line Chart with the Web Audio API

A blind user can grasp the shape of a trend — the climb, the dip, the sudden spike — by hearing it as rising and falling pitch, but only if you render the data to sound rather than expecting them to read a picture.

Sonification maps each data value to the frequency of a tone and plays the points in order, so a rising line becomes a rising glissando and an outlier becomes an audible jump. This page is part of alternative representations for charts; it adds an audio channel alongside the text summary and the data table, using the shared series model those channels also read. Audio conveys shape superbly and exact values poorly, so it complements the table rather than replacing it.

Diagnostic checklist

Does audio start on load or on hover, breaking the browser’s user-gesture requirement and startling users?
Is a fresh AudioContext created per play (leaking contexts) instead of one reused and resumed?
Are values mapped linearly to frequency, producing an unnatural ramp, rather than to perceived pitch?
Are the play/pause controls real buttons that are keyboard operable and labelled, or click-only affordances?
Does playback ignore prefers-reduced-motion, autoplaying a moving audio sweep for users who asked for calm?
Is there any announced start/end cue, or does the tone play with no indication of the value range it spans?

Each value maps to a pitch and the points play in sequence, so the ear hears the same rise, dip, and spike the eye sees in the line.

The encoding principle for a line chart with the web audio api: pair every hue with a shape so the category reads without colour, and keep each swatch above 3:1.

Broken vs fixed

// ❌ BROKEN: autoplays, leaks a context, and maps value straight to Hz.
function sonifyOnLoad(series: Series): void {
  const ctx = new AudioContext();        // new context every call → leak
  const osc = ctx.createOscillator();
  osc.connect(ctx.destination);
  osc.start();
  series.points.forEach((p, i) => {
    // A11Y: linear value→Hz sounds unnatural; and this runs with no user gesture,
    // so the browser suspends the context and nothing plays anyway.
    osc.frequency.setValueAtTime(p.y, ctx.currentTime + i * 0.2);
  });
  // no stop, no pause, no reduced-motion check
}

// ✅ FIXED: one reused context resumed on a gesture, perceptual pitch, real controls.
class Sonifier {
  private ctx: AudioContext | null = null;
  private osc: OscillatorNode | null = null;
  private gain: GainNode | null = null;

  constructor(private series: Series, private facts: SeriesFacts) {}

  // A11Y: must be called from a click/keydown handler so the browser unlocks audio.
  async play(): Promise<void> {
    if (!this.ctx) this.ctx = new AudioContext();
    if (this.ctx.state === "suspended") await this.ctx.resume();
    // A11Y: honour reduced-motion — offer a static readout instead of a sweep.
    if (matchMedia("(prefers-reduced-motion: reduce)").matches) return;

    const t0 = this.ctx.currentTime + 0.05;
    const step = 0.18;                       // seconds per point
    this.osc = this.ctx.createOscillator();
    this.gain = this.ctx.createGain();
    this.osc.type = "sine";
    this.osc.connect(this.gain).connect(this.ctx.destination);
    // Small fade in/out avoids clicks at start and stop.
    this.gain.gain.setValueAtTime(0.0001, t0);
    this.gain.gain.exponentialRampToValueAtTime(0.2, t0 + 0.03);

    this.series.points.forEach((p, i) => {
      // PERF: schedule all frequencies up front on the audio clock — no per-note timers.
      this.osc!.frequency.setValueAtTime(this.toPitch(p.y), t0 + i * step);
    });
    const end = t0 + this.series.points.length * step;
    this.gain.gain.setValueAtTime(0.2, end - 0.03);
    this.gain.gain.exponentialRampToValueAtTime(0.0001, end);
    this.osc.start(t0);
    this.osc.stop(end + 0.02);
  }

  pause(): void {
    // Stop cleanly; a fresh oscillator is created on the next play (they are single-use).
    this.osc?.stop();
    this.osc = null;
  }

  // Map value → PITCH on a log scale (musical), spanning ~C4..C6.
  private toPitch(y: number): number {
    const lo = this.facts.min.y, hi = this.facts.max.y;
    const norm = hi === lo ? 0.5 : (y - lo) / (hi - lo); // 0..1
    const minMidi = 60, maxMidi = 84;                    // two octaves
    const midi = minMidi + norm * (maxMidi - minMidi);
    return 440 * Math.pow(2, (midi - 69) / 12);          // MIDI → Hz (perceptual)
  }
}

The two decisions that matter most are audible. Mapping the normalised value through a MIDI-note scale before converting to hertz means equal perceived pitch steps for equal data steps, because pitch perception is logarithmic in frequency; a raw linear value-to-hertz map crushes the low end and stretches the high end into a siren. And resuming a single reused AudioContext inside a user-gesture handler is what actually makes sound come out — browsers start every context suspended and only unlock it after a genuine interaction.

Step-by-step fix

Reuse one context. Create the AudioContext lazily and keep it; resume it inside the play handler rather than constructing a new one each time.
Require a gesture. Wire play() to a button’s click/keydown so the browser unlocks audio; never autoplay.
Map to perceptual pitch. Normalise the value to 0–1 against the series range, then to a MIDI note range, then to hertz — not a raw value-to-hertz map.
Schedule on the audio clock. Set every note’s frequency with setValueAtTime ahead of time; do not drive notes from setTimeout.
Fade edges. Ramp gain up at the start and down at the end so tones do not click.
Respect reduced motion. If prefers-reduced-motion: reduce is set, skip the sweep and offer a static value readout instead.
Expose labelled controls. Provide real, keyboard-operable play and pause buttons with clear accessible names and a start/end value cue.

// Wire keyboard-operable controls to the sonifier.
function mountAudioControls(root: HTMLElement, sonifier: Sonifier, facts: SeriesFacts): void {
  const play = document.createElement("button");
  play.type = "button";
  // A11Y: announce the range so a listener knows what low and high mean.
  play.setAttribute("aria-label", `Play audio chart, values from ${facts.min.yFormatted} to ${facts.max.yFormatted}`);
  play.textContent = "Play audio";
  play.addEventListener("click", () => { void sonifier.play(); }); // gesture unlocks audio
  const pause = document.createElement("button");
  pause.type = "button";
  pause.textContent = "Pause";
  pause.addEventListener("click", () => sonifier.pause());
  root.append(play, pause);
}

Verification

// Assert the pitch mapping is monotonic and bounded within the intended range.
const s = new Sonifier(series, deriveFacts(series));
const lo = (s as unknown as { toPitch(y: number): number }).toPitch(series.points[0].y);
const midiToHz = (m: number) => 440 * Math.pow(2, (m - 69) / 12);
console.assert(lo >= midiToHz(60) - 1 && lo <= midiToHz(84) + 1, "pitch must stay within C4..C6");
// Higher value must yield higher (or equal) pitch.
console.assert(
  (s as unknown as { toPitch(y: number): number }).toPitch(deriveFacts(series).max.y) >=
  (s as unknown as { toPitch(y: number): number }).toPitch(deriveFacts(series).min.y),
  "max value must map to the highest pitch"
);

In DevTools, open the Media / Web Audio section and confirm exactly one AudioContext exists after several play/pause cycles — a growing count means you are leaking contexts. Then test by ear with a keyboard alone: Tab to the play button, press Enter, and confirm the tone rises and falls in step with the visible line and that pause stops it cleanly without a click.

Why audio complements, but never replaces, the table

Sonification is exceptional at communicating shape and relative movement — is the trend up or down, is there a spike, are the values steady or jagged — and it does this faster than reading a table row by row. What it cannot do is convey an exact figure: no listener can tell 4.2M from 4.3M by pitch, and interval-to-value estimation is unreliable even for trained ears. That is why audio is an additive channel in this section rather than a substitute. Pair it with the accessible data table for precise values and with the generated summary for the headline, and let each channel do what it is good at. A useful refinement is to play a short reference tone for the minimum and maximum before the sweep, or to announce the range in the button label as shown above, so the listener has an anchor for what “high” and “low” mean in the data’s own units.

Edge cases & gotchas

Suspended context after inactivity. Browsers can suspend an AudioContext when a tab is backgrounded. Always await ctx.resume() at the start of play(), not only on first creation.
Oscillators are single-use. An OscillatorNode cannot be restarted after stop(). Create a fresh one for each playback; do not cache and reuse one node.
Zero-variance series. When every value is equal, (y - lo)/(hi - lo) divides by zero. Guard and play a single steady tone, and say so in the label, rather than emitting silence or NaN.
Very long series. Hundreds of points at 0.18s each is a minute of audio. Cap duration by downsampling to a fixed number of representative points, or scale step so the whole sweep fits a sensible window.
Reduced-motion users still want the data. Skipping the sweep is not the same as skipping the information. When motion is reduced, keep the text summary and table fully available so nothing is lost.
Overlapping playbacks. Rapidly pressing play should not layer tones. Stop any in-flight oscillator in play() before scheduling a new one, or gate the button while playing.

Redundant texture on each bar for a line chart with the web audio api — the pattern carries series identity even where colour perception fails.

Frequently Asked Questions

Why can’t I just autoplay the sonification when the chart loads?

Because browsers block audio that starts without a user gesture — the context stays suspended and nothing plays — and because autoplaying sound is hostile to users, including screen-reader users who are already listening to speech. Start playback only from an explicit control the user activated, and additionally honour prefers-reduced-motion by not sweeping automatically. This is both a platform requirement and a usability one.

Should I map data values directly to frequency in hertz?

No. Pitch perception is logarithmic, so a linear value-to-hertz map sounds wrong: low values bunch together and high values scream. Normalise the value to 0–1 against the series range, map that to a musical note range (for example two octaves of MIDI notes), and convert the note to hertz. Equal data steps then sound like equal pitch steps.

Does sonification make a data table unnecessary?

No. Audio conveys the shape of a trend well but exact values poorly — no one can read 4.23 from a pitch. Treat sonification as an additional channel that complements the accessible data table and the text summary, not a replacement for either. Offer all three from the same underlying data so a user can pick the one that suits their task.

Alternative representations for charts — where audio fits among summaries, tables, and exports.
Generating a text summary of a chart trend — the headline that anchors the audio sweep.
Offering a downloadable data table fallback — exact values audio cannot convey.
requestAnimationFrame vs GSAP for data transitions — scheduling on a clock, the visual analogue of audio timing.