Non-verbal audio: what a voice is doing, whose turn it is, and whether anybody is playing with you. Everything runs in your browser.
Several small creatures playing at once, locked to one cycle at different densities — a gamelan's answer to putting a dozen instruments in a room without it becoming noise. They get tired, take breaks, want the spotlight, and take solos that end when they run out of breath rather than when a timer says so. Each keeps a model of every other one, so it can name what you are about to play and get out of the way before the collision. Make a sound and the cycle follows your pulse.
115 syllable clusters — 91 pairs and 24 triplets or longer — each a whole syllable synthesised with moving formants, encoded, and kept together so that hovering decodes all of its frames in one call. The emoji spell the syllable rather than being assigned by category: 👄😮 is "bah" and 🐍😁 is "see", every time, so a line can be typed and copied. Sung notes are marked in violet.
Sixteen families of sound swept across a parameter and encoded, giving 384 Mimi frames with one emoji each. Sort them by frequency, brightness, complexity, instrument, emoji kind or speech sound — all measured from the decoded audio, not from the probe that went in. Silent meta-emoji in their own section change how the next symbols play: octaves, semitones, scale degrees, tempo, duration, detail, wildcards, and a reset for the drift. Sliders and meta-emoji write the same state.
It hides, and comes back. The question it asks is whether you know when: a sound before the return means something a sound after it does not. It looks for you, shows you it is there, answers when you call, and checks back when the answers stop — presence is mutual by construction. It starts knowing nothing about you and keeps what it learns, lengthening the gap while you manage it and pulling back in when you lose the thread.
A simulated infant that reacts the way the protoconversation literature says, and a partner that draws it in, builds up, and comes off the gas before it is too much. The chart draws the infant's real arousal next to the partner's guess at it, because the gap between those is the whole difficulty. Second tab puts you in the infant's place.
The slow half arms a response; the fast half releases it on a cue at audio-block resolution. Measures cue-to-response latency and, more importantly, its spread — a constant offset can be compensated by whatever tracks the pulse, and jitter cannot be compensated by anything.
The measurement the emoji page is built on. Decodes single frames three ways: codebook 0 varying with the rest zeroed, real frames encoded from synthesised probes, and a real frame with its codebook 0 swapped out.
Kyutai's neural codec, encoding in a worker at the rate a live stream would arrive. Reports the real-time factor and, more to the point, whether the page is still animating while it does.
The same microphone read two ways at once: out of the encoded bitstream, and out of the samples. WebCodecs rather than a peer connection, so there is no call to break.
The policy around the codec rather than the codec: at most one frame being encoded, at most one waiting, everything else dropped and counted. Tested against a stub worker, so it does not need the weights.
A Keepon-shaped creature per player: three axes and a bob, driven by the beat and by the same stance that decides how that player sounds. The movement is arithmetic, so it is checked rather than admired.
Scripted amplitude envelopes at 2.67 ms blocks: does a slot release on its cue, only once, not on a blip, and does it say so when the cue it was waiting for never came?
Pairs appear to form and re-form in a room of four. This asks whether there is more of it than the same players with their timing relationship removed — a phase-shifted control, which keeps each player's own rate and rhythm and destroys only the alignment between them.
Timbre is identity and does not change; stance is what the sound is doing right now and changes constantly. Crossed, a room of four has four voices each with a range of manners rather than four beepers.
A roster, a bus that carries who made each sound, and attribution by register for sounds that arrive through the air. Each player gets its own band, all of them above where a speaking voice lives.
One place decides difficulty, for every game. Four axes rather than one slider, climbed gradually; a direction that falls flat is cooled and something else is tried instead of a smaller version of the same thing.
Energy spent by playing and recovered by sparseness, and an appetite for being attended to rather than for making noise. Checked over time rather than at one convenient instant.
The pulse emits every slot exactly once across ragged frames; the floor goes to one player at a time and ends because they are winded rather than because a timer fired; yielding actually thins the others out.
Five simulated minutes, four players, twenty-eight measures with a wall on each side. Solo length and its spread, who got heard, how long a figure survives, whether pairs interlock, and whether any of it improves over the run.
A factor oracle per neighbour. Recognition as repeated-suffix depth, and prediction of which figure a partner will move to next — always reported beside the two baselines that need no model at all.
What a player has learned about where in the cycle it belongs, kept outside every figure so it survives moving on to a different one. Learn a collision on one lick; the next lick should already know.
Fed cases whose answer is known: a pure loop, a four-bar loop (still a loop), genuine variety, parts that interlock by construction, parts that collide by construction, and a follower.
That it asks before playing, respects a decline, notices when you start hiding, and hands every difficulty decision to the director rather than deciding any of it itself.
Emoji are not characters: several carry a variation selector, so indexing into a string cuts them in half. Plus the probe synth being identical on every load, and the rule that a second of quiet starts a new line.
Codes come back as [1, C, T] channel-major, so joining frames along time means gathering each codebook's row separately. Runs against a fake codec, so it needs no weights.
An automaton built one symbol at a time that answers "where else have I been in this situation?". Two integer arrays are the whole idea: a suffix link to the earlier moment that ended the same way, and how much history they share.
Scripted stimulation against the infant model, the two models against each other, and the coo/grunt/laugh/fuss classifier, all on a synthetic clock.
Thirty simulated minutes across three seeds, with “some chaos but not too much” turned into a number: the entropy of the next move given the current one. Near zero is a metronome; near maximum is thrashing.
Synthesised waveforms with known answers: a tone at a known frequency, a glide of a known slope, a pause of a known length.
Hand-built packets with known answers, so no encoder is needed. Checks the TOC parse against RFC 6716 and the activity reader against DTX.
Scripted frames on a synthetic clock: a face leaving, a head turning, a glance aside that is only a glance, a blink that is not a departure, a jaw opening fast enough to be a turn versus one merely relaxing.
Mimi — the neural audio codec, 12.5 Hz and
1.1 kbps, is Kyutai's
(CC-BY-4.0). What actually runs here is not their release: it is a third-party ONNX
conversion and int8 quantisation published as
onnx-community/kyutai-mimi-ONNX,
pinned to revision 992dd4d5 and checked against recorded SHA-256s. So
there is a converter in the chain as well as the authors, and the Hugging Face Hub
is one origin rather than a mirror network — a branch there is mutable in a way a
released PyPI version is not, which is why the pin exists.
transformers.js — Hugging Face's JS runtime for ONNX models, vendored at 3.4.0 and hash-pinned rather than pulled from a CDN at page load. Version alone leaves a CDN able to serve different bytes under the same name.
The factor oracle is not ours. Written from published algorithms rather than anyone's source, so no licence is inherited, but the ideas belong to Allauzen, Crochemore & Raffinot (the structure, 1999), Lefebvre & Lecroq (the longest-repeated-suffix bookkeeping, 2000), and Assayag & Dubnov (navigating it to generate rather than to search, 2004) — the step that turns an index into an instrument, and the basis of IRCAM's OMax and its descendants.
Gamelan, borrowed as a solution to a problem this already had. Several players taking turns means several times the waiting; several playing freely means noise. A gamelan does neither — everyone plays at once, locked to one colotomic cycle, at different densities. The texture stays legible with a dozen instruments because no two compete for the same rate.
Keepon — the grooves are MyKeepon's dance engine, which got a startling amount of character out of a metronome plus a few random binary switches per degree of freedom. The on-screen bodies are its shape too.
The research is other people's too, and it is not code — these are findings that decided how the play behaves. Malloch & Trevarthen's work on communicative musicality supplies the timings a phrase is built from. Jaffe & Beebe's result that moderately coordinated interaction predicts better outcomes than maximally coordinated interaction is why the partner aims at a band with two walls rather than at a peak. Custodero's flow indicators are what "is this going well" is measured against. Each is cited where it is used.