Sixteen families of sound, swept across a parameter and encoded, giving a few hundred Mimi frames with one emoji each. Wave the mouse to play them; sort them by what they actually turned out to be, which is measured from the decoded audio rather than from the probe that went in.
A Mimi frame is 80 ms of sound described by 32 numbers. This builds a few hundred of them from sixteen families — a pure tone family sweeping pitch, a struck family sweeping decay, two vowel families moving a formant, and so on — because a family can be synthesised as one continuous signal, encoded in a single call and decoded in a single call. Encoding four hundred frames one at a time is forty seconds of waiting; by family it is about two.
Every sort is measured from the decoded audio, not from the probe. That distinction is the whole point of the page: this is 1.1 kbps trained on voices, so what comes out is frequently not what went in, and sorting by the input would be sorting by a fiction. Pitch is autocorrelation, not the loudest bin — in a vowel the loudest partial is a formant. Complexity is spectral entropy, because flatness measures 0.0000 for a sine, two sines and a five-partial chord and so cannot order anything within tonal.
The instrument, speech-sound and emoji-kind labels are guesses — hand-written thresholds over those measurements, not trained classification. They decline rather than reaching for a nearest match, which is why a sort by speech sound puts the unmatched at the bottom: a null is the absence of an answer, not a small value. Most of a palette built from bells and buzzers is unmatched, and that is the honest result.
Only two of the three sliders are codec inputs. A decode takes
audio_codes[1, 32, T] and nothing else — no pitch, no conditioning. So
frames is real duration and codebooks is real detail (Mimi is residual:
keep the first K and zero the rest — K=1 is hiss, K=2 already carries almost everything).
Rate is not a codec input at all; it is resampling after the fact, moving pitch and
speed together, and it is labelled that way rather than dressed up as a pitch knob.
The meta-emoji drift on purpose. An octave-up applies to everything after it, so a string is a little score rather than a list of notes — which is exactly why 🔄 exists. Drift with no way back is a trap. The sliders and the meta-emoji write to the same state, so whichever you touch, the panel above shows what is actually true.
Play and Join render differently, audibly. Play schedules one buffer per symbol, so every control applies per symbol. Join splices the whole run into one tensor and decodes it once, so the codec's own temporal model joins the frames — one utterance instead of a row of separate ones. Per-symbol rate changes cannot survive that, and Join says so when it drops them.