Put on headphones, scan your floor, and place singing creatures on it: frogs, crickets, a cicada, a katydid. Each one listens to its neighbours and nudges when it calls. Depending on how they listen, a species falls into calling together or takes turns, and calls roll across the room in waves. Nobody tells them to synchronize. You walk among them and hear each one from where you put it.
- Every creature is an oscillator coupled to its neighbours through a phase-response curve. The sign of the curve decides whether a species syncs or alternates.
- The coupling is spatial. Neighbours pull harder when they are close, so the chorus forms local pockets and traveling waves rather than one global pulse.
- I wrote the binaural render: measured MIT KEMAR HRIRs convolved per creature, with the measured interaural time difference applied as a fractional delay. ARKit only supplies where the phone is.
- Each call is synthesized from measurements of a field recording, and morphs continuously into an instrument as you turn up the Voice control.
The chorus
Each creature has its own natural period, spread a few percent around the measured call rate of its species. When it hears a neighbour call while it is at phase θ of its own cycle, it shifts its phase by an amount set by a phase-response curve Z(θ), scaled by a coupling gain and by how far away the neighbour is. That is the whole model, after Greenfield’s work on insect and frog choruses.
The sign of the curve is the one control that matters most. If the response pulls a creature toward the call it just heard, the group drifts into calling together. If it pushes it half a cycle away, the group takes turns. In the app that is the Together ↔ Apart slider. Frogs and the katydid start apart, the crickets and the cicada start together, the way those animals tend to behave.
Coupling falls off as 1/r² with distance, and each call reaches a neighbour after the time sound takes to cross the gap. In a room that delay is only a few milliseconds. The lag you can see and hear across the room, roughly 50 to 300 ms, comes mostly from the chain of local coupling and the spread of natural periods, and a unit test measures it. A gauge shows the Kuramoto order parameter r for the selected species, from 0 for no coherence to 1 for fully locked.
Timing is never quantized. Pitch comes from a shared harmony that moves slowly through a harmonic-series scale, and timing comes only from the coupling. A test renders the same coupling run with two different pitch choices and checks that every onset lands at the same time.
Why the render is mine
The creatures coordinate by phase, and you localize them by interaural phase and time. If Apple’s spatializer computed that part, the one thing the project is about would happen inside a box I can’t inspect or test. So the render is written in the app, and the platform only supplies the listener’s pose.
- Per creature and per ear: a 128-tap minimum-phase HRIR from the MIT KEMAR measurements, then the measured ITD as a cubic Lagrange fractional delay, then distance gain and a low-pass for air absorption. The gain falls off faster than free-field 1/r (about −22 dB at 3 m) and fades to silence between 5 and 8 m, so walking away from a group actually leaves it behind.
- HRIRs and ITD are interpolated bilinearly over the measured grid. When a creature’s direction moves by more than 1°, the block is rendered through the old and new filters and crossfaded, so turning never clicks. ITD and gain are smoothed per sample.
- Direct-form convolution with vDSP rather than FFT. At 128 taps the multiply counts come out about the same, and direct form adds no block latency.
- The audio thread does no locking or allocation: flat struct-of-arrays state, C11 atomics for voice-slot ownership, and a seqlock for the pose.
Two limits I’d rather state than hide. In this mode ARKit tracks the phone, not your head, so turning your head without turning the phone doesn’t change the render. And the KEMAR set is measured down to −40° elevation, so a creature closer to your feet than that is rendered at −40°.
The voices
The voices are synthesized, not sampled. For each species I took one field recording from iNaturalist and measured its spectral peaks, its call envelope, and its pulse structure. The call is rebuilt from those numbers: pulses and noise exciting a bank of modal resonators. No recording audio ships in the app.
Each species also has an instrument it turns into. The Voice control moves every parameter continuously from the creature’s measured values to the instrument’s. The resonances glide from the recorded spectrum onto the note’s harmonic series, ring times grow, noise gives way to pitch, and the call’s rhythm becomes the instrument’s articulation. Cricket pulses become mallet strikes and katydid rasps become plucks. It is one model moving between two settings, not a crossfade between two sounds.
The creatures are painted procedurally by a script I wrote for the project, as animated sprite sheets in the style of a naturalist’s plate. These are the idle frames.
Integration, not invention
Every piece of this comes from published work: phase-response models of animal choruses, the Kuramoto order parameter as a coherence readout, HRTF rendering, and head movement as the cue that separates front from back. What I did was put them together carefully. The headless engine has unit tests covering the coupling, the traveling waves, and the render math. It makes no new scientific claim.
The coupling constants are tuned in simulation for now. The next step is fitting them to statistics measured from real chorus recordings: the order parameter, the distribution of intervals between calls, and who leads whom.
Credits
HRTFs: W. G. Gardner and K. D. Martin, “HRTF measurements of a KEMAR dummy-head microphone,” MIT Media Lab Perceptual Computing Technical Report #280 (1994), and “HRTF measurements of a KEMAR,” J. Acoust. Soc. Am. 97(6), 3907–3908 (1995), via the SOFA conventions database. The voices are measured from iNaturalist recordings by sven_eisenbiegler (bullfrog), rwp84 (green frog), thecicadadude (cicada), lfelliott (katydid), simono (tree cricket) and bugsoundsjc (field cricket). Every observer whose recordings I analysed is credited in the app.
Support
Questions, bugs or feedback: [email protected]. I read everything.
To report a problem, email me what you were doing when it happened, your iPhone model and iOS version, which headphones you were using, and the app version from Field notes in Biophony.
- Biophony needs headphones and an iPhone that supports ARKit.
- If creatures won’t place, move the phone slowly over the floor until the coaching overlay goes away. A well-lit, textured floor works best.
- If you unplug your headphones, the sound pauses instead of switching to the speaker. Reconnect and tap Resume.
Swift · SwiftUI · ARKit · SceneKit · AVAudioEngine · Accelerate (vDSP) · C11 atomics · Python