Three embedded audio bugs that looked like hardware limits

5 October 2026 · RoboLynk LLC, Los Angeles

A palm-sized robot on an ESP32-S3 lost the first word of every question, captured about a quarter of real time, and stuttered halfway through speaking. Each one presented as “this chip isn’t fast enough.” None of them were.

The device is a desk companion built on an off-the-shelf 2.8″ board. You hold the screen, ask something, let go. Audio is captured through an ES8311 codec over I²S at 16 kHz and streamed to a server while you hold; the server transcribes, generates a reply, and streams spoken audio back.

Every bug below cost hours, and every one of them was in our code, in a place that looked innocent.

1. The first word of every question was missing

Someone would ask “why is the 4th of July important” and the transcript came back “July is important.” Short questions lost their subject entirely.

I went looking in the obvious places: packet loss on the websocket, the server’s transcription window, the microphone gain. I rewrote the network buffering twice. I suspected the microphone hardware and was ready to call it a codec problem.

The cause was ordering. The code did this:

ui.show("LISTEN")
gc.collect()
mic.start_recording()

A garbage collection on this board takes a meaningful fraction of a second. And people start speaking the instant the screen says LISTEN — not when the microphone opens, because they can’t see that. So the first word was spoken into a microphone that had not been switched on yet. It was never recorded at all, by anything.

The fix is the obvious one once you see it: start the microphone first, and let the housekeeping happen behind it while the buffer fills.

mic.start_recording()
ui.show("LISTEN")
gc.collect()          # now runs while audio is already being captured

2. The capture loop was sipping from the buffer

Even with the microphone started first, long questions came back truncated, and the device would eventually report a buffer overflow.

The capture loop read one chunk of audio, sent it over the network, and looped:

while recording:
    chunk = mic.read(30ms)     # one 30 ms slice
    ws.send(chunk)             # ~110 ms over TLS

Thirty milliseconds captured, a hundred and ten spent sending. The loop ran at roughly a quarter of real time, and the hardware buffer filled with the audio nobody was collecting until it overflowed and dropped everything after it.

The fix was to drain the buffer on every pass rather than take a sip from it, and to send in larger batches so the per-send cost is amortised:

while recording:
    batch = b""
    while mic.available() and len(batch) < MAX:
        batch += mic.read()    # take everything waiting
    ws.send(batch)

Capture went from 27% to 85% of real time.

3. Streamed speech had nowhere to wait

The replies stuttered. Not every time — just often enough to sound broken, usually mid-sentence.

We were playing audio straight from the network into the I²S hardware buffer, which held about 170 ms. That works perfectly while packets arrive on time. The moment one is late — a retransmit, a scheduling hiccup, a busy access point — the buffer runs dry and you hear a gap.

At 16 kHz mono 16-bit, 170 ms is about 5 KB of slack. That is not enough margin for anything crossing a home Wi-Fi network.

The fix was to hold roughly 600 ms back before starting playback, into a larger buffer, so the speaker always has a cushion of audio ahead of it. Speech starts a beat later and never breaks.

Worth noting what this is not: it is not a bigger buffer alone. A bigger buffer that you start draining immediately is just as vulnerable. The pre-roll is the point — you deliberately accept latency once, at the start, to buy continuity for the whole utterance.

What the three had in common

Every one of them presented as a hardware limit. Lost audio, dropped samples and stuttering playback are exactly what an underpowered chip looks like, and we have all read that microcontrollers are too slow for real-time audio.

They were, in order: an ordering mistake, a loop that did a quarter of the work it appeared to, and a missing buffer policy. The silicon was never the constraint.

If you are debugging audio on a microcontroller, measure these three before you touch anything else: the gap between your UI promise and your capture start, the ratio of captured time to wall time, and how many milliseconds of audio sit ahead of your DAC when playback begins.

A fourth one, for free: the display inverts every pixel

Photographs sent to the screen came back as negatives. I spent three rounds on endianness and channel order — RGB565 versus BGR565, little-endian versus big — and every theory failed, because the error was not in the field layout at all.

The panel is an inverted-type ILI9341, and the firmware never sends INVON/INVOFF. Every byte it receives is complemented before it lights a pixel. The proof had been sitting in the codebase the whole time: the UI paints its background with fill(BLACK), and the screen is white.

Which means the colour constants in the display driver are not a reference for the wire format. They were hand-tuned on an inverted panel, so they look internally consistent and are all wrong in the same direction. RED sent as raw bytes paints yellow.

The robot

It is called RoboLynk Lite. It flashes from a web page in about 70 seconds — no toolchain, no IDE — and the case prints in an hour and twenty minutes. The build guide, the case files and the flasher are free:

varozhan3212.github.io/robolynk-light · five-minute build video