Frog Wizard

Dictate anywhere from the i3bar tray

Most Linux applications have no dictation feature. dictate-tray is a tray icon for i3bar that adds one to all of them: click it, speak, click again, and the text is on the clipboard. Paste works everywhere, so dictation works everywhere: browser, editor, chat, terminal.

Transcription runs locally with Whisper. No audio leaves the machine. It is one Python file, about 500 lines.

Use

exec --no-startup-id dictate-tray
bindsym $mod+Shift+d exec --no-startup-id dictate-tray toggle

Press the binding, talk, press it again, paste. The icon says what is happening:

Icon State
grey mic idle
amber mic microphone opening, not capturing yet
red with bars recording, bars are the live input level
orange with bars stop pressed, capturing the last 400 ms
blue spinner transcribing
green check text is on the clipboard
yellow exclamation mark failed, hover for the reason

Left click toggles. Right click gives Cancel recording and Quit.

The Whisper model loads when the first recording starts, so it warms up while you talk. It stays loaded for back-to-back dictations and unloads after 60 idle seconds. With tiny on CPU the loaded worker holds about 0.8 GB of RAM, and nothing when idle.

Dependencies

X11 only. Arch:

sudo pacman -S python-gobject python-cairo python-numpy gtk3 libpulse xsel libnotify
pipx install openai-whisper

Debian: python3-gi python3-gi-cairo gir1.2-gtk-3.0 python3-cairo python3-numpy pulseaudio-utils xsel libnotify-bin.

Build spec

The rest of this post is the spec. It is written so that you, or a coding agent pointed at this URL, can rebuild the script without the source.

Shape

Two interpreters

The tray needs the system Python, because that is where PyGObject lives. Whisper usually lives in a pipx venv. The script bridges them by re-running itself as a worker under Whisper's interpreter, which it reads from the shebang of the whisper command:

def whisper_python():
    exe = shutil.which("whisper")
    if exe:
        try:
            with open(exe, "rb") as fh:
                first = fh.readline().decode().strip()
            if first.startswith("#!"):
                return first[2:].split()[0]
        except OSError:
            pass
    return sys.executable

The tray starts the worker with [whisper_python(), os.path.abspath(__file__), "--worker", MODEL], pipes for stdin and stdout, text mode, and stderr appended to ~/.cache/dictate-tray.log.

Worker protocol

Line-delimited JSON over stdout, file paths over stdin. Whisper and torch print to stdout, which would corrupt the protocol, so the worker first moves the real stdout to a private file descriptor and points fd 1 at stderr:

def worker_main(model_name):
    out = os.fdopen(os.dup(1), "w")
    os.dup2(2, 1)
    import numpy as np
    import whisper

    device = DEVICE
    try:
        model = whisper.load_model(model_name, device=device)
    except Exception:
        device = "cpu"
        model = whisper.load_model(model_name, device=device)
    out.write(json.dumps({"ready": device}) + "\n")
    out.flush()
    for line in sys.stdin:
        path = line.strip()
        if not path:
            continue
        try:
            audio = np.fromfile(path, dtype=np.int16).astype(np.float32) / 32768.0
            result = model.transcribe(
                audio, language="en", fp16=(device == "cuda"),
                condition_on_previous_text=False)
            reply = {"text": " ".join(result["text"].split())}
        except Exception as exc:
            reply = {"error": str(exc)}
        out.write(json.dumps(reply) + "\n")
        out.flush()

Passing a float32 numpy array to transcribe skips ffmpeg entirely. The input file is raw 16 kHz mono signed 16-bit PCM, no header.

The tray side is a Worker class with a lock around transcribe(path):

  1. Start the process if it is not running.
  2. If the ready line has not been read yet, read it.
  3. Write the path and a newline, flush, read one reply line.
  4. Record last_use. Raise on {"error": ...}, return text otherwise.
  5. An empty read means the worker died. Raise whisper worker died, see <log path>.

start() is also called at the moment recording begins, without waiting for ready, so the model loads while the user speaks. A GLib timer every 15 seconds terminates the worker when the tray is idle and last_use is older than DICTATE_IDLE.

Recording

subprocess.Popen(
    ["parec", "--raw", "--format=s16le", "--rate=16000",
     "--channels=1", "--latency-msec=30",
     "--client-name=dictate-tray"],
    stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)

A daemon thread reads parec stdout with os.read(fd, 4096) until EOF, appending to a buffer. For each chunk it computes the peak absolute sample (as int32, to avoid overflow on -32768) divided by 32768 and keeps the maximum in a shared peak attribute. After EOF it trims the buffer to an even byte count and hands it to the main loop with GLib.idle_add.

Details that matter:

Transcribe and copy

In another daemon thread: write the PCM to a temp file in $XDG_RUNTIME_DIR, call worker.transcribe(path), fail with No speech recognised on empty text, pipe the text to xsel --clipboard --input, delete the temp file in a finally. Report back to the main loop with GLib.idle_add. All GTK calls happen on the main loop, never in a thread.

On success, show the green check for 2.5 seconds and send a notification with the text. On failure, show the error icon for 4 seconds and send the reason. The notification uses a replace hint so repeated dictations do not stack:

notify-send -a Dictate -t 3000 -h string:x-canonical-private-synchronous:dictate "Copied to clipboard" "$text"

State machine

IDLE, STARTING, RECORDING, FINISHING, TRANSCRIBING, DONE, ERROR.

Tray icon

Single instance and toggle

Acceptance checks

#i3 #linux #python #whisper