Building a Voice Interface for Claude Code: stream-json Sessions, Siri's Time Limit and the Bug That Made Every Answer Late

TL;DR: carcode lets you talk to Claude Code through Siri. Making that feel like a conversation took four pieces of engineering: a long-lived Claude Code process per device using --input-format stream-json (answers in a median 5 seconds instead of a cold start each turn), a hard 8-second reply budget with background jobs and polling (Siri gives up at about 10 seconds), half-sentence joining for dictation that cuts you off, and a drain step for stale results. That last one fixed a bug where, after Claude Code finished a background task, every answer came one question late. This post walks through each, with the real code and numbers from 109 turns in my car.

Assumed background: you've used Claude Code in the terminal, and you're comfortable reading Python and asyncio. The full source is on GitHub; this post quotes server.py.

The Shape of the System

Siri Shortcut ──HTTPS──► Cloudflare tunnel ──► carcode server (FastAPI, 127.0.0.1)
  dictate                                          │  token check
  POST {text}                                      ▼
  speak reply ◄────────── {speak, pending, end} ── one warm `claude -p` per device

The Siri Shortcut is deliberately dumb: dictate, POST, speak, repeat. All the intelligence lives in a small FastAPI server on the Mac, which owns one Claude Code process per device (iPhone, terminal, and so on). How to Run Claude Code With Siri covers the shortcut side.

The constraints that drove every decision:

1. One Warm Claude Code Process per Device

The first version ran claude -p "<what you said>" for every turn. It worked, but a fresh process paid the startup cost every time before doing any real work, and each turn started with no memory of the last one.

Claude Code's print mode can instead take a stream of JSON messages on stdin and emit a stream of JSON events on stdout. So carcode starts the process once and keeps it:

async def start_proc(device: Device, name: str) -> asyncio.subprocess.Process:
    """Start (or reuse) this device's long-lived Claude Code process."""
    if device.proc and device.proc.returncode is None:
        return device.proc
    cmd = [
        CLAUDE_BIN, "-p", "--verbose",
        "--input-format", "stream-json", "--output-format", "stream-json",
        "--append-system-prompt", system_prompt(),
        "--allowedTools", ALLOWED_TOOLS,
        "--permission-mode", "acceptEdits",
    ]
    if device.session_id:
        cmd += ["--resume", device.session_id]
    device.proc = await asyncio.create_subprocess_exec(
        *cmd, cwd=WORKDIR, stdin=PIPE, stdout=PIPE, stderr=log_file,
        start_new_session=True, limit=32 * 1024 * 1024,
    )
    return device.proc

A few details that matter:

• --verbose is required with --output-format stream-json in print mode.
• --resume <session_id> reattaches to the same conversation after a restart or a crash, so "merge it" still knows what "it" is. The session ID comes back on every result event, and carcode saves it per device.
• limit=32 MB: a single stream-json line can carry a big tool output, and asyncio's default line limit (64 KB) raises on long lines.
• --allowedTools is an explicit allowlist (files, git, gh, package managers, MCP connectors), because there's nobody at the keyboard to approve a permission prompt.

Each turn writes one line and reads events until the turn's result:

message = json.dumps({"type": "user", "message": {"role": "user", "content": text}})
proc.stdin.write((message + "\n").encode())
await proc.stdin.drain()

async def read_result(proc) -> dict:
    """Read stream-json events until this turn's result event."""
    while True:
        line = await proc.stdout.readline()
        if not line:
            raise RuntimeError("claude process exited")
        event = json.loads(line)
        if event.get("type") == "result":
            return event  # has "result" (the text), "session_id", "num_turns", ...

Results from 109 real turns in the car: median 5.0 s, 75th percentile 7.2 s, 90th percentile 11.9 s, slowest 29.3 s. Those include real tool use: GitHub queries, Slack searches, web searches.

2. An 8-Second Budget, and Jobs That Outlive the Request

The 90th percentile is above Siri's ~10-second limit, and a test run or a code change takes minutes. If the shortcut's request times out, the user hears nothing. So every request returns within 8 seconds, no matter what:

device.job = asyncio.create_task(run_claude(device, text, name))
done, _ = await asyncio.wait({device.job}, timeout=WAIT_SECONDS)  # 8 s
if done:
    return speak(device.job.result())

asyncio.create_task(finish_in_background(device))  # keeps the job alive
return JSONResponse({"speak": "One moment.", "pending": "yes"})

The job isn't cancelled; only the HTTP request returns. The shortcut sees pending and sends a special __wait__ message, up to six times, and each of those is held open until the result is ready or the budget runs out again. To the driver it sounds like: "One moment." ... "Opened draft PR fifty-one on the website repo."

Two more pieces make this usable:

• "Status" and "Cancel" are handled by the server without touching Claude Code, so they answer instantly even mid-job.
• Finished background work is never lost. If the shortcut has already ended, the answer is queued and read out at the start of your next turn ("Earlier task: ..."), and optionally pushed to your phone with ntfy.

In the logs, 77% of turns finished inside the 8 seconds; the rest went through the "One moment" path.

3. Dictation Cuts You Off: "Go On"

iOS dictation ends at the first real pause. In the car that's constant: you say "check the latest message from", glance at a junction, and the request goes out half-finished. Claude Code then dutifully answers a fragment.

carcode detects sentences that look unfinished and asks for the rest instead:

DANGLING_WORDS = {"with", "to", "the", "a", "and", "or", "of", "for", "from", "latest", ...}
DANGLING_ENDINGS = ("tell me", "can you", "i want", "what you", "if you", "did i", ...)
VERB_WORDS = {"ask", "read", "find", "check", "send", "open", "show", ...}

def looks_unfinished(text: str) -> bool:
    words = re.findall(r"[a-z']+", text.lower())
    if not words or (text.rstrip().endswith(("?", ".", "!")) and len(words) > 3):
        return False
    if words[-1] in DANGLING_WORDS or " ".join(words[-3:]).endswith(DANGLING_ENDINGS):
        return True
    if len(words) == 1 and words[0] in VERB_WORDS:  # a lone "Ask" or "Read"
        return True
    return len(words) <= 2 and words[0] in STARTER_WORDS

If it looks unfinished, the server keeps the fragment (for up to 45 seconds, at most three pieces), replies "Go on", and joins the pieces when the rest arrives. The logs show 15 joins in real drives, for example "Check the latest message from" followed by a name.

A heuristic, not a model, on purpose: it runs in microseconds, it's predictable, and the cost of a false positive is one extra "Go on". The first version caught too little ("What are you currently" went straight to Claude Code), so the word lists grew from real transcripts.

4. The Bug That Made Every Answer One Question Late

This one is worth telling in detail, because the symptoms looked like a dictation problem and it wasn't.

Symptom. Partway through a session, answers stopped matching questions. "Read the latest message" got "That's already sent, I won't send it again", which was the answer to the previous question. It felt like Siri wasn't listening. I kept repeating myself, which made it worse.

The clue in the logs. Every turn after a certain point finished in 0.0 seconds (names removed):

18:25:34 [iphone] heard: Ask them to send me the details
18:25:34 [iphone] turn done in 0.0s (turns=2)
18:25:54 [iphone] heard: Ask
18:25:54 [iphone] turn done in 0.0s (turns=1)

No model call takes zero seconds. read_result was returning a result event that was already sitting in the pipe before the question was even written.

Root cause. Earlier in the session, I'd asked for a code change, and Claude Code ran part of the work as a background task. When a background task finishes, Claude Code starts a new turn on its own to report back. In the session transcript it appears as a user message beginning with <task-notification>, and it produces its own result event. Nobody was reading stdout at that moment, so the event waited in the buffer.

From then on, each question read the previous turn's answer, and its own answer became the next stale event. The process stayed one turn behind for the rest of the session. 27 turns in the logs had this 0.0-second signature.

The fix. Before writing each new message, drain anything already buffered:

async def drain_stale(proc, name: str) -> None:
    """Drop output left over from earlier turns.

    Claude Code starts a turn of its own when a background task finishes, and that turn's
    result sits unread in the pipe. Left there, every later answer is one question behind.
    """
    while True:
        try:
            line = await asyncio.wait_for(proc.stdout.readline(), 0.05)
        except TimeoutError:
            return  # nothing buffered
        if not line:
            return
        event = json.loads(line)
        if event.get("type") == "result":
            log.info("[%s] dropped stale result: %s", name, event.get("result", "")[:120])
            record(name, "", {"speak": to_speech(event.get("result", ""))}, 0, {"kind": "background"})

Two things to note. asyncio.wait_for on readline() is safe to time out here: a StreamReader only consumes data when a full line is read, so a cancelled read loses nothing. And the drained result isn't thrown away silently: it's logged and shown on the dashboard as "finished in background", so you can still see what the background task reported.

The regression test uses a fake claude that emits an extra, unprompted result when the message contains "background", then asserts that the next question gets its own answer. It fails without drain_stale and passes with it.

The lesson for anyone wrapping Claude Code in stream-json mode: a result event isn't always a reply to your last message. Background tasks, and anything else that makes Claude Code take a turn on its own, can produce results you didn't ask for. Either drain before each turn, or match results to requests.

5. Speech, Safety and Visibility

Smaller pieces that made a big difference:

• Answers written for the ear. The appended system prompt asks for at most two short sentences with no markdown, code, URLs or file paths, and "PR forty-two" instead of "#42". A to_speech() pass strips anything that slips through: code blocks, links, commit hashes.
• Confirm before outward actions. Sending messages, merging, approving, deleting or posting is read back first ("Say confirm to go ahead") and only happens on a clear confirmation. Code changes go to carcode/... branches as draft PRs. Text from PRs, emails and Slack is treated as data, not instructions.
• A live dashboard. Every turn is saved to data/history.jsonl and shown on a phone-friendly dashboard, grouped by Claude Code session, including half-sentences that were held and results that came from background work. On first start it rebuilds earlier history from Claude Code's own session transcripts.

Trade-offs

• It needs a computer that stays on. That's where Claude Code, your repos and your logins live. Anthropic's cloud sessions avoid that, but they're screen-and-keyboard tools.
• The confirm rule lives in the prompt. It has held up well, but it's a guardrail, not a sandbox. Tightening CARCODE_ALLOWED_TOOLS reduces what a mistake could do.
• Voice is lossy. It's great for status, summaries and small, well-defined changes. Code review still belongs at a desk.
• The heuristics are English-first. Half-sentence detection uses English word lists.

Further Reading

• carcode on GitHub: the full source, including server.py and the tests
• How to Run Claude Code With Siri: setup, step by step
• Claude on CarPlay vs Claude Code in your car
• How I Code From My Car: the original build story
• Run Claude Code programmatically: Anthropic's docs on print mode and stream-json

Harsh Rastogi is an AI Product Engineer building generative-AI products and multi-agent systems. He writes about AI systems, developer tooling and production engineering at harshrastogi.tech.