04 / Local AI / document processing

A document should not have to leave your machine
to be read aloud.

A local-first PDF text-to-speech tool that extracts text in the browser, synthesizes speech on-device, and highlights the sentence being read. Built as Evangeline.

Live product Source Live

PDF Playback Cache Adaptive playback / local audio path

Adaptive playback
Chapter 04

LOCAL INFERENCE / INDEXEDDB

01 / Context

Reading aloud should not require an upload

Cloud TTS services ask for the document. That is unacceptable for legal, medical, or confidential files. Privacy-preserving alternatives existed, but they either required technical setup that non-technical users could not navigate, or they dumped synthesis on a slow local machine with no way to hide the latency.

PDF Read Aloud — shipped as Evangeline — is a local-first reader: extract in the browser, speak on-device, highlight the sentence in view.

02 / Constraints

Privacy, latency, and a storage budget

The raw PDF should not leave the tab during the reading flow. Speech should not leave the machine either.

Local inference is slower than a hosted API. If the player only synthesizes the current sentence, the listener hears the gap. If it synthesizes the entire document first, the wait is unbounded.

Browser storage is finite. A cache has to be large enough to hold real documents and strict enough not to grow forever.

03 / System

A streaming local audio pipeline

PDF.js extracts text client-side. Text is normalized and split into sentences. Chunks go to a local FastAPI service running Kokoro ONNX, or to the browser's own speech engine.

The prefetch engine tracks playback pace and keeps upcoming audio ready. A 150 MB IndexedDB cache with LRU eviction stores generated audio across sessions.

Sentence-level highlighting follows playback, and click-to-jump moves both the audio and the visual position. The full stack ships in one Docker container.

04 / Decisions

Decisions made against constraints.

Decision / 01

Keep the document on the machine

Problem: Cloud text-to-speech requires uploading document text. That is a non-starter for legal, medical, or confidential files, regardless of a provider's privacy policy.
Decision: Extract text with PDF.js in the browser and synthesize with a Kokoro ONNX model served by a local FastAPI backend.
Why: The premise of the tool is that document text never leaves the user's device. Local inference is the product boundary, not a fallback.

Decision / 02

Stay ahead of the listener

Problem: On-device synthesis has real latency. Synthesizing strictly one sentence ahead causes audible stalls the moment playback catches up.
Decision: An adaptive prefetch engine watches playback pace with an EWMA of synthesis latency and queues one to six chunks ahead.
Why: Prefetch removes stalls without over-synthesizing content the listener may never reach.

Decision / 03

Do not pay for the same sentence twice

Problem: Re-synthesizing a document every session wastes time and defeats fast playback on a second listen.
Decision: Cache audio in IndexedDB with a 150 MB LRU budget, which holds several full-length documents without blowing typical browser quotas.
Why: Once a sentence is cached, playback can start without waiting on the model. LRU eviction keeps the cache from growing unbounded.

Decision / 04

Offer a zero-setup path

Problem: Kokoro ONNX needs the FastAPI backend. That is heavier than some people want for a quick read.
Decision: Support the browser Web Speech API as a second engine, with Kokoro as the high-quality local path.
Why: The tool should work for a casual read and a serious one without forcing a single tradeoff on every user.

05 / Implementation

Browser extraction, local inference, persistent cache

The frontend is HTML, CSS, and JavaScript with vendored PDF.js. FastAPI plus Uvicorn serves POST /v1/audio/speech and GET /v1/audio/voices. Kokoro ONNX runs in the same Docker image; the first load pulls the model into a volume and keeps it there.

Content-security policy and a server-side URL validator keep text on localhost. Tab visibility pauses and resumes playback. Rate control uses Kokoro's speed parameter.

Cached audio skips synthesis entirely on repeat listens, so a known document can start in under a second.

06 / Behavior

How the system behaves.

Current segment The sentence being spoken is highlighted and kept in view. Playback position and visual position stay coupled.
Prefetch window The engine keeps a moving window of upcoming sentences in flight — typically one to six chunks, sized from measured latency.
Cache Heard sentences remain in IndexedDB. Returning to a chapter does not wait on the model.

07 / Result

What improved.

A document can be read aloud without sending its text through an external speech service.

Adaptive prefetch hides local-inference latency during continuous listening.

A bounded cache makes the second session feel immediate without unbounded disk use.

08 / Reflection

What the work demonstrates.

The product is a privacy boundary with a playback problem attached. Local inference only works as a reading experience if the system stays ahead of the listener.

Performance here is not a benchmark number. It is whether the next sentence is ready before the current one ends.

Next system

01 / API governance / product system

YASP

Open