Presenter side: presenter tab, prompter loop, desk UI, e2e + eval harnesses #5

Merged
morten-olsen merged 14 commits from presenter-side into main 2026-09-10 15:52:27 +02:00
Owner

Summary

Implements steps 1–2 of ADR 0001 (the ADR convention itself is introduced here, with ADR 0002 as a draft):

  • Presenter tab — "Present" on the stage opens the slides alone in a named tab via window.open + a React portal over the same in-memory stores (no BroadcastChannel). Stylesheets mirror into the child document; useBoxSize observes resize from the element's own realm so rough strokes regenerate on window resize. The surface is letterboxed to a fixed aspect with hidden overflow, assuming the draft slide-viewport direction.
  • Turn-loop extraction + fan-out seam — the coalescing turn loop moves from agent/ into loop/; the deck author becomes a thin composition over it, and session.ts fans committed transcript chunks out to a consumer list. Behavior-preserving (all prior tests unchanged).
  • The prompter — a second loop instance reading presentations.script: next_line({block_id}) points into stable script block ids (the pointer-not-paragraph hot path), next_line({text, resume_at}) composes answers/bridges with a rejoin block, advise() for meta guidance. Events land as prompter.called/prompter.turn with audioAt; console state is a pure fold. Per-role LLM connections (resolveConnection('author'|'prompter'), unset fields inherit).
  • Console + desk UI — teleprompter margin on the stage (exact script text, pen-circled cursor, composed lines on a highlighter smear), one lag gauge per loop; on the desk the script is first-class with a live block-split note, prompter toggle in the GO LIVE pull, rehearse-with-prompter, and a Settings prompter-model override.
  • Playwright e2e harness (task e2e) — tests own the browser: fresh profile, seeded pairing + stub LLM, the real CLI bridge (@lazysketch/cli/bridge), production build. Three specs cover the scripted pipeline, prompter-mode runs, and the desk flow.
  • Live prompter eval (task eval:prompter) — script + annotated paraphrase-delivery fixture scored on pointer trajectory; first Haiku pass landed 8/8 (~0.9s median turn) after adding the composed-line rule.

Bugs found and fixed by the harnesses: event seqs are now allocated atomically in the db adapter (two session instances used to mint colliding seqs that log consumers silently dropped), and a run-watcher early-finish (an ending marker is void once a later started/resumed follows).

Test plan

  • task check / task lint:eslint / task test (203 unit tests) green
  • task e2e — 3 Playwright specs against the production build, stub LLM
  • Live validation: task eval:prompter fixture 8/8 on claude-haiku-4-5 through a paired tab; presenter tab and prompter console verified in a real browser session

Commits are unsigned (1Password was locked during autonomous work — per instruction).

## Summary Implements steps 1–2 of [ADR 0001](docs/adr/0001-presenter-side.md) (the ADR convention itself is introduced here, with [ADR 0002](docs/adr/0002-slide-viewport.md) as a draft): - **Presenter tab** — "Present" on the stage opens the slides alone in a named tab via `window.open` + a React portal over the same in-memory stores (no BroadcastChannel). Stylesheets mirror into the child document; `useBoxSize` observes resize from the element's own realm so rough strokes regenerate on window resize. The surface is letterboxed to a fixed aspect with hidden overflow, assuming the draft slide-viewport direction. - **Turn-loop extraction + fan-out seam** — the coalescing turn loop moves from `agent/` into `loop/`; the deck author becomes a thin composition over it, and `session.ts` fans committed transcript chunks out to a consumer list. Behavior-preserving (all prior tests unchanged). - **The prompter** — a second loop instance reading `presentations.script`: `next_line({block_id})` points into stable script block ids (the pointer-not-paragraph hot path), `next_line({text, resume_at})` composes answers/bridges with a rejoin block, `advise()` for meta guidance. Events land as `prompter.called`/`prompter.turn` with `audioAt`; console state is a pure fold. Per-role LLM connections (`resolveConnection('author'|'prompter')`, unset fields inherit). - **Console + desk UI** — teleprompter margin on the stage (exact script text, pen-circled cursor, composed lines on a highlighter smear), one lag gauge per loop; on the desk the script is first-class with a live block-split note, prompter toggle in the GO LIVE pull, rehearse-with-prompter, and a Settings prompter-model override. - **Playwright e2e harness** (`task e2e`) — tests own the browser: fresh profile, seeded pairing + stub LLM, the real CLI bridge (`@lazysketch/cli/bridge`), production build. Three specs cover the scripted pipeline, prompter-mode runs, and the desk flow. - **Live prompter eval** (`task eval:prompter`) — script + annotated paraphrase-delivery fixture scored on pointer trajectory; first Haiku pass landed 8/8 (~0.9s median turn) after adding the composed-line rule. Bugs found and fixed by the harnesses: event seqs are now allocated atomically in the db adapter (two session instances used to mint colliding seqs that log consumers silently dropped), and a run-watcher early-finish (an ending marker is void once a later `started`/`resumed` follows). ## Test plan - `task check` / `task lint:eslint` / `task test` (203 unit tests) green - `task e2e` — 3 Playwright specs against the production build, stub LLM - Live validation: `task eval:prompter` fixture 8/8 on claude-haiku-4-5 through a paired tab; presenter tab and prompter console verified in a real browser session Commits are unsigned (1Password was locked during autonomous work — per instruction).
Step 1 of the presenter side (docs/adr/0001-presenter-side.md, the ADR
convention is new here too): "Present" in the stage tray opens a named
tab and portals the lit sheet — slides and celebrations, no chrome —
into it over the same in-memory stores. The stage tab stays the single
owner of mic, workers, and agent loops, and becomes the console estate
for the coming prompter. Stylesheets mirror into the child document
behind a <base>; useBoxSize observes resize from the element's own
realm so rough strokes regenerate inside the child window.
Step 2 groundwork (ADR 0001): the coalescing turn loop — queueing,
coalesced deltas, serialized pump, recent-context fold, lag/processed
stores, abort+timeout — moves from agent/ into its own loop/ module,
with renderTranscript/speakerMarker (the delta vocabulary both loops
share). The deck author is now a thin composition: its prompt and tool
parsing over a loop instance; the prompter will be a second instance.
session.ts fans committed chunks out to a consumer list instead of the
one hardwired agent. Behavior-preserving: the agent interface and all
183 tests are unchanged.
The presenter surface now letterboxes the sheet in a fixed-aspect
(1024x800 working numbers), overflow-hidden box — composition already
independent of window and chrome, so landing the real fixed logical
viewport (draft ADR 0002: one composition everywhere, zoom out when
content no longer fits) changes what's inside the box, never the
window contract. ADR 0001 picks up the two-lag-gauges note: the loops
run on individual clocks.
ADR 0001 plumbing: presentations gain script (the prepared talk the
prompter reads) and prompterInstruction; sessions gain prompterEnabled;
prompter.called/prompter.turn join the event vocabulary. All optional —
no Dexie bump, transfer format stays v1, script.md lands at the CLI
workspace root. scriptBlocks() gives the prompter its address space:
stable ordinal ids over splitScript. Settings gain the per-role
connection seam — resolveConnection('author'|'prompter'), full
connection overridable, unset fields inherit — and the author now
resolves through it. session.ts splits its author wiring into
session.agent.ts and log plumbing into session.transcript.ts (the
prompter wiring will mirror the former).
New apps/e2e workspace (ADR 0001, validation): Playwright drives the
production build with a fresh profile per test, localStorage seeded
with a known pairing (fixed port/token) and a deterministic OpenAI-
compatible stub LLM the test scripts — committed tests never touch a
real endpoint. The test-side harness talks the real protocol through
the CLI's own bridge server (now exported as @lazysketch/cli/bridge):
push a presentation, fire a scripted run, assert on the streamed event
log. First spec covers the whole pipeline end to end.

Found and fixed by the harness: event seqs were minted by each session
instance, so two writers on one session (dev StrictMode double-mount +
CLI watcher's seq high-water dedup) silently lost events — seq is now
allocated atomically in the db adapter at write time.

task e2e (builds first) / task e2e:install (one-time browser download).
ADR 0001 step 2 core. prompter/ is the loop module's second instance:
next_line({block_id}) points into the script's stable block ids — the
pointer-not-paragraph hot path — or composes a line for off-script
moments (answer proposals, bridges back) carrying its rejoin block;
advise() is guidance that isn't words to say. The whole script rides
the cached system prompt; turns carry the delta, the cursor, and
measured time only (elapsed now, slot with step 3) — timing prose in
script/prompterInstruction is reasoned over, never parsed. Actions
land as prompter.called/prompter.turn events (audioAt attached) and
fold into the console state — live and resume are one fold
(session.prompter.ts, sibling of session.agent.ts).

The stage console renders the teleprompter margin: exact script text,
the pointed block circled in pen, composed lines on a highlighter
smear, advice under the header — and the tray carries one lag gauge
per loop, since the loops run on individual clocks. The prompter
resolves its own LLM connection (per-role seam). Prompter mode rides
sessions.prompterEnabled: `lazysketch run --prompter` over a new
optional protocol field; the desk toggle is still to come.

Covered by unit tests (tools, loop cursor, state fold) and a second
Playwright spec driving both loops through one stub end to end.
stage-view splits into .surfaces/.tray support files.
The talk script gets its own section on the desk (collapsed to an
invitation when empty — "unlocks the prompter" — or a one-line summary
once written): an on-sheet hand-font editor that saves on blur and
shows the prompter's address space live while writing ("3 prompter
lines (b1–b3)"), with the prompter instruction beside it. The GO LIVE
pull gains a drawn Prompter checkbox when a script exists; rehearsals
prefill from the talk script and can run "with prompter" — the fastest
in-browser evaluation loop. Settings exposes the prompter model
override (blank = inherit). The teleprompter's pointed line bumps to
glanceable size.

Covered by a third Playwright spec driving the whole flow through the
UI: write script on desk → split note → toggle appears → rehearse with
prompter → stage console shows the script with the cursor.
prompter eval harness; first live tuning pass lands 8/8 on Haiku
Some checks are pending
ci/woodpecker/pr/quality Pipeline is running
ci/woodpecker/push/quality Pipeline is running
ci/woodpecker/push/compliance Pipeline is running
ci/woodpecker/pr/compliance Pipeline was successful
c60eea03d5
task eval:prompter drives the exact browser prompter loop from Node
(createPrompter -> streamChat, .env credentials — the sanctioned live
path): a script plus a DELIVERY transcript (paraphrases, a digression,
an audience question, a skip-ahead), with expectations annotated per
chunk ({expect:b4}, {expect:composed->b5|b6}, {quiet}) and a scored
pointer trajectory. Sample fixture pair included.

Live findings (claude-haiku-4-5 via the paired browser tab, realtime
pace): pointer tracking, digression discipline, and the skip-ahead all
held from the start; the audience answer consumed upcoming script
content and resumed past it — defensible, now codified: composed lines
stay to 1-2 spoken sentences and resume_at names the first block the
answer did NOT cover. 8/8 after the rule, median turn 896ms.

Also fixes a real run-watcher bug the live run exposed: an early
'stopped' marker (dev StrictMode double-mount) ended runs after one
chunk — an ending marker now only counts while no later started/resumed
marker follows it.
readme: the presenter side — present tab, script, prompter
All checks were successful
ci/woodpecker/push/compliance Pipeline was successful
ci/woodpecker/pr/compliance Pipeline was successful
ci/woodpecker/push/quality Pipeline was successful
ci/woodpecker/pr/quality Pipeline was successful
63da5219ce
The contract inverts, motivated by the four-pillars story the feature
exists for: with points delivered out of order, a pointer back into the
script points at a plan that is already dead — recovery is a rewrite
from here, which is exactly next-token prediction. The model's streamed
reply IS the next thing to say: the script reproduced while the talk
tracks it, the live rewrite when it doesn't (spent points dropped,
unspent salvaged), an answer when the audience asks — and it works
identically when the script is bare talking points. Take a breath; the
line is already on the console.

Plumbing: streamChat exposes content deltas; the loop passes text
through with turn attribution and stamps causedByAudioAt on onTurn;
the prompter accumulates the stream (console draft, pen blue — the
active edge) and finalizes it as a prompter.called {kind:'line'} event.
Tools become the cold path: mark_position (the block the line delivers
— resurrecting exactly what 0001 folded away) and advise. The Say card
is the console hero; pre-0003 composed payloads still fold.

Live-tuned on Haiku (task eval:prompter, headless via .env): 7/7 and
5/7 across runs at ~1.4s median; residual echo-on-near-verbatim noted
in the ADR, plus the negative result that over-instruction ("decide
silently first" + inline example) regresses hard on floor models.
Sessions that ran with the prompter replay its margin at the playhead —
the folded Say line, cursor, and advice at each moment of the talk: the
post-talk review surface for judging every line it offered. Same fold
as live (foldPrompterEvents, now exported from the session module).

README's "On stage for real" now opens with the reason the feature
exists: the moment the script stops being true and reading it back
can't help — take a breath, the rewritten next line is already there.
Honest small print notes the echo residual. Verified against roughened
STT-shaped delivery too: 9/9 sane lines, split audience questions
handled, garbled input never leaked into output.
whisper-cli an actual talk recording, then srt-to-delivery.mjs shapes
the SRT into STT-commit-sized chunks (VAD-ish gaps, max-hold-ish caps)
for task eval:prompter — the loop used to validate the prompter against
the recording of the talk that motivated ADR 0003. ADR 0001 notes the
top rung still open: a faster-than-realtime in-browser harness (offline
frame source + a clock seam) that would include real ASR chunking.
prompter voice hardening: every token must be sayable into the mic
All checks were successful
ci/woodpecker/push/compliance Pipeline was successful
ci/woodpecker/pr/compliance Pipeline was successful
ci/woodpecker/pr/quality Pipeline was successful
ci/woodpecker/push/quality Pipeline was successful
9bb883dc23
Validated against the real recording of the talk that motivated ADR
0003 (whisper small.en, first 20 min pre-round-table, 159 STT-shaped
chunks vs the 67-block script). The lines were right from the first
run — instant mid-talk localization, correct material under audience
questions, correct tracking through a 30-block drift — but coaching
leaked into the Say line at pressure moments ("you're doing great",
"I can see you're having a technical hiccup"), which is unreadable
verbatim mid-stream.

Three prompt changes, each re-validated on the full recording: the
identity sentence itself now says the reply is read verbatim into the
microphone; a rule bans addressing the presenter (coaching belongs in
advise); and the speech-topology rule — the prompter is never spoken
to, transcript speech is aimed at the audience or the slide-drawing
agent ("Atlas, put these up" is part of the talk, not a message).
Coach-voice: systematic at pressure moments -> 2 of 159, both on
direct-address bait (a literal "can you hear me?" mic check). The
eval now counts unsayable-smelling lines per run, and the sample
fixture gains an Atlas-aside chunk (8/8 with it).
the slide viewport: one composition everywhere, the lamp pulls back
All checks were successful
ci/woodpecker/pr/compliance Pipeline was successful
ci/woodpecker/pr/quality Pipeline was successful
ci/woodpecker/push/compliance Pipeline was successful
ci/woodpecker/push/quality Pipeline was successful
ci/woodpecker/push/deploy Pipeline was successful
7d47c3d76e
ADR 0002 implemented (draft -> accepted). Every slide is composed
against one fixed logical sheet (1024x800, one tunable constant) and
uniformly projected into whatever shows it — transparency physics: the
wobble scales with the ink, and rough strokes now regenerate only
against the stable logical size, never on window resize. Stage,
presenter tab, and replay render the identical sheet.

When content outgrows the sheet the lamp pulls back: a discrete ~0.85
ladder grows the logical page until the content fits. Overflow is
measured on the content box plus data-slide-overflow probes (the
canvas's centered block column clips its own spill — the first
screenshot pass showed clipped bullets and no pull-back), and a
subtree observer remeasures on changes that land without a parent
render — the async elk layout, found when the presenter tab measured
pre-layout and stuck at zoom 1.

The presenter tab also surfaced a real bug: a cloned stylesheet <link>
never fetches in the about:blank tab under production builds (dev only
worked because Vite injects inline styles) — the window now mirrors
styles by serializing the opener's parsed CSSOM, verified against the
prod build (ink-resolved strokes, Shantell in the popup).
morten-olsen deleted branch presenter-side 2026-09-10 15:52:27 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
incubator/lazysketch!5
No description provided.