In-app EPUB/PDF reading — implementation plan

Companion to ADR-0026. Executable, sliced build plan for the generic document reader (the untrusted-book reader) and its Reading State overlay. Each slice is independently shippable + validated; feature branch per slice, rebased to land ([[feedback_rebase_never_merge]]). foliate-js is third-party (not ours like conform-ed) — vendored + pinned, patched in our vendored copy or upstreamed, never forked into a parallel renderer.

Grounding (verified 2026-06-16)

• The reader namespace is free; the generic reader takes document-reader. A legacy repo-setup scaffold under apps/learner-web/src/features/reader/ (a token-grained ReaderToken { surface, gloss, dictId, pos } / ReaderLocator { chapterId, blockId, wordId, progress } sample, reachable only through a flag-gated /labs/reader route) was deleted 2026-06-16 along with its route, the isReaderLabsEnabled / VITE_FEATURE_READER flag, and its coverage gate — no-longer-reachable boilerplate, to be replaced wholesale by the real Language Reader imported from another project ([[feedback_no_legacy_pre_launch]]). The generic reader is greenfield at apps/learner-web/src/features/document-reader/; the future Language Reader may reclaim the freed reader namespace. The two stay uncoupled by construction — their locators differ (wordId vs CFI) and their trust models are opposite (trusted/reprocessed content vs untrusted/hostile books). Same iframe mechanism if reused, opposite rationale: layout-of-trusted vs isolation-of-hostile.
• External Package model is reusable as-is. external-package.ts: external_package_kind = h5p|cmi5|scorm-1.2|scorm-2004; packageRuntimeProvider = lumi|static-package|scorm-again (a static-package provider exists, with no matching kind — same shape we mirror). Version is an immutable snapshot (what keeps CFIs valid under stored highlights). Storage goes through package-storage-adapter.ts (PackageStorageAdapter.putPackage(bucket, key, files); Supabase signed-URL serving, as the QTI asset pipeline does). Launch + progress already exist: launch-session/service.ts (createLaunchSession), progress_snapshot, and package_progress.
• Erasure is enforced, not optional. erasure-registry.ts (ADR-0014) makes every person-linked table declare an ErasureClass; erasure-registry.test.ts fails if a new person-linked table is unregistered. Any Reading State table is therefore a registry entry by construction, not an afterthought.
• foliate-js API (the seam). relocate event → { range, index, fraction }; CFI.fromRange(range) → CFI; Overlayer + create-overlayer.attach(overlay) → render a highlight from a stored CFI; .goTo({ index, anchor }) → restore. No built-in storage — “storage is entirely the host application’s responsibility.” PDF backend is PDF.js (Apache-2.0).

Slices

Tiered per ADR-0026 — Slices 1–3 are Tier 1 (the nice-to-have: render + coarse progress); Slices 4–5 are Tier 2 (Reading State: bookmarks + annotations). Slice 6 wires CC; Slice 7 is the robustness/security lane.

Slice 1 — foliate-js + the sandboxed document reader — BUILT 2026-06-16

• Pinned foliate-js@1.0.1 via npm (raw source; no vendoring needed) + ambient types.
• DocumentReaderHost renders EPUB in foliate’s sandboxed iframe in its own feature dir (no shared code with features/reader/). Book script blocked by per-section CSP (script-src 'none') via foliate’s transformTarget hook — allow-scripts kept (WebKit/iOS events). See O-2.
• Mantine chrome (TOC, nav, font controls) still to come (it’s the i18n-bearing layer).
• Pure-frontend slice over a fixture EPUB served locally; bun test for the locator/render bridge (captureLocator/restoreLocator over CFI), no DB yet.
• Outcome: an arbitrary EPUB renders, navigates, and is isolated.

Slice 2 — schema + delivery + upload — BUILT 2026-06-16

• Schema: epub/pdf added to external_package_kind + external_package_launch_kind; foliate added to package_runtime_provider; schema-config.test.ts enum guards + the API value arrays (import-service.ts) updated. The whole migration set was regenerated from the schema per docs/runbooks/regenerate-migrations.md (steps 2–6 are DB-free file generation — the agent’s to run; only the dev-DB reset/apply, steps 1/7, is Anton’s, and it gates nothing since the container integration lane applies the committed migrations to its own throwaway Postgres and the unit lane uses PGlite pushSchema). The hand-written customs (storage_buckets, signing role) were restored byte-identical; the baseline’s only content change is the three enum additions.
• Delivery (apps/api/src/features/document-delivery/): GET /workflow/document-packages/:assignmentId/delivery → entitlement-gated JSON { kind, title, externalPackageVersionId, url, expiresInSeconds } (signed bytes URL). Lean — not the cmi5 launch ceremony (ADR-0026 impl note). DocumentObjectStore port (createSignedUrl/putObject), Supabase impl at the edge. 7 PGlite service tests + 5 route tests.
• Upload (POST /workflow/document-packages, org-admin): validates by magic (PDF %PDF-; EPUB OCF mimetype), stores ONE opaque object in the existing external-package-runtime-packages bucket (no new bucket), creates the row graph via the shared importExternalPackage seam (runtimeProvider: foliate, importStatus: ready). 7 PGlite tests (incl. an upload→deliver round-trip) + 4 route tests. 23 document-delivery tests total; api typecheck/lint/format green.

Slice 3 — coarse progress/dwell → teacher view (Tier 1 complete) — NEXT

• Two remaining pieces pair here: (a) the learner-web delivery route that calls the delivery endpoint, fetches the signed bytes, and mounts DocumentReaderHost (the user-facing payoff; greenfield in apps/learner-web/src/features/document-reader/); (b) progress: the reader emits locator progress + dwell on relocate → package_progress for the teacher view.
• Progress lane — DECIDED 2026-06-16 (Anton): xAPI statements → projection. Progress goes via experienced/progressed xAPI statements (duration + locator) → the existing projection → package_progress, keeping a single writer to that projection. NB this re-introduces statement emission for documents even though delivery itself bypasses the cmi5 launch ceremony (ADR-0026 impl note) — the two are distinct: no launch credential/fetch ceremony, but progress is still expressed as xAPI so the teacher-analytics projection stays the one source of truth.
• Outcome: “Johnny spent 3 min on chapter 4” is answerable for the teacher.

Slice 4 — Reading State model: bookmarks + annotations (Tier 2)

• New reading_state tables (bookmark, annotation = highlight + note), keyed (person, external_package_version, locator) where locator is a typed column (cfi for EPUB, page+rect for PDF) — never a page number. Document-type-agnostic; the locator is the only format-specific field.
• Register in the erasure registry (ADR-0014; likely scrub/cascade, subject-tie column) and gate behind consent (ADR-0008). The erasure-registry.test.ts guard makes this mandatory.
• API: CRUD over a learner’s own bookmarks/annotations (learner-self authz). On open, hydrate stored CFIs → Overlayer.attach re-renders highlights; bookmark control → CFI.fromRange.

Slice 5 — teacher rollup surface (Tier 2)

• Teacher-portal panel: per-learner reading engagement (dwell, bookmark/annotation counts) under the Offering, consent-gated. Spreadsheet-conservative per the LMS-domain UI rule. (Mantine MCP first.)

Slice 6 — CC EPUB3 routing (depends on the CC import lane, ADR-0022) — BUILT 2026-06-17

• Shipped richer than designed (commits 871912b, 79c2083, de950d0). Rather than trusting the idpfepub_epubv3p0 resource type, the CC mapping service (cartridge-import-service.ts) content-sniffs each resource’s entry file with the reading lane’s detectDocumentKind (EPUB OCF application/epub+zip / PDF %PDF-) ahead of the kind routing — so a book reaches the reader whether it is declared under the EPUB3 type or shipped as a generic webcontent resource, and PDFs land too (not just EPUB). HTML/QTI never sniff as a document, so it is safe in front. A detected document lands through the same uploadDocument seam as an org-admin upload (one opaque object on the foliate runtime), assigned into the import’s Offering + activity Unit; failures are quarantined-with-reason. The import report gained an imported.documents[] array, surfaced in the teacher-portal report as “Readings imported”.
• Unzipped EPUBs handled too. When a cartridge spreads an EPUB’s loose OCF files across one webcontent resource (no single .epub), detectUnzippedEpubRoot finds the OCF-mandated META-INF/container.xml and repackageUnzippedEpub rebuilds them into one valid .epub (canonical mimetype first/stored, prefix stripped) before the same landing path.
• Covered by cartridge-import-service.test.ts (PGlite). The dedicated CC-with-EPUB fixture belongs to the CC conformance lane (CC plan Slice 6), which is still unbuilt.

Slice 7 — robustness/security fixture lane

• The EPUB analog of a conformance corpus: a vendored set of adversarial + malformed + weird EPUBs (scripted content, oversized/zip-bomb-ish, broken nav, exotic CFI, fixed-layout) under tests/fixtures/epub/ with provenance. Assert: renders-or-fails-soft, scripts never execute, the sandbox never reaches the host origin, CFIs round-trip. This is the real “is it safe” gate.

Demarcation (enforced, not advisory)

• The generic reader lives in features/document-reader/; when the real Language Reader is imported (possibly into the freed features/reader/), the two share no code. The token locator (wordId) and the document locator (CFI/page) are distinct types; a shared “thin bridge interface” is declined — the trust models differ and a shared seam invites a hostile book onto the clean-input path.
• Reading State (locator-grained) ≠ the Language Reader’s session/progress (token-grained). They may look parallel (both capture/restore a locator); they are not one model.

Engine/dependency policy

• foliate-js + PDF.js are vendored + pinned; fixes land in the vendored copy or upstream, never as a fork-and-diverge. No AGPL/strong-copyleft dependency enters the bundle (ADR-0026: Vivliostyle excluded; vendoring would worsen copyleft, not dodge it).

Verification

• Per slice: bun test (touched area) + bun run typecheck; schema/UI/DB slices add bun run validate:full and (Slice 4+) bun run test:integration.
• Security bar (Slice 7): the adversarial EPUB set must render-or-fail-soft with zero script execution and zero host-origin reach.

Open questions (carry into build)

• O-1. RESOLVED 2026-06-16. The legacy features/reader/ scaffold (no-longer-reachable repo-setup boilerplate) was deleted; the real Language Reader will be imported from another project later. The generic reader is greenfield in features/document-reader/; the reader namespace is free. One open thread it leaves: the test:coverage gate (formerly reader-only) is now ungated — decide its replacement target.
• O-2. EPUB scripting policy — RESOLVED 2026-06-16 (browser-verified). All book JS is blocked via a strict per-section CSP (script-src 'none'), injected through foliate’s official transformTarget hook — no foliate patch. Critically, we keep foliate’s allow-scripts sandbox token: dropping it (a tempting “harden the sandbox” move) breaks in-content navigation in WebKit/iOS (the e2e lane proved an in-content link click does nothing in WebKit without it — WebKit bug 218086). So security comes from the CSP, not from removing the token, and the lane runs in both chromium and webkit. EPUB3 scripted interactivity is unsupported (fail-soft); confirmed acceptable.
• O-3. PDF — IN Tier 1, BUILT + browser-verified 2026-06-16. DocumentReaderHost renders PDF via foliate’s experimental PDF.js adapter, vendored under features/document-reader/vendor/ (provenance-pinned to the upstream commit) over a pinned pdfjs-dist@5.5.207; the adapter is imported dynamically so pdf.js stays out of the EPUB-only bundle. The e2e lane renders a PDF fixture to canvas in both chromium and webkit. Two scoped follow-ups: (a) wire the pdf.js cMapUrl + standardFontDataUrl (copy those dirs to a served path) so non-embedded standard fonts and CJK render fully (matters for the Chinese language-learning case); (b) a PDF locator (page+rect) for Reading State — EPUB CFI is the only locator wired today.
• O-4. Offline reading. Do imported books participate in the pglite/offline-data sync (packages/offline-data) like Module Packages, or online-only in v1?
• O-5. Annotation durability across reader upgrades. Immutable Version bytes keep CFIs valid; confirm we never re-process an imported book in place (that would invalidate anchored highlights — any reprocessing is the Language Reader’s separate pipeline, on its own copy).