QTI graphic-interaction authoring: the coordinate/shape editor

Status: accepted

Builds on ADR-0023 (authoring asset pipeline — the background image is an uploaded item asset), ADR-0021 (native item authoring; the author→serialize→ingest pipeline), and the existing native authoring lane (eleven non-graphic kinds shipped). It unblocks the image-backed graphic interaction kinds, which are the only authoring gap left once the asset pipeline exists.

The tension

Seven of the remaining interaction kinds put shapes and points on top of a teacher-supplied image and score by where the learner clicks/drags:

Kind	What the teacher must author on the image
`hotspotInteraction`	one+ hotspot regions (rect/circle/ellipse/poly), each an identifier
`selectPointInteraction`	correct point area(s) (shape+coords) + per-area score (area mapping)
`graphicGapMatchInteraction`	drop-target hotspots + draggable gap images; correct gap→hotspot pairs
`graphicAssociateInteraction`	hotspots + correct hotspot↔hotspot associations (lines)
`graphicOrderInteraction`	hotspots + their correct order
`positionObjectStage` / `positionObjectInteraction`	a stage image + object image(s) + correct placement area(s)

(drawingInteraction is image-backed too but needs no coordinate authoring — it’s a background image the learner draws on, manual-graded. It ships on the asset pipeline alone, no editor.)

Every one of these reduces to the same authoring primitive: draw/move/resize shapes and drop points on an image and read their coordinates back as QTI shape + coords (pixel space: rect=x1,y1,x2,y2; circle=cx,cy,r; ellipse=cx,cy,rx,ry; poly=x1,y1,…; point=x y). That interactive editor — drag handles, resize/rotate transforms, polygon vertex editing, hit-testing, pointer/touch, undo — is expensive to build well and is a thoroughly solved, generic problem. Nothing QTI-specific lives in it; the QTI-specific part is the thin mapping from a shape to its shape/coords string and the per-kind wiring (which identifier, which score, which pairing).

So the real question, per the request, is build vs adopt, and whether the maintenance cost of a dependency beats hand-rolling a canvas editor.

Investigation: build vs adopt

The interactive-shapes-on-an-image editor has mature, permissively licensed libraries (registry-verified 2026-06-15):

konva (MIT, 0 deps) + react-konva (MIT) — a 2D scene graph: Image background, Rect/ Circle/Ellipse/Line (polygon), draggable nodes, a Transformer for resize/rotate handles, pointer events. First-class React binding. Gives every primitive the full QTI shape set + points + association lines + object placement need, with maximal control over what each shape means.
fabric (MIT, 0 deps) — comparable interactive-object canvas with built-in selection/transform handles; React integration is community wrappers (weaker fit for a React/TanStack app than konva).
@annotorious/react (BSD-3-Clause) — purpose-built image region annotation (rect/polygon). Excellent for the hotspot-style kinds, but its model is W3C Web Annotations (a region + a body), which fights the non-region kinds (points for selectPoint/positionObject, association lines, object placement). Best as a runner-up if we only ever did hotspots.
leaflet (BSD-2) + leaflet-draw (MIT) — the “image as a CRS.Simple map, draw shapes over it” pattern. Battle-tested, but the map abstraction is a poor conceptual fit and leaflet-draw is effectively unmaintained (1.0.4, stale) — a maintenance liability.

Excluded:

tldraw — license is SEE LICENSE IN LICENSE.md (the tldraw license, not OSS-permissive, watermark/commercial terms). Not adoptable under our licensing stance.
react-image-annotate — no license field in the published package — legal risk; exclude.
@excalidraw/excalidraw (MIT) — a whole whiteboard app (31 deps, pre-1.0); wrong abstraction and far too heavy for “shapes over an image.”

Verdict — adopt the primitive, build only the QTI glue. The editor’s hard parts (handles, transforms, vertex editing, hit-testing, touch) are exactly what these libraries provide; none of it is QTI-specific, so hand-rolling it is pure undifferentiated cost (weeks to build, ongoing browser/ touch/a11y maintenance). The QTI mapping (shape→coords, identifier/score/pairing wiring) is unavoidable glue we own either way and is small. A single small MIT dependency (react-konva, 0/4 transitive deps) is a far lower total cost than a bespoke canvas editor. This is not a place to innovate; Mantine has no equivalent, so a specialized dependency here is justified.

Decision

1. Adopt `react-konva` (MIT) as the shape-editing primitive; do not build it from scratch

Wrap it in a single teacher-portal feature component (src/features/offerings/.../GraphicShapeEditor) — only the teacher authors items, so it stays feature-local (promote to packages/ui only if a second app ever needs it, per the repo rule). We do not adopt a whiteboard (Excalidraw/tldraw) and do not hand-roll the canvas.

2. Build only the QTI coordinate adapter + per-kind wiring (domain)

In @emergent/domain, a pure shape ↔ { shape, coords } adapter (the five QTI shapes + point) plus per-kind draft schemas/builders that extend the existing itemDraftSchema, feeding the same author→serialize→ingest pipeline (ADR-0021). The editor emits shapes; the adapter emits QTI.

3. Coordinate space is the image’s intrinsic pixels

QTI coords are pixel offsets into the rendered image. The editor locks the drawing surface to the background image’s intrinsic dimensions (display-scaled, stored unscaled). It reads naturalWidth/ naturalHeight from the loaded asset; we additionally persist intrinsic width/height on the assessment_item_asset row (a small ADR-0023 follow-up) so delivery can validate the coordinate basis. The background image is loaded through the ADR-0023 serving endpoint (signed URL).

4. Keyboard-accessible numeric fallback alongside the canvas

Canvas editors are not natively keyboard-accessible, and emergent holds accessibility high. Every shape also appears as an editable numeric coordinate row (identifier + shape type + coord fields), which doubles as a precision tool. The numeric table is the authoritative, accessible editing path; the canvas is the convenience layer over it.

5. Per-kind mapping (all feed the existing ingest pipeline)

Kind	Editor output → QTI
hotspot	shapes → `qti-hotspot-choice[shape,coords,identifier]`; correct = chosen identifier(s); `match_correct`
graphicOrder	shapes → `qti-hotspot-choice`; correct = ordered identifier list
graphicAssociate	shapes → `qti-associable-hotspot[matchMax]`; correct = identifier↔identifier pairs (lines)
graphicGapMatch	shapes → `qti-associable-hotspot` + uploaded `qti-gap-img`; correct = gapImg→hotspot directedPairs
selectPoint	shapes → `qti-area-mapping` areas (shape,coords,score) + min/max points; mapResponsePoint scoring
positionObject	stage image + object image(s) → `qti-position-object-stage`/`-interaction` + area mapping

6. Scope is authoring only

Delivery/rendering of the graphic kinds is conform-ed’s reference skin (the engine already lists them in its rendered palette). This ADR adds no delivery code; we confirm the reference skin renders each kind when its authoring slice ships.

Consequences

One new front-end dependency in teacher-portal: react-konva (MIT, tiny transitive surface).
We own a small pure adapter + per-kind authoring UI; the expensive interactive editor is the library’s to maintain.
A small ADR-0023 follow-up: capture image intrinsic width/height on upload (needs the asset service to read image dimensions — a metadata addition, no new Supabase surface).
drawingInteraction (background image, no shapes) can ship immediately on the asset pipeline, independent of this editor.
Accessibility is satisfied by the numeric fallback being the source of truth, not an afterthought.

Decisions for Anton

All five accepted by Anton on 2026-06-15 (“yes to all your recommendations”). First slice: hotspot + selectPoint behind one react-konva editor.

Adopt the primitive, build only the QTI glue (don’t hand-roll the canvas editor, don’t adopt a whiteboard). (accepted)
Library = react-konva (MIT). Runner-up @annotorious/react (BSD) if we ever scoped to hotspots only; fabric (MIT) is equivalent but weaker React fit. (accepted)
Keyboard-accessible numeric coordinate table as the authoritative editing path, canvas as the convenience layer. (accepted)
Persist image intrinsic width/height on assessment_item_asset (small ADR-0023 follow-up) so the coordinate basis is validated at delivery. (accepted)
Sequencing: hotspot + selectPoint first (one editor, simplest scoring), then graphicOrder/graphicAssociate/graphicGapMatch, then positionObject; drawing (no editor) any time. (accepted)