QTI 3.0: demonstrable conformance and full platform coverage

Status: accepted — built (2026-06-16)

Build status (2026-06-16). Decision §5 (native interaction authoring) is done: the author→serialize→ingest loop exists and the teacher UI authors the full delivered palette — choice, order, text-entry, extended-text, inline-choice, slider, upload, associate, match, hottext, gap-match (ADR-0021 §5, branch qti-interaction-authoring/qti-item-assets) plus the whole graphic family (hotspot, select-point, graphic-order, graphic-associate, graphic-gap-match, position-object, drawing) and media (audio/video) carved out into ADR-0024, with the authored-asset upload pipeline in ADR-0023. §3 export + Content-Package (items and tests), §1 conformance lane, and §4 results emission are now done (branch qti-conformance-coverage; test export landed on the @conform-ed 0.0.23 publish of the view→document inverse). The last decision — §6 PNP/feedback/adaptivity/stimulus/rubric — is now done too (branch qti-authoring-breadth): all six constructs are authorable end-to-end, the rubric path unblocked by the @conform-ed 0.0.24 view+inverse extension (now pinned), with the stimulus bank + person-scoped PNP profile persistence (schema/RLS/services/routes/migrations) gated by a Podman integration test. The conformance lane is now 25 emitted-artifact gates. docs/design/qti-conformance-plan.md tracks slice-level status.

Builds on ADR-0001 (QTI ASI as the canonical assessment model), ADR-0009 (imported standard boundary), ADR-0010 (authoring converges on QTI), ADR-0019 (server-anchored delivery + universal server-side response processing), and ADR-0020 (rich assessmentTest authoring). It turns an implicit “we use QTI” into an explicit, provable “this is a conformant QTI 3.0 platform.”

The tension (named, not hand-waved)

We lead with QTI, yet QTI is the one standard we cannot point at external proof for. Every other standard in the repo is validated against an official artifact in a CI lane — test:ob:conformance (1EdTech OB 3.0 JSON Schema), test:clr:conformance (1EdTech CLR 2.0 JSON Schema), test:cmi5:conformance, test:lrs:conformance:conform-ed. QTI has no test:qti:conformance lane and no vendored official QTI artifact in this repo. QTI is exercised only through conform-ed’s own zod contracts — “matches our model of the spec,” which is circular.

A second point, discovered while grounding this ADR: the capability gap is not in the engine. @conform-ed/qti-react already delivers and scores the full interaction palette (associate, choice, drawing, end-attempt, extended-text, gap-match, graphic, hottext, inline-choice, match, media, order, slider, text-entry, upload); @conform-ed/qti-xml already serializes every ASI artifact plus assessmentResult, manifest, metadata, and AccessForAll PNP, and validates in-memory instances (validateQtiXmlContent, written for “the export-conformance round trip”). conform-ed even vendors the official QTI 3.0.1 + AFA-PNP XSDs and the official 1EdTech example corpus, with its own xsd-conformance / corpus round-trip tests (run locally in that repo).

The gap is therefore at emergent (the product): it ingests QTI and delivers/scores all of it, but it exposes no QTI export (zero serializeQti* calls), emits no QTI results, authors no items natively, and has no conformance lane of its own — even though it makes the conformance claim.

Verified baseline (2026-06-15, corrected). An earlier draft of this ADR said emergent “authors ~5 interaction types.” Grounding against the code shows that was never true: the only item-acquisition path is the import panel (QtiItemIngestPanel.tsx → ingestQtiAssessmentItem), and git history contains no native interaction editor, past or present. Native item/interaction authoring count is 0; items are import-only. What was completed (ADR-0020, slices A–F + follow-up G–J) is the assessmentTest structure authoring surface, now exhaustive (sections, timing, selection/ordering, nav/submission, conditions over the full RpExpressionView union, outcomes, outcomeProcessing, testFeedbacks, item-ref weights/templateDefaults). Authoring therefore splits into two axes: (1) assessmentTest structure — done; (2) item/interaction — absent at the time of writing. This ADR owns axis (2) plus interchange (export/results/packaging) and the conformance lane.

Update (2026-06-16): axis (2) is now built — §5 native authoring (this ADR) plus the graphic/media families split into ADR-0024. The open work is interchange + the conformance lane (see the build-status banner above).

A third point: this is pre-launch (no users/data; [[feedback_no_legacy_pre_launch]]) and conform-ed is ours ([[feedback_internal_lib_release_all_packages]]). Nothing forces a compatibility compromise; missing capability is fixed upstream, then exposed here.

Decision

1. Conformance is proven against external ground truth, in emergent’s own CI lane

A new test:qti:conformance lane (mirroring the OB/CLR lanes) is the bar for claiming “conformant.” Because no XSD validator is available in-process (xmllint absent; pure-JS XSD validation is not viable), the authoritative non-circular evidence is the official 1EdTech qtiv3-examples corpus round-trip: every conformant official example is run through emergent’s real pipeline (validateQtiXmlContent → headless projection → re-serialize → re-validate), and emergent’s own emitted artifacts (exported items/tests, assessment results, packages) are validated with validateQtiXmlContent. The official corpus is external ground truth — passing it is not circular. A curated, representative subset of the official corpus is vendored into tests/conformance/fixtures/qti/ (one per interaction kind + a test, a result, a stimulus, a package) for a deterministic CI lane; the full 327-file corpus is exercised by an opt-in .local lane that fetches it (the conform-ed pattern). The official XSDs are vendored alongside as provenance/reference.

2. conform-ed is the complete engine; emergent’s job is to expose it — do not rebuild

No QTI parsing, serialization, scoring, or schema logic is re-implemented in emergent. Every slice wires an existing @conform-ed/qti-xml / @conform-ed/qti-react capability into an emergent product surface. If a genuine engine gap is found (e.g. an interaction the corpus exercises that the engine mishandles), it is fixed upstream in conform-ed and released ([[feedback_internal_lib_release_all_packages]]), not patched in emergent.

3. QTI export and Content-Package round-trip (make interchange bidirectional) ✅ done (2026-06-16)

emergent gains QTI export: items export their verbatim stored source XML (GET /workflow/qti-items/:itemId/qti.xml, server-only qti_source_xml); tests serialize via the domain serializeAssessmentTest wrapper (serializeQtiAssessmentTest ∘ assessmentTestDocumentFromView ∘ assessmentTestViewFromTestStructure — the conform-ed view→document inverse, no emergent re-implementation) at GET /workflow/qti-tests/:assessmentDefinitionId/qti.xml; and full QTI 3.0 Content Packages (zip + imsmanifest.xml via serializeQtiManifest) are produced — both an offering’s item bank (.../qti-offerings/:offeringId/package.zip) and a self-contained test package (.../qti-tests/:assessmentDefinitionId/package.zip: the imsqti_test_xmlv3p0 resource plus the QTI source of every item version it references, with hrefs rewritten to the bundled files). Import already exists (qti-ingest.ts); this closes the loop, and the conformance lane validates each emitted artifact against the external XSD.

4. QTI results reporting

Finalized attempts emit a QTI assessmentResult (serializeQtiAssessmentResult) as an export surface, validated in the conformance lane. The internal gradebook / OneRoster / CLR projections are unchanged; this adds the QTI-native results artifact the delivery-conformance surface expects.

5. Interaction authoring breadth — built from zero ✅ done (2026-06-16)

A native item-authoring UI is created (it did not exist at baseline) covering the full palette the engine delivers and scores. The chosen shape is author a structured item model → serialize to QTI XML (serializeQtiAssessmentItem) → feed the existing ingest path (ingestQtiAssessmentItem), so authored items are validated, normalized, stored, delivered, and scored by the same pipeline as imported ones, and become exportable for free (this couples authoring with §3 export — the serializer is already wired). Delivery, scoring, and import already cover every interaction; this closed the authoring gap.

Delivered: packages/domain/src/qti-item-authoring.ts (itemDraftSchema union + buildAuthoredItemDocument → serializeQtiAssessmentItem) and the teacher authoring panels cover choice, order, text-entry, extended-text, inline-choice, slider, upload, end-attempt, associate, match, hottext, and gap-match here; the graphic family (hotspot, select-point, graphic-order, graphic-associate, graphic-gap-match, position-object, drawing) and media (audio/video) were split into ADR-0024 (coordinate/shape editor + the ADR-0023 asset-upload pipeline). Only customInteraction/PCI now remains unauthored natively (and generic PCI is quarantined at delivery) — scoped, with the corpus residual, in ADR-0025. It stays importable, so platform coverage is the engine’s full palette throughout.

6. Accessibility (PNP/APIP), feedback/adaptivity, shared stimulus, rubric blocks ✅ done (2026-06-16)

AccessForAll PNP is authorable and exported via serializeQtiAccessForAllPnp (the AFA-PNP XSD joins the conformance fixtures). Feedback (feedbackBlock, modal/inline), adaptive items / templateProcessing, shared assessmentStimulus, and rubric blocks are authored and round-tripped via the existing engine + serializers.

Status (2026-06-16): built. All six constructs are authorable end-to-end, each gated by an emitted-artifact conformance test (the lane is now 25) and never re-implementing the engine (ADR-0021 §2 — drafts map to conform-ed normalized documents, validated against the published contract, then serialized). Built slice-by-slice in the documented order:

• A — feedback (feedbackDraft → scored RP + modal/inline feedbackBlock).
• B — adaptive/template (adaptive flag + templateProcessing: random draws + constraints).
• C — item catalog (AccessForAll catalogInfo alternatives, linked to the body node).
• E — shared stimulus (O-2): native authoring + the assessment_stimulus(_version) bank (offering-membership sync, no keys), an org-admin-gated author/export service + routes, the teacher-portal author panel, and an item-side stimulus-reference editor.
• D — learner PNP (O-1): the person-scoped, learner-owned assessment_person_pnp_profile (owner RLS in the baseline + a public.fn_person_pnp_proxy_allowed SECURITY DEFINER helper migration for the teacher/admin proxy), an owner/proxy-gated upsert + …/pnp.xml export service + routes, and the teacher-portal proxy author panel.
• F — rubric blocks: unblocked once @conform-ed 0.0.24 carried rubric blocks on the assessmentTest view + inverse (now pinned); authored at test/part/section level in the structure editor and serialized to conformant QTI.

Slices D + E added schema: the full migration set was regenerated (the new tables fold into initial_baseline; the PNP proxy policy is its own script-generated custom migration, covered by migration-drift.test.ts), and the ephemeral-container integration gate (qti-stimulus-pnp.integration.test.ts) proves author → persist → export end-to-end. The PNP profile is registered cascade in the erasure registry (ADR-0014) and added to the identity-linking pristine probes. Detailed plan: docs/design/qti-authoring-breadth-plan.md.

7. Sequencing — native authoring first (revised 2026-06-15; progress 2026-06-16)

The original plan put the conformance lane (§1) first as the cheapest credibility win. With the baseline corrected, the decision (Anton, 2026-06-15) is to attack the largest product gap first: native interaction authoring (§5), built on the author→serialize→ingest loop so export (§3) falls out of the same work. The conformance lane (§1) then lands as the companion gate the moment emergent emits its first QTI artifact — it validates those emitted items/tests via validateQtiXmlContent and cites conform-ed’s terminal corpus result rather than re-proving the engine. Revised order: §5+§3 (author + export) → §1 (conformance gate on emitted artifacts) → §4 (results) → Content-Package round-trip → §6 (item-level feedback/PNP/adaptive/stimulus/rubric). Each slice extends the conformance lane with its new artifact.

Progress (2026-06-16): ✅ §5 authoring, ✅ §3 export + Content-Package (items + tests), ✅ §1 conformance lane, ✅ §4 results emission — all done on branch qti-conformance-coverage (gates green: validate:full, test:integration, test:qti:conformance). The one remaining decision is §6 (PNP/feedback/adaptivity/stimulus/rubric authoring UI), its own session per qti-authoring-breadth-plan.md.

Consequences

• emergent gains a test:qti:conformance lane + vendored official QTI fixtures, QTI export endpoints, a Content-Package builder, results emission, a wider authoring UI, and PNP authoring. Most of this is wiring; the engine work is already done in conform-ed.
• The defensible public claim moves from “QTI-3-aligned” to “conformant against the official 1EdTech QTI 3.0 example corpus” once §1 is green, and to “full” as §3–§6 land.
• Any engine defect surfaced by the corpus is fixed in conform-ed and pulled in via a release — emergent stays a pure consumer of the standard’s implementation.
• This is a multi-slice program executed incrementally on dsv1 (feature branch per slice, rebased to land — [[feedback_rebase_never_merge]]); docs/design/qti-conformance-plan.md is the executable slice plan.

Decisions for Anton

1. Conformance evidence = official example-corpus round-trip + export-conformance, not in-process XSD validation (no xmllint; pure-JS XSD isn’t viable). Revisit if an XSD validator becomes available. (accepted 2026-06-16)
2. Vendor a curated official-example subset for the CI lane, full corpus via an opt-in .local fetch lane (the conform-ed pattern) — balances determinism against repo weight. (accepted 2026-06-16)
3. Engine gaps fixed upstream in conform-ed, never patched in emergent. (accepted 2026-06-16)
4. Sequencing: native interaction authoring + export first (author→serialize→ingest loop), then the conformance lane as the companion gate on emitted artifacts, then results, Content-Package round-trip, then item-level PNP/feedback/adaptivity/stimulus/rubric. (accepted 2026-06-15; §5 authoring shipped 2026-06-16; supersedes the original conformance-first order)
5. Native authoring shape = author a structured item → serializeQtiAssessmentItem → reuse the existing ingest pipeline, so authored and imported items share one validation/normalize/ store/deliver/score path and export is intrinsic. (accepted 2026-06-16 — built; see §5)
6. Graphic-interaction authoring + the asset-upload pipeline are split into their own ADRs (ADR-0024 coordinate/shape editor; ADR-0023 TUS upload + server finalize), keeping this ADR focused on interchange + the conformance lane. (accepted 2026-06-16)