Skip to content
emergent engineering

QTI 3.0 conformance + coverage — implementation plan

Companion to ADR-0021. Executable, sliced build plan. The lane is overwhelmingly emergent-side wiring of the complete @conform-ed QTI engine + a conformance lane; engine gaps (if any) are fixed upstream in conform-ed and released.

Grounding (verified 2026-06-15)

Section titled “Grounding (verified 2026-06-15)”
  • @conform-ed/qti-react delivers + scores: associate, choice, drawing, end-attempt, extended-text, gap-match, graphic, hottext, inline-choice, match, media, order, slider, text-entry, upload (the full QTI 3.0 palette).
  • @conform-ed/qti-xml serializes: serializeQtiAssessmentItem / …Test / …Section / …Stimulus / …AssessmentResult / …Manifest / …Metadata / …AccessForAllPnp / …OutcomeProcessingDocument / …ResponseProcessingDocument / …UsageData; validates with validateQtiXmlContent (built for the export round trip); parses with parseXmlDocument.
  • conform-ed vendors the official QTI 3.0.1 ASI XSD + AFA-PNP XSD (scripts/fetch-qti-schemas.ts) and the official 1EdTech qtiv3-examples corpus (327 files: items, tests, packaging, results, usageData, PCI, CAT — scripts/fetch-qti-corpus.ts), with xsd-conformance / corpus round-trip tests in that repo.
  • emergent today (verified 2026-06-15): ingests only (qti-ingest.ts → validateQtiXmlContent + headless projection; items acquired solely via QtiItemIngestPanel.tsx), authors 0 interactions natively (no item editor has ever existed — items are import-only), no serializeQti*, no results, no test:qti:conformance. The assessmentTest structure authoring surface, by contrast, is now exhaustive (ADR-0020 A–F + G–J): that axis is done; this plan owns the item/interaction axis + interchange + the conformance lane.

Slices (each independently shippable + validated; feature branch per slice, rebased to land)

Section titled “Slices (each independently shippable + validated; feature branch per slice, rebased to land)”

Revised execution order (2026-06-15, per Anton): native interaction authoring leads, and because authoring is built on the author→serialize→ingest loop, export (Slice 2) ships with it. The conformance lane (Slice 1) then lands as the companion gate on emitted artifacts (validate authored/exported items via validateQtiXmlContent; cite conform-ed’s corpus result rather than re-proving the engine). Net order: Slice 4 + Slice 2 (author + export) → Slice 1 (conformance gate) → results → Content-Package → Slice 5. The slice contents below are unchanged; only their order is.

Status (2026-06-16):

  • Slice 4 (native authoring) — ✅ done. Full non-graphic palette here; the graphic family + media + asset upload landed via ADR-0024 / ADR-0023. The serializeQtiAssessmentItem loop is live (packages/domain/src/qti-item-authoring.ts).
  • Slice 2 (export surface) — ✅ done (items + tests). ITEM export (GET /workflow/qti-items/:itemId/qti.xml, stored canonical source verbatim; server-only qti_source_xml column). TEST export (GET /workflow/qti-tests/:assessmentDefinitionId/qti.xml) via the domain serializeAssessmentTest wrapper = serializeQtiAssessmentTest ∘ assessmentTestDocumentFromView ∘ assessmentTestViewFromTestStructure — the conform-ed view→document inverse shipped in @conform-ed 0.0.23 (now pinned), so no emergent-side normalizer re-implementation (§2 held).
  • Slice 1 (conformance lane) — ✅ done. test:qti:conformance (deterministic vendored subset)
    • :local (full corpus). Coverage on the full 327-file official corpus today: 315 valid, 241 items ingested, 9 quarantined, 12 backlog. Export-conformance gate on emergent’s own emitted item is green.
  • Results emission (§4) — ✅ done. serializeAssessmentResult (domain, contract-guarded)
    • GET /workflow/assessments/:attemptId/result.xml (on-demand pull, owner-or-org-admin, only finalized attempts; item identifiers mapped from the test’s authored item refs). Conformance lane extended with the emitted result.
  • Content-Package (Slice 3) — ✅ done (items + tests). buildQtiItemContentPackage (domain: manifest contract-guarded + fflate zip; an optional imsqti_test_xmlv3p0 test resource depending on each bundled item) + GET /workflow/qti-offerings/:offeringId/package.zip (item bank) and GET /workflow/qti-tests/:assessmentDefinitionId/package.zip (self-contained test + the source of every referenced item version, hrefs rewritten to the bundled files). Round-trip gated by validateQtiPackageArchive (service test + conformance lane).
  • Slice 5 (§6 PNP/feedback/adaptive/stimulus/rubric) — planned, build deferred to its own session. It is purely teacher-portal authoring UI (the engine already round-trips all five); detailed slice plan in qti-authoring-breadth-plan.md. F (rubric) additionally needs the conform-ed test view to carry rubric blocks (a view+inverse extension).

Slice 1 — QTI conformance lane (companion gate on emitted artifacts)

Section titled “Slice 1 — QTI conformance lane (companion gate on emitted artifacts)”
  • Vendor a curated representative subset of the official corpus into tests/conformance/fixtures/qti/ (one item per interaction kind + a test + a result + a stimulus + a package) with a provenance README.md; vendor the official QTI 3.0.1 ASI XSD
    • AFA-PNP XSD as reference. (Sourced from the official 1EdTech corpus.)
  • tools/scripts/run-qti-conform-ed.ts + test:qti:conformance (mirror run-ob-conform-ed.ts; this lane is pure in-process, no DB/podman needed — QTI XML in, validation out).
  • tests/conformance/qti-credential.conformance.test.ts → qti.conformance.test.ts: for every vendored example, assert emergent’s real pipeline (ingestQtiAssessmentItem and the test/stimulus/result equivalents over validateQtiXmlContent + headless projection) accepts and projects it; report any it cannot (the diagnostic). Start asserting the green set; xfail + list the rest as the measured backlog feeding later slices.
  • Optional .local full-corpus lane (test:qti:conformance:local) that fetches all 327 (the conform-ed pattern; skips when absent).
  • Outcome: an external-ground-truth lane; the claim becomes “conformant against the official 1EdTech QTI 3.0 example corpus (covered subset),” and we now know the real coverage.

Slice 2 — QTI export + results emission

Section titled “Slice 2 — QTI export + results emission”
  • packages/domain: serializeAssessmentItem(version) / serializeAssessmentTest(structure) wrappers over conform-ed serializers (canonical model → QTI XML).
  • apps/api: export endpoints (GET …/items/:id/qti.xml, …/assessments/:id/qti.xml) and finalized-attempt assessmentResult emission (serializeQtiAssessmentResult).
  • Extend the conformance lane with export-conformance: serialize → validateQtiXmlContent → re-parse round-trip equality.

Slice 3 — QTI 3.0 Content-Package round-trip

Section titled “Slice 3 — QTI 3.0 Content-Package round-trip”
  • imsmanifest.xml (serializeQtiManifest + serializeQtiMetadata) + zip packaging incl. shared assessmentStimulus; import the produced package back through the existing import path; assert round-trip. Conformance lane gains a package fixture.

Slice 4 — native interaction authoring (built from zero) ← LEADS

Section titled “Slice 4 — native interaction authoring (built from zero) ← LEADS”
  • A native teacher item-authoring UI created from nothing (none exists; items are import-only) covering the full delivered palette (choice, text-entry, extended-text, inline-choice, order, match, associate, gap-match, hottext, hotspot/select-point/graphic family, slider, drawing, upload, media, end-attempt), each author→serialize→ingest→deliver→score→review verified.
  • Shape: author a structured item model → serializeQtiAssessmentItem → the existing ingestQtiAssessmentItem path (one validation/normalize/store/deliver/score pipeline for authored + imported items; export intrinsic). Start with a vertical slice (choiceInteraction) proving the full loop, then fan out per interaction kind. (Mantine MCP first; reuse the delivery components for preview.)

Slice 5 — PNP/APIP, feedback/adaptivity, shared stimulus, rubric

Section titled “Slice 5 — PNP/APIP, feedback/adaptivity, shared stimulus, rubric”
  • AccessForAll PNP authoring + serializeQtiAccessForAllPnp (AFA-PNP XSD in fixtures); feedbackBlock (modal/inline), adaptive items / templateProcessing, shared stimulus, rubric blocks — authored + round-tripped via the engine; each added to the conformance lane.

Engine-gap policy

Section titled “Engine-gap policy”

Any official example the engine itself mishandles (not just our wiring) is fixed in conform-ed and released single-version ([[feedback_internal_lib_release_all_packages]]); emergent bumps the pin. emergent never patches QTI semantics locally.

Verification

Section titled “Verification”
  • Per slice: bun test (touched pkg) + bun run typecheck; after UI slices bun run validate:full.
  • Conformance: bun run test:qti:conformance green on the vendored subset is the slice bar; :local over the full corpus is the stretch bar.
  • Slice 2+ that touch the DB/finalize also run bun run test:integration.

Open questions (carry into build)

Section titled “Open questions (carry into build)”
  • O-1. Curated-subset size for the CI lane vs .local full corpus (start ~20 files).
  • O-2. Export id/URI scheme for items/tests in emitted QTI (reuse the credential/CASE base-URL convention).
  • O-3. Whether results emission is on-finalize (push) or on-demand export (pull) — default pull, matching the credential export surfaces.