QTI 3.0 residual coverage: generic PCI host, XInclude, rich results, CAT

Status: proposed (2026-06-16); §1 superseded + BUILT (2026-06-16) — the generic PCI host turned out to already exist in conform-ed, and the tier-2 delivery seam shipped (conform-ed 0.0.25) and was consumed by emergent in this revision. §2–§5 remain as below.

Update (2026-06-16): §1 is built — the PCI gap is closed for tier-2

§1 below was written assuming emergent must wait on a generic PCI host that did not exist. That premise was wrong: conform-ed’s PCI host (createPciSkin + createPciModuleRegistry + mountPci, AMD eval, content-driven loading, the full getInstance→getResponse→getState lifecycle) was already built and corpus-proven (ADR-0006/0007). conform-ed then shipped the tier-2 delivery seam in 0.0.25 (ADR-0012): a hash-pinned createPciCatalog (default-deny allowlist + sha256 integrity), getState suspend/resume persistence, and package module loading. emergent now consumes it:

• Tier-1 (install model), already shipped — first-party PCIs that ship a bespoke widget and/or a custom RP operator (the math-entry archetype) are installed deps registered at runtime. An admin cannot vet an operator.
• Tier-2 (vetted catalog), built in this revision — an org admin vets a standard-RP content PCI for an offering (assessment_pci_module: moduleId → server-only AMD source, content-addressed by a server-computed sha256). The ingest gate flips such an item from quarantined to deliverable exactly when its type resolves in the offering’s catalog and its RP needs no uninstalled operator (the capability gate already enforced the operator half). The learner builds a conform-ed createPciCatalog from the offering’s catalog and the host integrity-checks each module before it runs. Surfaces: @emergent/domain ingest option, assessment_pci_module + erasure entry, the API vet/list/serve routes (org-admin vet; offering-entitled serve), the createEmergentQtiRuntime({ catalog }) / buildEmergentPciCatalog delivery helpers, and an admin-console catalog panel. Verified by unit, PGlite, route, and delivery tests plus a real-Postgres integration test (vet → deliverable → serve).

Storage note (refinement of ADR-0012’s “asset pipeline” suggestion): a PCI module is executable JS, so its source lives in a governed server-only column (like qti_source_xml), not the media item-assets bucket (whose allow-list is image/audio/video and is Supabase-owned). Same content- address + integrity + allowlist guarantees, no bucket change. (Scaling many large bundles to Storage is a future option.)

So the only substantive residual that remains is the small conform-ed parser work in §2/§3; §1’s “defer + keep quarantining” decision is superseded.

---

Status: proposed (2026-06-16)

Builds on ADR-0001 (QTI ASI as the canonical model), ADR-0009 (imported-standard boundary + quarantine), ADR-0021 (demonstrable conformance + full coverage), and ADR-0023 (authoring asset pipeline). ADR-0021’s program is built — §1 conformance lane, §3 item+test export + Content-Packages, §4 results emission, §5 native authoring (now incl. endAttemptInteraction), §6 PNP/feedback/adaptive/catalog/stimulus/rubric. This ADR is the triage of what the official 1EdTech corpus still surfaces as not-yet-covered, so “full QTI 3.0” has a named, scoped residual rather than an implicit gap. None of it is built; each item is a decision for Anton.

How the residual was measured

The opt-in full-corpus lane (bun run test:qti:conformance:local, ADR-0021 §1) runs all 327 official 1EdTech qtiv3-examples through emergent’s real pipeline. Current reach: 315/327 valid, 241 items ingested, 9 quarantined, 12 not-valid. The 12 not-valid + 9 quarantined, triaged by actual root cause (not just directory), are the entire residual below.

The tension

emergent is a consumer of conform-ed’s engine (ADR-0021 §2): we never re-implement QTI semantics. Every residual here is therefore one of three things — (1) a conform-ed engine gap (fix upstream, release, bump the pin), (2) an emergent authoring surface that waits on (1), or (3) out of scope by design. The risk is leaving these implicit and later mistaking “the corpus isn’t 100% green” for a regression. Naming them converts an open-ended “are we done?” into a short, owned backlog. Every quarantined/unsupported instance already fails safe — a learner never sees a broken item (ingest quarantines; delivery filters to published) — so none of this is urgent; it is completeness.

Decision

1. Generic Portable/Custom Interaction (PCI) host — the substantive gap (conform-ed)

customInteraction and portableCustomInteraction parse into the document model (the contracts carry QtiPortableCustomInteractionSchema / QtiCustomInteractionSchema), but the reference runtime cannot render a generic PCI, so ingest quarantines them (ADR-0009) — not delivered, not scored. This is the whole quarantine set: the corpus PCI examples (packaging/pci/src/measuring_ph, fractions*). The one PCI that works is the bespoke @conform-ed/pci-math-entry, wired as a first-class interaction rather than via a generic host.

Full support needs a generic PCI host = a sandboxed runtime implementing the IMS PCI communication contract (the PCI JS module API: configuration, getResponse/getState, the render/oncompleted lifecycle, the markup/CSS/module bundling) so any conformant PCI module loads, renders, captures a response, and round-trips state. This is conform-ed engine work and a non-trivial program (a sandbox + a stable host API), released single-version. emergent then adds an authoring surface (upload a PCI module bundle + its config), which depends on the ADR-0023 asset pipeline for the module/asset bytes.

Decision: defer as a standalone conform-ed program (its own ADR there), not part of any emergent lane. emergent’s quarantine behaviour is the correct interim: PCI items import, are marked quarantined with their reason, and never reach a learner. Authoring waits on the host. Revisit when a concrete PCI need appears (most language-learning items are covered by the 19 native kinds + the math-entry PCI).

2. XInclude resolution (conform-ed parser) — tests/rtest01-set01/02/03 + rtest01-fragment

The rtest01-set0N.xml files are assessment items that use <xi:include> to pull a shared content fragment (rtest01-fragment.xml, whose root is a bare <div> — correctly not a standalone ASI document). conform-ed’s parser does not resolve XInclude, so the set items parse-error and the fragment is unsupported (as a standalone doc, correctly).

Decision: a small, well-scoped conform-ed parser task — a pre-parse pass that resolves xi:include against the surrounding package, then parses the assembled document. Fix upstream + release; emergent bumps the pin and the three set items move from backlog into the green set. Until then, XInclude-using imports quarantine/parse-fail — rare in practice, and emergent’s own authored/exported QTI never emits XInclude, so export-conformance is unaffected. Low priority.

3. Rich assessmentResult parsing (conform-ed) — results/full-example.xml

A complete, 221-line assessmentResult (a non-linear test, two items, mapResponse + ordering scoring) parse-errors. emergent’s own emitted results are conformant (ADR-0021 §4 gates prove serialize → validate), so this is import-direction only: reading a third-party results document that exercises a results construct the parser doesn’t yet cover.

Decision: triage upstream in conform-ed — reduce full-example.xml to the failing construct, fix the results parser, release. Add the file to the deterministic conformance fixtures once green. Priority rises only when emergent needs to ingest external QTI results (today we emit, not consume, them).

4. Computer Adaptive Testing (CAT) — separate spec layer

CAT is a distinct QTI 3.0 extension (conform-ed ships cat/v1_0 contracts but no delivery runtime — CAT routes item selection through an external adaptive engine). The corpus CAT/test.xml is an HTML explainer page (<!DOCTYPE html>), not a QTI document, so its “unsupported” mark is a non-issue; the real artifact is CAT_Test01_Adaptive_EchoAdapt.zip.

Decision: out of scope for the current product (summative + formative language assessment). If adaptive testing becomes a goal it is its own program (conform-ed CAT delivery runtime + emergent authoring/integration), tracked in a future ADR. SUPERSEDED by ADR-0037 (2026-06-23): adaptive testing is now in scope as a basic-but-real program — an in-process Rasch/1PL reference engine behind the cat/v1_0 seam, with the industrial engine as a later REST-adapter swap. Still not part of “full QTI 3.0 ASI” — CAT remains a distinct spec layer.

5. Out of scope by design (no action) — the remaining 6 not-valid files

• packaging/{lti,testWithLti}/links/*.xml (4) — LTI link resources (imslti_xmlv1p1), not QTI items. emergent is an LTI tool, not a consumer (ADR-0022); correctly not handled.
• packaging/externalMetadata/testitems/metadata_*.xml (2) — resource-metadata sidecar fragments, not standalone ASI documents.

These stay in the not-valid count permanently and correctly; the lane’s report should annotate them as expected-exclusions so the number isn’t read as a defect.

Consequences

• “Full QTI 3.0” now has a named, owned residual: generic PCI host (§1, the only substantive one), two small conform-ed parser fixes (§2 XInclude, §3 rich results), one distinct spec layer (§4 CAT — no longer out of scope; now a built basic-but-real program per ADR-0037, still separate from “full QTI 3.0 ASI”), and six permanent expected-exclusions (§5).
• All of it is conform-ed-side or its own program; no QTI-corpus feature work is implied beyond PCI authoring, which waits on the host.
• The corpus baseline stays a 315/327 with a documented why for every one of the 12 — a healthy steady state, not an open gap. A future tightening could split the lane’s report into green / engine-backlog (§2/§3) / expected-exclusions (§4/§5) so regressions are unambiguous.
• emergent’s fail-safe posture (quarantine + published-only delivery) means none of this can reach a learner as a broken item.

Decisions for Anton

1. Generic PCI host is a deferred standalone conform-ed program; emergent keeps quarantining generic PCI in the meantime; PCI authoring waits on the host. SUPERSEDED + BUILT (2026-06-16): the host already existed; conform-ed 0.0.25 shipped the tier-2 catalog seam and emergent built the org-admin-vetted tier-2 consumer (see the Update section above). The remaining operator step is the DB reset+apply for the new assessment_pci_module table (the migration set is regenerated + drift-green; the containerized integration lane already applies it). (done)
2. XInclude resolution (§2) and rich-results parsing (§3) are filed as conform-ed parser tasks, fixed upstream + released, low priority (export-conformance and learner delivery are unaffected today). (proposed)
3. CAT (§4) is out of scope for the current product; revisit via a dedicated ADR if adaptive testing is adopted. SUPERSEDED by ADR-0037 (2026-06-23): adaptive testing is now in scope and built basic-but-real (in-process Rasch/EAP engine behind the cat/v1_0 seam); it remains a distinct spec layer, not part of “full QTI 3.0 ASI”. (superseded)
4. The 6 LTI-link + external-metadata files (§5) are permanent expected-exclusions; annotate the conformance report rather than chase them. (proposed)