Common Cartridge 1.4 import: decompose into native structures + a QTI 1.2→3 bridge

Status: accepted

Builds on ADR-0001 (QTI 3 ASI as the canonical assessment model), ADR-0009 (imported-standard boundary: Ingest QTI, keep H5P/SCORM/cmi5 opaque), and ADR-0021 (QTI conformance; engine work upstream, emergent exposes it). ADR-0009 named Common Cartridge “a transport that is decomposed on import, never run” but left the mechanism unspecified and made one assumption that grounding has since falsified (see §LTI). This ADR fixes the mechanism.

The tension (named)

CC 1.4 is a heterogeneous transport: a manifest <organizations> tree plus typed <resources> (QTI assessments/question-banks, webcontent, web links, discussion topics, LTI links, assignment/variant extensions). “Full import that maps onto native structures, not a blackbox” (Anton) therefore means decompose the whole cartridge into a new Offering, not run it as one opaque unit.

Two facts discovered while grounding this ADR shape every decision:

1. CC 1.4 carries QTI ASI 1.2.1 (questestinterop), not QTI 3. conform-ed ships the Zod contract for the CC-profiled 1.2.1 (ccv1p4_qtiasiv1p2p1_v1p0.ts), but its engine (@conform-ed/qti-xml parse/normalize/validate/serialize; @conform-ed/qti-react deliver/score) only knows QtiVersion = "2.2" | "3.0.1". There is no 1.2 normalizer, no 1.2→3 conversion, no headless projection for 1.2. emergent’s ingest (qti-ingest.ts) is hardwired to QTI 3, so it cannot consume CC’s QTI as-is. The product team’s hypothesis was correct: existing QTI support cannot be blindly reused.
2. emergent is an LTI tool, not a platform/consumer. lti-tool.ts models emergent being launched by external platforms (it stores a platform’s issuer/client_id/jwks_url; AGS publishes scores back; NRPS reads membership in). A CC imsbasiclti link needs the reverse — emergent launching an external tool — which does not exist, and CC links are often LTI 1.1 (no 1.3 registration) which emergent does not support at all. ADR-0009’s “LTI links route to the LTI layer” assumed a consumer layer that isn’t built.

Pre-launch (no users/data; [[feedback_no_legacy_pre_launch]]) and conform-ed is ours ([[feedback_internal_lib_release_all_packages]]): missing capability is built upstream and exposed here, never worked around in-app.

Decision

1. Course-shaped decompose: one .imscc → one new Offering

The manifest <organizations> tree maps to a Learning Unit tree — a branch <item> → a grouping Unit, a leaf <item> referencing a resource → an activity Unit holding the mapped content. Each <resource> routes to a native landing zone (table below). CC identifiers are kept as provenance, never as keys; each import mints fresh UUIDs. There is no reconcile: re-importing the same cartridge creates another Offering (the OneRoster-style living reconcile, ADR-0012, is deliberately not repeated for a static content package).

2. The QTI 1.2→3 bridge lives upstream and emits QTI 3 XML

@conform-ed/qti-xml gains convertCcQtiV1ToV3(questestinteropXml) → QTI 3.0.1 ASI XML. emergent’s existing ingestQtiAssessmentItem runs unchanged on the output — same validate→normalize→view→capability→quarantine path. Converted items are therefore real QTI 3 artifacts: they deliver, score, round-trip, and export (ADR-0021) for free, and are provable against the official QTI 3 corpus. The CC-profiled 1.2.1 subset (response_lid Single/Multiple/ Ordered, response_str, render_choice, render_fib, material/mattext, hints, solutions, resprocessing) maps cleanly onto QTI 3 choiceInteraction / textEntryInteraction / extendedTextInteraction + response processing, so the conversion is bounded, not open-ended.

A CC question-bank converts to loose Item-Bank Items; a CC assessment (a 1.2 test with sections + items) converts to QTI 3 assessmentTest XML and lands as an Assessment Test (assessment_definition) referencing the ingested Items — preserving the test structure the cartridge encodes. This requires a new test-level ingest (today’s path is item-only) reusing the ADR-0020 assessmentTest model.

3. Decompose is a new @conform-ed/common-cartridge package; emergent owns the mapping

The new package takes the .imscc bytes, unzips (fflate), validates imsmanifest.xml against the CC 1.4 contracts, and returns a neutral DecomposedCartridge — the organizations tree plus resources[] typed by CC resource-type, with hrefs + file contents. It depends on @conform-ed/contracts + @conform-ed/qti-xml (for the bridge). emergent owns all domain mapping (CC type → Item Bank / Assessment Test / static-package / Unit tree). This is the wire.ts line of externalisation-boundary-v1.md: spec-shaped artifact handling upstream, integration-shaped mapping in emergent.

4. Resource routing table

CC 1.4 resource	Native landing
QTI question-bank	Item Bank Items (bridge → QTI 3 → existing ingest)
QTI assessment	Assessment Test (assessment_definition) + its Items
webcontent, top-level org item	static-package External Package on an activity Unit
webcontent, dependency of another resource	bundled asset of that parent resource (not its own Unit)
EPUB3 (idpfepub_epubv3p0), or any .epub/.pdf shipped as webcontent	document External Package → in-app reader (ADR-0026) — BUILT; detection is content-sniffed, not type-trusted, so it catches disguised and unzipped-OCF books too (see Implementation note below)
web link (imswl_*)	native learning_unit_external_link on its activity Unit — BUILT (import + teacher report; learner render is the shared discovery follow-up; see cc-long-tail-landing-analysis.md)
LTI link, discussion topic, assignment, IWB, APIP, Open Video	quarantined into the import report with a friendly reason + preserved payload (no native home yet — analysis above)

webcontent needs a new web-content External Package kind/launchKind — only the static-package runtime provider exists today. Arbitrary HTML/JS is not expressible in the canonical model, so it is served as static files rather than pretending to be transparent; this does not violate ADR-0009, whose transparent/opaque line was always QTI-specific.

5. “Full” = nothing silently dropped

Every resource is either mapped to a native structure or recorded in a per-import report with its preserved payload and a reason — mirroring the QTI-ingest quarantine. The report and provenance live on a cartridge_import record with a status lifecycle (pending → ready → failed, mirroring package_import_status). v1 maps QTI + webcontent; LTI / web link / discussion / assignment are quarantined-with-reason, each a named follow-up (LTI being the largest — it needs a whole LTI consumer layer).

6. Surface + processing

A teacher (or org-admin) uploads a .imscc in teacher-portal to bootstrap their own Offering — the normal Canvas/Moodle/Blackboard workflow — authorized like offering creation (teacher-manage / org-admin), distinct from the admin-only QTI item-ingest panel. Processing is async, keyed on the cartridge_import record; the report surfaces on completion.

7. Conformance (mirrors ADR-0021)

No CC fixtures exist yet (conform-ed has only the QTI-3 corpus). Vendor a small set of real CC 1.4 cartridges; the lane = decompose assertion + bridge round-trip (1.2.1 → QTI 3 → validateQtiXmlContent, then deliver/score the projection) + an emergent import integration test asserting the Offering / Unit tree / Item Bank / Assessment Test / report. A .local lane exercises a larger corpus (the conform-ed pattern).

Considered and rejected

• Content-only import (dump QTI into the Item Bank, ignore the org tree and non-QTI resources): rejected — discards the course structure the cartridge encodes, contrary to “map onto native structures.”
• Convert QTI 1.2 inside emergent, or project straight to the headless view skipping QTI 3 XML: rejected — violates the upstream-first rule (ADR-0021 §2) and forfeits the free export/round-trip/conformance the QTI-3-XML target buys.
• Build LTI consumer support now so CC LTI links launch: deferred — a large standalone program (1.1 + 1.3 launch, security, custody) that dwarfs the rest of the import. Quarantined meanwhile. The whole quarantined long tail (LTI / discussion / assignment / web-link) and its native-landing options are sized in cc-long-tail-landing-analysis.md.
• A new native Page/Content model for webcontent: deferred — large new surface that still cannot faithfully hold arbitrary HTML/JS. static-package is the honest v1 landing.
• Reconcile-on-re-import (match CC ids, version in place): rejected for v1 — unjustified reconcile-engine cost for static packages (cf. the OneRoster reconcile, ADR-0012).

Consequences

• conform-ed gains convertCcQtiV1ToV3 (in qti-xml) and a new @conform-ed/common-cartridge package; both released single-version and pinned in emergent. Any 1.2-subset case the bridge mishandles is fixed upstream, never patched in emergent.
• emergent gains: a cartridge_import record + report, a new web-content External Package kind, a test-level QTI ingest path, a CC mapping service, an async import endpoint + teacher-portal upload, and CC conformance fixtures/lane.
• ADR-0009’s LTI-routing line is superseded: CC LTI links are quarantined until a consumer layer exists. ADR-0009 carries a status note pointing here.
• Executed incrementally (feature branch per slice, rebased to land — [[feedback_rebase_never_merge]]); docs/design/common-cartridge-import-plan.md is the slice plan.

Decisions for Anton (resolved in the 2026-06-15 grilling)

1. Scope = course-shaped (Offering + Unit tree + routed content), not content-only. (accepted)
2. QTI 1.2→3 bridge in conform-ed, emitting QTI 3 XML, reusing emergent’s existing ingest unchanged. (accepted)
3. Decompose in a new @conform-ed/common-cartridge returning a neutral DecomposedCartridge; emergent owns all mapping. (accepted)
4. webcontent → static-package External Package (new web-content kind); deps bundled. (accepted)
5. Long-tail (LTI/web link/discussion/assignment) quarantined-with-report in v1; map only QTI + webcontent. “Full” = nothing silently dropped. (accepted)
6. CC assessment → Assessment Test + Items; question-bank → loose Items (new test-level ingest). (accepted)
7. Fresh Offering + cartridge_import provenance/report per import; no reconcile. (accepted)
8. Trigger in teacher-portal, teacher/org-admin authz, async. (accepted)

Implementation status (2026-06-17)

BUILT (branch qti-authoring-breadth). The full import path is live: decompose → fresh Offering + Learning Unit tree → QTI ingest (assessment/question-bank) → webcontent → web-content External Package + the cartridge_import provenance/report record. Two deviations from the design, with their justification:

• web-content serving runtime built (not just the kind). The web-content package_kind / launch_kind enum values were added, and webcontent resources are bundled (their files, verbatim paths preserved so relative refs resolve) → stored unzipped under a per-version key prefix in the existing external-package-runtime-packages bucket (no new bucket) → registered as a web-content External Package + version + static-package asset + assignment. Learner serving is a proxy, not a redirect (a redirect would rebase the page’s relative assets onto the storage origin): the bearer-guarded resolve route (GET /workflow/web-content-packages/:assignmentId/delivery) checks offering entitlement and mints a short-lived signed path token; the unauthenticated serve route (GET /web-content/:assignmentId/:token/:assetPath, deliberately outside /workflow so the auth guard doesn’t apply) verifies the token and streams one bundle file, so index.html and its relative assets all load from the same token-scoped space. The learner-web viewer renders the entry in a sandbox="allow-scripts" cross-origin iframe (opaque origin — no access to the learner app, cookies, or storage). The token is HMAC-signed with WEB_CONTENT_DELIVERY_TOKEN_SECRET (mirrors ONEROSTER_PROVIDER_TOKEN_SECRET); if unset the serve/resolve routes 503 and nothing else is affected. The serve route only streams files declared in the bundle manifest, and rejects path traversal/absolute paths.

• Import is synchronous, not async. There is no background-worker infrastructure, so the import runs in-request and returns its report (and cartridgeImportId) directly. The cartridge_import record carries the designed status lifecycle (pending → ready → failed, a dedicated cartridge_import_status enum) and is written pending up front so failures leave a trail — the seam an async/polling flip hangs off later, with no schema change.

Implementation note (2026-06-17) — EPUB3 routing BUILT (content-sniffed, not type-trusted)

The routing table’s EPUB3 → document External Package line is now built. Two findings shaped it:

• Detection is content-based, not type-based. Real cartridges sometimes ship an .epub/.pdf as a generic webcontent resource rather than under the EPUB3 resource type — so trusting the declared type would silently mis-land those as static web content (or quarantine them). Instead the importer sniffs the resource’s entry file (href, else the first declared file) with the reading lane’s detectDocumentKind (the EPUB OCF application/epub+zip header / %PDF-). This catches a document whatever its declared type, runs safely ahead of the kind routing (HTML/QTI never sniff as a document), and avoids false positives from an incidental PDF asset inside an HTML page (only the entry is sniffed, not every file).
• One landing path with the upload lane. A detected document lands through the same uploadDocument seam as an org-admin upload (ADR-0026): one opaque object on the foliate runtime, assigned into the import’s Offering + Unit, so it is immediately readable + offline-cacheable. The import report gained a documents array (surfaced in the teacher-portal report as “Readings imported”); no schema change — it rides the existing report_payload.

Unzipped EPUBs are now handled too (2026-06-17). A cartridge can embed an EPUB’s internal OCF files (mimetype, META-INF/container.xml, the OPF, chapters…) across one webcontent resource rather than a single .epub — the entry sniff alone would miss that. So after the single-file sniff, the importer looks for the OCF-mandated META-INF/container.xml among the resource’s files (detectUnzippedEpubRoot, content-based like the sniff — a plain web-content bundle never carries one), then repackages the tree under that root into one valid .epub (repackageUnzippedEpub: canonical mimetype first/stored, prefix stripped so internal paths stay relative) and lands it through the same uploadDocument seam. The repackaged bytes round-trip back through detectDocumentKind as an epub. Both packaged and unzipped EPUBs (declared-type and disguised-as-webcontent) now reach the reader.

Implementation note (2026-06-17) — §7 conformance lane BUILT

The conformance lane is live. Two real, MIT-licensed cartridge exports (instructure/common-cartridge-viewer, the same corpus conform-ed pins) are vendored under tests/conformance/fixtures/common-cartridge/ with provenance + SHA-256 pins. test:cc:conformance (common-cartridge.conformance.test.ts) runs the real import pipeline over PGlite (real schema, real SQL, real QTI ingest — deterministic + offline, like the QTI lane) and asserts the persisted Offering + Unit tree, that CC QTI ingests as real QTI 3 (≥4 items: mc/tf/mr/essay), that web content lands, and that the cartridge_import record settles ready. The split with conform-ed is clean: the decompose + QTI 1.2.1→3 bridge round-trip are gated upstream over the same fixtures (cc-conformance.test.ts); this lane proves the emergent half — that those resources persist as native structures here — rather than duplicating the bridge check.

The lane’s spine is §5 made executable: expectNothingSilentlyDropped asserts every declared <resource> is accounted for in the import report (mapped or quarantined-with-reason). An opt-in test:cc:conformance:local (cc-corpus.local.test.ts, EMERGENT_CC_CORPUS_DIR else the conform-ed viewer corpus) holds that invariant over a larger set — locally, 5 real cartridges including a 202-resource Canvas course, all importing end to end with nothing dropped. This lane is also the real-execution integration assertion Decision §6/plan-Slice-4 named (it drives the actual service against real cartridges); a separate container-backed Postgres lane was judged redundant on top of it.

Implementation note (2026-06-17) — report-surface + document-count follow-ups

Two small completeness gaps are closed. (1) Web content is now surfaced in the teacher-portal import report — webContents was landing + serving server-side but the teacher result type carried only assessments/questionBanks/documents; it now carries web content too (a “Web content imported” section). (2) cartridge_import.imported_document_count was added (the migration set regenerated via the runbook; DB reset/apply remain Anton’s), so EPUB/PDF documents are counted in a dedicated column rather than only inside report_payload. (3) Long-tail quarantine payload preservation (O-4 / Slice 0): each quarantined LTI / discussion / assignment / web-link resource now carries a friendly, kind-specific reason plus a preserved title and detail (the weblink / LTI launch URL where present), surfaced in the teacher report — §5’s “preserved payload and a reason” made real, riding report_payload with no schema change. A native home for that long tail (vs. quarantine) remains deferred by design — its analysis is cc-long-tail-landing-analysis.md.

Implementation note (2026-06-17) — web-link native landing (first long-tail home)

web-link is the first long-tail resource to get a native home instead of quarantine. A new learning_unit_external_link table (offering + activity Unit + title + url + source_system; no person data, so no erasure entry — like unit_release_rule, and not client-synced yet) holds the titled URL. The importer’s landWebLink extracts the <url href> and inserts the row on the resource’s Unit, reporting imported.webLinks[]; cartridge_import gained imported_web_link_count, and the teacher report shows a “Links imported” section. A link with no URL, or one not placed in the org tree (no Unit), still quarantines-with-reason. Learner rendering is deferred to a shared discovery surface: the learner offering page is offline-sync-first and lists only synced assessments — web-content, documents, and web-links all await either joining the sync rail or a learner resource-list. So today the link is captured natively and visible to the teacher; surfacing it to learners is the shared next step.