Caliper Analytics 1.2 — conformance program plan (v1)

Status: planned. Decisions recorded in ADR-0041 (the rail) and ADR-0042 (emission governance).

Product reversed the earlier Caliper deferral and wants maximal coverage before launch. This is the program plan: emergent becomes a fully conformant Caliper 1.2 Sensor and Endpoint, with Caliper riding the same derived telemetry projection as a new xAPI emitter.

The guiding stance: a conformance gap is a product capability the standard surfaced, not a checkbox — and Caliper must stay a cheap derived projection off the existing event log (ADR-0007), never a second instrumentation pass or a second source of truth.

Decisions (from the grilling session)

#	Decision	Choice
1	Roles	Both Sensor (emit) and Endpoint (ingest)
2	Source	One shared projection seam off the event log / domain streams → emits both Caliper Events and xAPI Statements (emergent gains an xAPI emitter; today the LRS only receives)
3	Profiles	All 15 metric profiles; build the missing surfaces
4	New surfaces	Media, Search, Survey as full product features (standards-driven product discovery)
5	Endpoint home	Extend apps/lrs into a dual-protocol telemetry store
6	Board granularity	8 structural MUSTs + Sender & Endpoint role profiles; per-profile coverage on the info-model axis
7	Delivery	Live async batched transport (data[] Envelopes, bearer, at-least-once via interaction_event idempotency keys), self-dogfooded
8	PII / governance	Pseudonymous-by-default actor IRIs (account-IFI); real identity behind per-endpoint trust config; erasure-registry external/ifi; per-endpoint export flag (ADR-0042)
9	Done bar	Internal test:caliper:conformance + dossier; external 1EdTech certification a separate later milestone
10	Plan boundary	This program owns the rail + thin telemetry contracts; Media/Search/Survey each get their own design session

Architecture

A single Telemetry Projection seam reads canonical state/event streams — interaction_event, item_session/practice_session, assessment_attempt, document_progress_event, discussion post/thread, gradebook tables, LTI launch, catalog_resource — and fans out through two wire adapters (Caliper and xAPI), both schema-owned by @conform-ed/contracts. An async batched transport worker assembles Envelopes (data[]) and POSTs them (bearer auth) to configured endpoints, defaulting to our own Caliper endpoint and the xAPI emitter → our own LRS, so both roles run as one real loop. The Caliper Endpoint lives inside apps/lrs, reusing its server, client-credential auth, and storage, persisting to a new caliper.* schema.

Phased plan (upstream-first)

Phase 0 — conform-ed Caliper contracts + board (UPSTREAM; gates everything)

Complete the Caliper Zod contracts for all 15 profiles’ entities/events/vocabularies — building on the existing Bootcamp-derived schemas and CALIPER_TEXTUAL_EVENT_RULES (wiring + value-set verification, not greenfield). Add Sender and Endpoint role profiles to the Caliper board (keep the 8 structural MUSTs). Regenerate the coverage map (today 78/1957 modelled → near-complete), publish @dev, and re-pin in emergent.

Phase 1 — projection seam + xAPI emitter (uses surfaces that already exist)

Build the shared seam for the 11 profiles whose surfaces exist today: Session, Assessment(+Item), Assignable, Grading, Reading, Forum/Thread/Message, ToolLaunch, ToolUse, ResourceManagement, Feedback, Annotation. Pseudonymous actor identity (ADR-0042); per-profile action/verb vocabulary mapping with a value-set drift guard.

Phase 2 — endpoint + transport + dogfood loop

caliper.* schema (Anton’s migration), ingest route + bearer auth + Envelope validation/persist; the async batched transport worker; self-dogfood (Caliper→endpoint, xAPI→LRS).

Phase 3 — conformance lane + board flip + dossier

test:caliper:conformance (emitted artifacts validated against the official Bootcamp schema; endpoint accept-valid / reject-malformed) + a per-profile event-coverage test. Flip overlays/caliper-v1.2.yaml from all-not-applicable to done (8 MUSTs + role profiles); regenerate index.md + the dossier. Result: Caliper “done” for every surface that exists today.

Phase 4 — the three new features (each its own design session)

Each is a real product feature designed separately; each lands its thin telemetry contract, which lights up its profile on the already-built rail (see contracts below).

Phase 5 — external 1EdTech Caliper certification (separate later milestone)

Hosted endpoint + submission.

Thin telemetry contracts for the new features

What each new feature must emit so the rail can light its Caliper profile (and the xAPI peer):

• Media → MediaObject (Video/Audio) + MediaLocation; MediaEvent actions Started/Paused/Resumed/Ended/Restarted/JumpedTo/ForwardedTo/SkippedTo/ChangedVolume/etc. ⇒ MediaProfile (peer: xAPI media profile statements).
• Search → Query + SearchResponse; SearchEvent action Searched (and ToolUse Used) ⇒ SearchProfile.
• Survey → Survey/Questionnaire/QuestionnaireItem + Question/Response entities; SurveyEvent/QuestionnaireEvent/QuestionnaireItemEvent/SurveyInvitationEvent ⇒ SurveyProfile (consider reusing the QTI engine in a non-scored mode).

Governance (ADR-0042 summary)

Emitted Person/Agent carry a stable pseudonymous IRI (the account-IFI pseudonym) by default — a pseudonym is still a “persistent IRI”, satisfying CAL-ENT-1/ID-1 without exporting PII. Real-identity emission is opt-in per trusted endpoint. The caliper.* tables and the external emit get erasure-registry declarations (external class, ifi link); erasure to a foreign store is best-effort by nature, and that limit is documented for certifiers.

Risks / watch-items

• conform-ed contract completion is the long pole, but the Bootcamp schemas + textual event rules already exist — it is wiring + value-set verification.
• The xAPI emitter is new scope riding along; keep it lean by reusing the LRS’s StatementV2Schema.
• Migrations (caliper.* schema + erasure declarations) are Anton’s domain — one batch, regenerated per the migrations runbook.
• Emitted action/verb vocabularies must be value-set-checked (drift guard) so terms stay in the enums.