How it works

New package
Mockup: this portal is a static preview. No authentication, no live data, nothing is saved.
Working specification: this is a build document, not shipped behaviour. Requirement IDs here are intended to be handed to engineering as-is.

TravelSpy insight platform — technical specification

How the tag, the insight engine and the embedded surface fit together

Status Draft v0.1 Owner Platform engineering Last updated 15 Sep 2026 Reviewers Modelling, Security, Privacy

1. Scope & principles

What this system is, and deliberately is not

Core principle. TravelSpy does not handle messages. It handles insights.

The host owns the conversation, the content and the customer relationship. We ingest behavioural signals, derive insight, and return that insight. We never become the system of record for a conversation, we never render the host's commercial content, and we never sit in the delivery path of anything the traveller reads as coming from the brand.

In scope

  • Collecting behavioural, source and contextual signals from the host's own channels.
  • Resolving those signals to a durable traveller profile under consent.
  • Deriving scored, explainable insight from that profile.
  • Delivering insight back in real time, to a machine (API/stream) or to a human (embedded surface).

Explicitly out of scope

  • Storing, routing, moderating or replaying conversation or message content.
  • Acting as a chat provider, inbox, helpdesk or CRM.
  • Owning the traveller's identity or authentication.
  • Taking inventory risk or transacting a booking.
  • Any decision with legal or similarly significant effect made without a human in the loop.

Design tenets

PRIN-1

Insight is a derived artefact, never raw payload passthrough. If a field cannot be justified as model input, it is not collected.Test: every stored field maps to a documented feature.

Must
PRIN-2

Every insight ships with a confidence value and its evidence. An unexplainable score is a defect, not a feature.

Must
PRIN-3

The host site must remain fully functional if TravelSpy is unavailable. All integration points degrade silently.

Must
PRIN-4

Consent state travels with the record and is enforced at the point of use, not only at the point of collection.

Must
PRIN-5

Prefer boring, observable infrastructure. Novelty belongs in the models, not the pipes.

Should

2. System overview

Signal in, insight out, two delivery paths

Host sitePages, search, booking flow. Owns all content.
Host CMPConsent decision, authoritative.
Host backendBookings, CRM, server-side events.

↓  signals only (no message content)  ↓

CollectorEdge tag. Batches, buffers, consent-gated.
Ingest APISchema validation at the boundary.

↓  validated event envelope  ↓

Identity resolutionSession → device → traveller.
Feature storeOnline + offline, shared definitions.
Insight engineScores, confidence, evidence.

↓  insight envelope  ↓

Pull: Insight APIREST. For servers and batch consumers.
Push: insight streamSSE. For anything that must be live.
Sync: destinationsCRM, warehouse, ad platforms.

↓  rendered for a human  ↓

Embedded surface (iframe)Real-time feedback only. Subscribes to the stream, renders insight, posts intent back to the host.

3. Component responsibilities

Who owns what, and what each side may assume

ComponentOwnsMust never
Collectorpublic/js, ~9 KB budgetSignal capture, batching, consent gate, offline bufferRender UI, block the main thread, read form values
Ingest APIAuthN, schema validation, rate limiting, quarantineAccept an unvalidated envelope into the bus
Identity resolutionStitching, the traveller ID, consent propagationJoin on an identifier lacking a consent basis
Feature storeFeature definitions shared by training and servingLet online and offline definitions diverge
Insight engineScores, confidence, evidence, model versioningEmit a score without evidence or version
Insight APISynchronous reads, scoped tokensReturn data outside the token's workspace scope
Insight streamLive delivery, resume, backpressureHold state the client cannot rebuild from a snapshot
Embedded surfaceTwo variants, see §8Rendering insight for its declared audience, capturing structured intentStore conversation, price outside the host's guardrails, navigate the top window

4. Data contracts

Two envelopes. Everything else is internal.

4.1 Signal envelope (inbound)

One shape for tag and server-side ingestion. Rejected at the boundary if it fails schema validation; rejected payloads go to a quarantine topic, never to the bus.

{
  "schema": "travelspy.signal.v1",
  "workspace": "ws_live_9f21c480",
  "occurred_at": "2026-04-02T09:14:22.418Z",
  "idempotency_key": "01JQ8W3K7P2F5N9Z",
  "subject": {
    "session_id": "ses_7f21aa",
    "device_id": "dev_c40b91",
    "identifier": null            // hashed, only when consented
  },
  "consent": { "analytics": true, "personalisation": true, "advertising": false },
  "context": {
    "url_path": "/japan/kansai",  // path only, never query string
    "referrer_class": "editorial",
    "locale": "en-GB",
    "device_class": "desktop",
    "network": { "country": "GB", "type": "broadband", "vpn": false }
  },
  "signal": {
    "name": "destination_viewed",
    "attributes": {
      "destination": "JP", "region": "Kansai",
      "trip_type": "culture", "nights": 11,
      "party": { "adults": 2, "children": 0 }
    }
  }
}

4.2 Insight envelope (outbound)

Identical shape whether it arrives by REST, by stream or by destination sync. The embedded surface renders this and nothing else.

{
  "schema": "travelspy.insight.v1",
  "traveller_id": "trv_8813af",
  "computed_at": "2026-04-02T09:14:22.996Z",
  "sequence": 4182,               // monotonic per traveller, drives resume
  "model_versions": { "affinity": "18.3", "spend": "11.0", "window": "9.4" },
  "insights": [
    {
      "key": "destination_affinity",
      "value": { "code": "JP", "region": "Kansai" },
      "confidence": 0.91,
      "evidence": [
        { "signal": "repeat_destination_views", "weight": 0.94, "observations": 11 },
        { "signal": "referrer_class", "weight": 0.81, "observations": 6 }
      ]
    },
    {
      "key": "spend_band",
      "value": { "band": 6, "range": [6400, 8200], "currency": "GBP" },
      "confidence": 0.84,
      "evidence": [ { "signal": "premium_option_dwell", "weight": 0.77, "observations": 2 } ]
    },
    {
      "key": "booking_window",
      "value": { "opens_in_days": 9, "closes_in_days": 21 },
      "confidence": 0.72,
      "evidence": [ { "signal": "date_picker_refinement", "weight": 0.62, "observations": 4 } ]
    }
  ],
  "suppressed": ["advertising"]   // consent-derived, enforced at use
}
CTR-1

Both envelopes MUST carry an explicit schema version. Consumers MUST reject unknown major versions rather than best-effort parse.

Must
CTR-2

Schema changes are additive within a major version. Removing or retyping a field requires a new major version and a documented dual-write window.

Must
CTR-3

Every signal MUST carry an idempotency_key. Ingest MUST deduplicate on it for at least 24 hours, because the collector retries.

Must
CTR-4

The insight envelope MUST NOT contain free text originating from a traveller. Values are enums, numbers, ranges and codes only.This is what keeps us out of the message-handling business.

Must
CTR-5

sequence SHOULD be monotonic per traveller so a reconnecting client can ask for everything after a known point.

Should

5. Collector (the script)

The only code we ask the host to deploy

COL-1

Loader MUST be async and MUST NOT block rendering. Total transferred budget 10 KB gzipped for the loader plus core.

Must
COL-2

No network request of any kind before a consent decision is recorded. A dormant tag is the default state.Acceptance: fresh session with no CMP response produces zero requests to our origin.

Must
COL-3

MUST capture only declared signals. No automatic DOM scraping, no keystroke capture, no form value capture, no query strings.

Must
COL-4

Batch signals and flush on a timer, on batch size, and on visibilitychange to hidden. Final flush MUST use navigator.sendBeacon so it survives unload.

Must
COL-5

Buffer to memory when offline; retry with exponential backoff and jitter. Drop the buffer, never the host page, on sustained failure.

Must
COL-6

Expose TravelSpy('forget') to erase locally and trigger server-side propagation.

Should
COL-7

SHOULD self-report tag health (version, coverage, error rate) so the portal install page reflects reality.

Should
COL-8

Server-side ingestion parity for hosts that will not ship a browser tag.

Later

6. Insight engine

Where signal becomes something worth acting on

INS-1

Serving and training MUST read feature definitions from the same store. A feature computed two ways is a defect.

Must
INS-2

Every emitted insight carries confidence, evidence[] and the model_version that produced it.

Must
INS-3

Recompute is incremental and event-driven. A single new signal MUST NOT trigger a full profile rebuild.

Must
INS-4

Emit an insight only when it materially changes. Define per-key thresholds; suppress noise below them.Rationale: the stream is a change feed, not a metronome.

Must
INS-5

Drift monitoring with automatic rollback to last known-good version. Bias testing against protected attributes before any promotion.

Must
INS-6

Human overrides are captured and fed back as training signal, and are visible in the evidence trail.

Should
INS-7

Customer-supplied models registered alongside the TravelSpy library, scored through the same contract.

Later

7. Real-time transport

How live insight actually reaches a browser

7.1 Transport decision

OptionFitVerdict
Server-sent eventstext/event-streamOne-way server→client, which is exactly our shape. Auto-reconnect, Last-Event-ID resume and retry: backoff are built into the browser.Chosen
WebSocketFull duplex we do not need, plus our own reconnect, heartbeat and resume logic. More proxy and CDN friction.Rejected
Long pollingWorks everywhere. Higher latency and request overhead.Fallback

Known constraint that must shape the design. Over HTTP/1.1, browsers cap concurrent connections at roughly six per browser + domain, shared across tabs. A traveller with several tabs open on the host site could exhaust that budget and starve the host's own requests.

Therefore: the stream MUST be served over HTTP/2 or later (where concurrent streams are negotiated, typically 100), and the client MUST hold one connection per browser, not one per tab or per frame.

7.2 Requirements

RT-1

Stream served as text/event-stream over HTTP/2+, with Cache-Control: no-cache and proxy buffering disabled.

Must
RT-2

Exactly one stream connection per browser. Fan out to tabs and frames via BroadcastChannel, with a leader election so a single tab owns the socket.Directly mitigates the six-connection cap.

Must
RT-3

Every event carries id: set to the insight sequence. On reconnect the server MUST honour Last-Event-ID and replay only what was missed.

Must
RT-4

Server sends a comment keep-alive at least every 20 seconds to stop intermediaries closing an idle connection, and sets retry: explicitly.

Must
RT-5

Replay buffer is bounded. If a client is too far behind, the server MUST send a snapshot event instead of a replay and the client MUST accept it as authoritative.

Must
RT-6

Close the stream when the document is hidden beyond a grace period; reopen on visible. Battery and connection budget are not ours to spend.

Must
RT-7

Coalesce rapid changes to the same insight key within a short window so the UI does not flicker.

Should
RT-8

Automatic downgrade to long polling when EventSource is unavailable or repeatedly fails.

Later

7.3 Wire format

: keep-alive

retry: 3000

event: insight.changed
id: 4182
data: {"traveller_id":"trv_8813af","key":"booking_window",
data: "value":{"opens_in_days":9},"confidence":0.72}

event: insight.snapshot
id: 4183
data: {"traveller_id":"trv_8813af","insights":[ ... ]}

event: consent.withdrawn
id: 4184
data: {"traveller_id":"trv_8813af","scopes":["personalisation"]}

8. The iframe surfaces

Two embeds, two audiences, one stream

Decision. There are two embedded surfaces, not one. They consume the same insight stream and differ only in who is looking at them.

Conflating them was the original design error: a panel that tells someone to “present the full trip now, do not discount” is written for your revenue team, and must never be rendered to the person being described.

8.1 The two surfaces

SurfaceAudienceRecommends?Shows
Traveller surface/embed/traveller The traveller, on the host's public site Yes — a packaged trip Plain-language confidence (“we think…”), an assembled itinerary, a way to correct us
Team surface/embed/team Agents, advisors and commercial staff, behind authentication Yes — a next best action Numeric confidence, evidence, spend band, margin guidance

Amendment to §3. The component table previously said the embedded surface must never render commercial content. That was too broad and contradicted the traveller surface, which exists precisely to present an assembled trip. The rule is narrower: we render offers built from the host's own contracted inventory, priced inside the host's guardrails. We never invent inventory, never price outside the floor, and never become the merchandising system of record.

8.2 Behaviour, both surfaces

EMB-1

Subscribes to the insight stream for the current traveller and re-renders on change. Cold start MUST render from a snapshot before the first delta arrives.

Must
EMB-2

Holds no durable state. Everything it displays MUST be reconstructible from a snapshot after a hard reload.

Must
EMB-3

Captured input is constrained: selections, confirmations, corrections and numeric ranges. No free-text field that we persist.Acceptance: no request body from either frame contains an author-written string.

Must
EMB-4

Renders a visible confidence indication. Nobody is shown a guess presented as a fact.

Must
EMB-5

Degrades to a static last-known state if the stream drops, with a visible stale indicator. It MUST NOT show an error the host's customer has to interpret.

Must
EMB-6

Reports its content height to the parent so the host can size the frame. Iframes expose no size information by default, so this is an explicit postMessage until native responsive sizing is broadly available.

Should
EMB-7

Themeable via URL parameters only. No host CSS injection, no style leakage in either direction.

Should
EMB-8

Agent-facing variant, later. Superseded — promoted to a first-class surface by EMB-9 onward.

Closed

8.3 Separating the audiences

EMB-9

The two surfaces ship from separate URLs with separate tokens. A workspace may enable either or both. One surface MUST NOT be reachable by changing a parameter on the other.

Must
EMB-10

The team surface token is issued only to an authenticated staff context and MUST be rejected if the embedding origin is a public host page.Acceptance: pasting a team embed onto the public site renders nothing.

Must
EMB-11

Commercial vocabulary — margin, discount, propensity, spend band, lead score — MUST NOT appear in the traveller surface. Enforce with a copy lint in CI against a banned-term list.

Must
EMB-12

Confidence is presented differently per audience: plain language on the traveller surface (“we think”, “fairly sure”), numeric plus evidence on the team surface.

Must
EMB-13

The traveller surface MAY present a packaged trip, assembled from host-contracted inventory and priced within host guardrails. It MUST NOT display a price the host's own systems would not honour.

Must
EMB-14

Both surfaces consume the identical insight envelope. Any divergence is presentation-only — there is no second model and no second truth.

Should
EMB-15

A correction made by the traveller (“actually it is three adults”) propagates to the team surface within one stream tick, so an agent is never working from a stale belief.

Should

8.4 Parent ↔ frame protocol

Both directions are strictly typed and origin-pinned. The frame is the source of truth for insight; the host is the source of truth for page context.

DirectionTypePurpose
frame → hostadvisor.readyFrame booted, stream connected
frame → hostadvisor.resizeContent height changed
frame → hostadvisor.insight_changedHost may mirror insight into its own UI
frame → hostadvisor.intentTraveller expressed structured intent; host decides what to do
host → framehost.contextCurrent page, product or departure in view
host → framehost.consentConsent changed; frame must re-evaluate immediately
// In the frame. Never post to "*".
parent.postMessage(
  { source: 'travelspy-advisor', type: 'advisor.intent',
    payload: { key: 'destination_confirmed', value: 'JP' } },
  HOST_ORIGIN                     // configured per workspace, verified server-side
);

// In the host.
window.addEventListener('message', function (e) {
  if (e.origin !== 'https://advisor.travelspy.io') return;   // 1. pin the origin
  if (!e.data || e.data.source !== 'travelspy-advisor') return;
  if (!ALLOWED_TYPES.has(e.data.type)) return;               // 2. validate the shape
  handle(e.data.type, e.data.payload);
});

9. Security

Assume the host page is hostile and the frame is a target

SEC-1

The widget MUST be served from a dedicated origin, separate from the marketing site and the portal, so a compromise cannot pivot into a session.

Must
SEC-2

targetOrigin MUST always be an exact origin. Using * is prohibited, because a host can navigate the frame and intercept the message.

Must
SEC-3

Every receiver MUST verify event.origin and validate message shape. Any window in the frame hierarchy can post to any other.

Must
SEC-4

Embedding is restricted by a per-workspace origin allowlist, enforced with Content-Security-Policy: frame-ancestors. An unregistered origin gets no frame and no token.

Must
SEC-5

Never combine allow-scripts with allow-same-origin on a same-origin embed — the framed document can then remove its own sandbox, making the attribute worthless.

Must
SEC-6

Stream and API tokens are short-lived, workspace-scoped and traveller-scoped. A leaked embed token MUST NOT read another traveller.

Must
SEC-7

The frame MUST NOT request top navigation. Browsers already gate this behind sticky activation; we do not rely on that as our only control.

Must
SEC-8

Ship a strict CSP on the widget document: no inline script, no eval, connect-src limited to our API and stream origins.

Should
SEC-9

Evaluate credentialless iframes for hosts running COEP. Chromium-only today, so not a baseline requirement.

Later

10. Privacy & consent

The constraint that shapes the architecture

PRV-1

No profile is created and no insight computed until a consent decision is recorded. The host CMP is authoritative.

Must
PRV-2

Consent state is carried on the record and enforced at the point of use. A traveller with analytics but not advertising consent is modelled but never exported to an ad platform.

Must
PRV-3

Withdrawal propagates to every connected destination and returns a completion receipt. Target: under 60 seconds internally, under 24 hours end to end.

Must
PRV-4

Retention is configurable per market and enforced by deletion, not archival.

Must
PRV-5

No special category data, no financial account data, no precise GPS, no private message content, at any layer.

Must
PRV-6

Every profile read is written to an audit log that survives the profile itself.

Should

11. Budgets & service levels

Numbers to build against and to test

MeasureTargetBreach
Collector transfer size≤ 10 KB gzippedBlocks release
Collector main-thread time≤ 15 ms per pageBlocks release
Signal → insight latencyp95 ≤ 900 msPage on-call
Insight API readp95 ≤ 200 ms at edgePage on-call
Stream deliveryp95 ≤ 400 ms from computePage on-call
Widget first meaningful render≤ 1.2 s on 4GBlocks release
Stream availability99.9% monthlySLA credit
Ingest durabilityNo acknowledged signal lostIncident review

12. Failure modes

What happens when each part breaks

FailureRequired behaviourTraveller sees
CDN downTag never loads; host page unaffectedNothing
Ingest unavailableCollector buffers, backs off, drops buffer rather than degrading the pageNothing
Engine degradedServe last known insight with a staleness marker; suppress low-confidence keysSlightly older guidance
Stream dropsBrowser auto-reconnects and resumes from Last-Event-ID; snapshot if too far behindBrief stale indicator
Widget origin blockedHost renders its own fallback; no error surfaced by usNothing
Consent withdrawn mid-sessionFrame clears immediately, stream closes, deletion propagatesWidget disappears cleanly
Model drift detectedAutomatic rollback to last known-good version, alert raisedNothing

13. Delivery roadmap

Each phase ships something usable and has an explicit exit test

Phase 0Complete

Prove the surfaces

  • Portal mockup, install page and embeddable widget shell.
  • Contracts sketched and reviewed; this document.

Exit: stakeholders agree the insight-not-messages boundary.

Phase 1Next

Signal in

  • Collector with consent gate, batching and beacon flush (COL-1–5).
  • Ingest API with schema validation and quarantine (CTR-1–3).
  • Identity resolution to a stable traveller ID.
  • Install page reports real tag health (COL-7).

Exit: a real host site produces validated signals for seven days with zero page-performance regressions.

Phase 2

Insight out (pull)

  • Feature store with shared definitions (INS-1).
  • First three insight keys: affinity, spend band, booking window.
  • Insight API with scoped tokens and evidence payloads (INS-2).
  • Portal traveller view reads live data instead of fixtures.

Exit: p95 read latency under 200 ms and every score renders its evidence.

Phase 3

Insight out (push)

  • SSE stream over HTTP/2 with keep-alive and retry: (RT-1, RT-4).
  • Resume via Last-Event-ID plus bounded replay and snapshot fallback (RT-3, RT-5).
  • Single connection per browser via BroadcastChannel leader election (RT-2).
  • Change-threshold emission so the stream stays quiet (INS-4).

Exit: six tabs open on the host site consume one connection, and a forced network drop resumes with no duplicate or missing sequence.

Phase 4

The live surfaces

  • Both surfaces consume the stream and re-render on change (EMB-1–2, EMB-14).
  • Separate URLs, separate tokens, staff-only gating on the team surface (EMB-9–10).
  • Audience copy lint in CI and per-audience confidence rendering (EMB-11–12).
  • Structured intent capture and the parent protocol (EMB-3, 8.4).
  • Origin allowlist, frame-ancestors and scoped embed tokens (SEC-4, SEC-6).
  • Stale and degraded states designed, not improvised (EMB-5).

Exit: penetration test passes with no finding above low, a team embed pasted onto a public page renders nothing, and the copy lint blocks a build containing the word “margin” in traveller strings.

Phase 5

Scale and prove

  • Destination sync with consent enforced at export (PRV-2).
  • Randomised holdout and incrementality reporting wired to real data.
  • Drift monitoring, automatic rollback, bias testing (INS-5).
  • Correction propagation from traveller to team surface (EMB-15).

Exit: a measured conversion lift published from a holdout, reconciled against the customer's own reporting.

14. Open decisions

Needed before Phase 2 closes

QuestionWhy it mattersNeeded by
Who owns the traveller ID?Our identifier, or the host's?Determines whether profiles are portable between hosts and how erasure propagatesPhase 1
Per-key change thresholdsSets how chatty the stream is and therefore the entire cost modelPhase 2
Banned-term list for traveller copyEMB-11 needs a concrete list before the copy lint can be builtPhase 4
Structured intent taxonomyThe constraint that keeps us out of free text; must be expressive enough to be usefulPhase 4
Multi-host traveller identityCommercially attractive, privately fraught. Needs a legal position, not just a technical onePhase 5
Long-polling fallbackOnly worth building if we see real EventSource failures in the fieldPhase 5
Raise an amendment against this document rather than editing requirement IDs in place — they are referenced from tickets.