How the tag, the insight engine and the embedded surface fit together
What this system is, and deliberately is not
Core principle. TravelSpy does not handle messages. It handles insights.
The host owns the conversation, the content and the customer relationship. We ingest behavioural signals, derive insight, and return that insight. We never become the system of record for a conversation, we never render the host's commercial content, and we never sit in the delivery path of anything the traveller reads as coming from the brand.
PRIN-1Insight is a derived artefact, never raw payload passthrough. If a field cannot be justified as model input, it is not collected.Test: every stored field maps to a documented feature.
MustPRIN-2Every insight ships with a confidence value and its evidence. An unexplainable score is a defect, not a feature.
MustPRIN-3The host site must remain fully functional if TravelSpy is unavailable. All integration points degrade silently.
MustPRIN-4Consent state travels with the record and is enforced at the point of use, not only at the point of collection.
MustPRIN-5Prefer boring, observable infrastructure. Novelty belongs in the models, not the pipes.
ShouldSignal in, insight out, two delivery paths
↓ signals only (no message content) ↓
↓ validated event envelope ↓
↓ insight envelope ↓
↓ rendered for a human ↓
Who owns what, and what each side may assume
| Component | Owns | Must never |
|---|---|---|
| Collectorpublic/js, ~9 KB budget | Signal capture, batching, consent gate, offline buffer | Render UI, block the main thread, read form values |
| Ingest API | AuthN, schema validation, rate limiting, quarantine | Accept an unvalidated envelope into the bus |
| Identity resolution | Stitching, the traveller ID, consent propagation | Join on an identifier lacking a consent basis |
| Feature store | Feature definitions shared by training and serving | Let online and offline definitions diverge |
| Insight engine | Scores, confidence, evidence, model versioning | Emit a score without evidence or version |
| Insight API | Synchronous reads, scoped tokens | Return data outside the token's workspace scope |
| Insight stream | Live delivery, resume, backpressure | Hold state the client cannot rebuild from a snapshot |
| Embedded surfaceTwo variants, see §8 | Rendering insight for its declared audience, capturing structured intent | Store conversation, price outside the host's guardrails, navigate the top window |
Two envelopes. Everything else is internal.
One shape for tag and server-side ingestion. Rejected at the boundary if it fails schema validation; rejected payloads go to a quarantine topic, never to the bus.
{
"schema": "travelspy.signal.v1",
"workspace": "ws_live_9f21c480",
"occurred_at": "2026-04-02T09:14:22.418Z",
"idempotency_key": "01JQ8W3K7P2F5N9Z",
"subject": {
"session_id": "ses_7f21aa",
"device_id": "dev_c40b91",
"identifier": null // hashed, only when consented
},
"consent": { "analytics": true, "personalisation": true, "advertising": false },
"context": {
"url_path": "/japan/kansai", // path only, never query string
"referrer_class": "editorial",
"locale": "en-GB",
"device_class": "desktop",
"network": { "country": "GB", "type": "broadband", "vpn": false }
},
"signal": {
"name": "destination_viewed",
"attributes": {
"destination": "JP", "region": "Kansai",
"trip_type": "culture", "nights": 11,
"party": { "adults": 2, "children": 0 }
}
}
}
Identical shape whether it arrives by REST, by stream or by destination sync. The embedded surface renders this and nothing else.
{
"schema": "travelspy.insight.v1",
"traveller_id": "trv_8813af",
"computed_at": "2026-04-02T09:14:22.996Z",
"sequence": 4182, // monotonic per traveller, drives resume
"model_versions": { "affinity": "18.3", "spend": "11.0", "window": "9.4" },
"insights": [
{
"key": "destination_affinity",
"value": { "code": "JP", "region": "Kansai" },
"confidence": 0.91,
"evidence": [
{ "signal": "repeat_destination_views", "weight": 0.94, "observations": 11 },
{ "signal": "referrer_class", "weight": 0.81, "observations": 6 }
]
},
{
"key": "spend_band",
"value": { "band": 6, "range": [6400, 8200], "currency": "GBP" },
"confidence": 0.84,
"evidence": [ { "signal": "premium_option_dwell", "weight": 0.77, "observations": 2 } ]
},
{
"key": "booking_window",
"value": { "opens_in_days": 9, "closes_in_days": 21 },
"confidence": 0.72,
"evidence": [ { "signal": "date_picker_refinement", "weight": 0.62, "observations": 4 } ]
}
],
"suppressed": ["advertising"] // consent-derived, enforced at use
}
CTR-1Both envelopes MUST carry an explicit schema version. Consumers MUST reject unknown major versions rather than best-effort parse.
CTR-2Schema changes are additive within a major version. Removing or retyping a field requires a new major version and a documented dual-write window.
MustCTR-3Every signal MUST carry an idempotency_key. Ingest MUST deduplicate on it for at least 24 hours, because the collector retries.
CTR-4The insight envelope MUST NOT contain free text originating from a traveller. Values are enums, numbers, ranges and codes only.This is what keeps us out of the message-handling business.
MustCTR-5sequence SHOULD be monotonic per traveller so a reconnecting client can ask for everything after a known point.
The only code we ask the host to deploy
COL-1Loader MUST be async and MUST NOT block rendering. Total transferred budget 10 KB gzipped for the loader plus core.
MustCOL-2No network request of any kind before a consent decision is recorded. A dormant tag is the default state.Acceptance: fresh session with no CMP response produces zero requests to our origin.
MustCOL-3MUST capture only declared signals. No automatic DOM scraping, no keystroke capture, no form value capture, no query strings.
MustCOL-4Batch signals and flush on a timer, on batch size, and on visibilitychange to hidden. Final flush MUST use navigator.sendBeacon so it survives unload.
COL-5Buffer to memory when offline; retry with exponential backoff and jitter. Drop the buffer, never the host page, on sustained failure.
MustCOL-6Expose TravelSpy('forget') to erase locally and trigger server-side propagation.
COL-7SHOULD self-report tag health (version, coverage, error rate) so the portal install page reflects reality.
ShouldCOL-8Server-side ingestion parity for hosts that will not ship a browser tag.
LaterWhere signal becomes something worth acting on
INS-1Serving and training MUST read feature definitions from the same store. A feature computed two ways is a defect.
MustINS-2Every emitted insight carries confidence, evidence[] and the model_version that produced it.
INS-3Recompute is incremental and event-driven. A single new signal MUST NOT trigger a full profile rebuild.
MustINS-4Emit an insight only when it materially changes. Define per-key thresholds; suppress noise below them.Rationale: the stream is a change feed, not a metronome.
MustINS-5Drift monitoring with automatic rollback to last known-good version. Bias testing against protected attributes before any promotion.
MustINS-6Human overrides are captured and fed back as training signal, and are visible in the evidence trail.
ShouldINS-7Customer-supplied models registered alongside the TravelSpy library, scored through the same contract.
LaterHow live insight actually reaches a browser
| Option | Fit | Verdict |
|---|---|---|
| Server-sent eventstext/event-stream | One-way server→client, which is exactly our shape. Auto-reconnect, Last-Event-ID resume and retry: backoff are built into the browser. | Chosen |
| WebSocket | Full duplex we do not need, plus our own reconnect, heartbeat and resume logic. More proxy and CDN friction. | Rejected |
| Long polling | Works everywhere. Higher latency and request overhead. | Fallback |
Known constraint that must shape the design. Over HTTP/1.1, browsers cap concurrent connections at roughly six per browser + domain, shared across tabs. A traveller with several tabs open on the host site could exhaust that budget and starve the host's own requests.
Therefore: the stream MUST be served over HTTP/2 or later (where concurrent streams are negotiated, typically 100), and the client MUST hold one connection per browser, not one per tab or per frame.
RT-1Stream served as text/event-stream over HTTP/2+, with Cache-Control: no-cache and proxy buffering disabled.
RT-2Exactly one stream connection per browser. Fan out to tabs and frames via BroadcastChannel, with a leader election so a single tab owns the socket.Directly mitigates the six-connection cap.
RT-3Every event carries id: set to the insight sequence. On reconnect the server MUST honour Last-Event-ID and replay only what was missed.
RT-4Server sends a comment keep-alive at least every 20 seconds to stop intermediaries closing an idle connection, and sets retry: explicitly.
RT-5Replay buffer is bounded. If a client is too far behind, the server MUST send a snapshot event instead of a replay and the client MUST accept it as authoritative.
RT-6Close the stream when the document is hidden beyond a grace period; reopen on visible. Battery and connection budget are not ours to spend.
MustRT-7Coalesce rapid changes to the same insight key within a short window so the UI does not flicker.
ShouldRT-8Automatic downgrade to long polling when EventSource is unavailable or repeatedly fails.
: keep-alive
retry: 3000
event: insight.changed
id: 4182
data: {"traveller_id":"trv_8813af","key":"booking_window",
data: "value":{"opens_in_days":9},"confidence":0.72}
event: insight.snapshot
id: 4183
data: {"traveller_id":"trv_8813af","insights":[ ... ]}
event: consent.withdrawn
id: 4184
data: {"traveller_id":"trv_8813af","scopes":["personalisation"]}
Two embeds, two audiences, one stream
Decision. There are two embedded surfaces, not one. They consume the same insight stream and differ only in who is looking at them.
Conflating them was the original design error: a panel that tells someone to “present the full trip now, do not discount” is written for your revenue team, and must never be rendered to the person being described.
| Surface | Audience | Recommends? | Shows |
|---|---|---|---|
| Traveller surface/embed/traveller | The traveller, on the host's public site | Yes — a packaged trip | Plain-language confidence (“we think…”), an assembled itinerary, a way to correct us |
| Team surface/embed/team | Agents, advisors and commercial staff, behind authentication | Yes — a next best action | Numeric confidence, evidence, spend band, margin guidance |
Amendment to §3. The component table previously said the embedded surface must never render commercial content. That was too broad and contradicted the traveller surface, which exists precisely to present an assembled trip. The rule is narrower: we render offers built from the host's own contracted inventory, priced inside the host's guardrails. We never invent inventory, never price outside the floor, and never become the merchandising system of record.
EMB-1Subscribes to the insight stream for the current traveller and re-renders on change. Cold start MUST render from a snapshot before the first delta arrives.
MustEMB-2Holds no durable state. Everything it displays MUST be reconstructible from a snapshot after a hard reload.
MustEMB-3Captured input is constrained: selections, confirmations, corrections and numeric ranges. No free-text field that we persist.Acceptance: no request body from either frame contains an author-written string.
MustEMB-4Renders a visible confidence indication. Nobody is shown a guess presented as a fact.
MustEMB-5Degrades to a static last-known state if the stream drops, with a visible stale indicator. It MUST NOT show an error the host's customer has to interpret.
MustEMB-6Reports its content height to the parent so the host can size the frame. Iframes expose no size information by default, so this is an explicit postMessage until native responsive sizing is broadly available.
EMB-7Themeable via URL parameters only. No host CSS injection, no style leakage in either direction.
ShouldEMB-8Agent-facing variant, later. Superseded — promoted to a first-class surface by EMB-9 onward.
EMB-9The two surfaces ship from separate URLs with separate tokens. A workspace may enable either or both. One surface MUST NOT be reachable by changing a parameter on the other.
MustEMB-10The team surface token is issued only to an authenticated staff context and MUST be rejected if the embedding origin is a public host page.Acceptance: pasting a team embed onto the public site renders nothing.
MustEMB-11Commercial vocabulary — margin, discount, propensity, spend band, lead score — MUST NOT appear in the traveller surface. Enforce with a copy lint in CI against a banned-term list.
MustEMB-12Confidence is presented differently per audience: plain language on the traveller surface (“we think”, “fairly sure”), numeric plus evidence on the team surface.
MustEMB-13The traveller surface MAY present a packaged trip, assembled from host-contracted inventory and priced within host guardrails. It MUST NOT display a price the host's own systems would not honour.
MustEMB-14Both surfaces consume the identical insight envelope. Any divergence is presentation-only — there is no second model and no second truth.
ShouldEMB-15A correction made by the traveller (“actually it is three adults”) propagates to the team surface within one stream tick, so an agent is never working from a stale belief.
ShouldBoth directions are strictly typed and origin-pinned. The frame is the source of truth for insight; the host is the source of truth for page context.
| Direction | Type | Purpose |
|---|---|---|
| frame → host | advisor.ready | Frame booted, stream connected |
| frame → host | advisor.resize | Content height changed |
| frame → host | advisor.insight_changed | Host may mirror insight into its own UI |
| frame → host | advisor.intent | Traveller expressed structured intent; host decides what to do |
| host → frame | host.context | Current page, product or departure in view |
| host → frame | host.consent | Consent changed; frame must re-evaluate immediately |
// In the frame. Never post to "*".
parent.postMessage(
{ source: 'travelspy-advisor', type: 'advisor.intent',
payload: { key: 'destination_confirmed', value: 'JP' } },
HOST_ORIGIN // configured per workspace, verified server-side
);
// In the host.
window.addEventListener('message', function (e) {
if (e.origin !== 'https://advisor.travelspy.io') return; // 1. pin the origin
if (!e.data || e.data.source !== 'travelspy-advisor') return;
if (!ALLOWED_TYPES.has(e.data.type)) return; // 2. validate the shape
handle(e.data.type, e.data.payload);
});
Assume the host page is hostile and the frame is a target
SEC-1The widget MUST be served from a dedicated origin, separate from the marketing site and the portal, so a compromise cannot pivot into a session.
MustSEC-2targetOrigin MUST always be an exact origin. Using * is prohibited, because a host can navigate the frame and intercept the message.
SEC-3Every receiver MUST verify event.origin and validate message shape. Any window in the frame hierarchy can post to any other.
SEC-4Embedding is restricted by a per-workspace origin allowlist, enforced with Content-Security-Policy: frame-ancestors. An unregistered origin gets no frame and no token.
SEC-5Never combine allow-scripts with allow-same-origin on a same-origin embed — the framed document can then remove its own sandbox, making the attribute worthless.
SEC-6Stream and API tokens are short-lived, workspace-scoped and traveller-scoped. A leaked embed token MUST NOT read another traveller.
MustSEC-7The frame MUST NOT request top navigation. Browsers already gate this behind sticky activation; we do not rely on that as our only control.
MustSEC-8Ship a strict CSP on the widget document: no inline script, no eval, connect-src limited to our API and stream origins.
SEC-9Evaluate credentialless iframes for hosts running COEP. Chromium-only today, so not a baseline requirement.
The constraint that shapes the architecture
PRV-1No profile is created and no insight computed until a consent decision is recorded. The host CMP is authoritative.
MustPRV-2Consent state is carried on the record and enforced at the point of use. A traveller with analytics but not advertising consent is modelled but never exported to an ad platform.
MustPRV-3Withdrawal propagates to every connected destination and returns a completion receipt. Target: under 60 seconds internally, under 24 hours end to end.
MustPRV-4Retention is configurable per market and enforced by deletion, not archival.
MustPRV-5No special category data, no financial account data, no precise GPS, no private message content, at any layer.
MustPRV-6Every profile read is written to an audit log that survives the profile itself.
ShouldNumbers to build against and to test
| Measure | Target | Breach |
|---|---|---|
| Collector transfer size | ≤ 10 KB gzipped | Blocks release |
| Collector main-thread time | ≤ 15 ms per page | Blocks release |
| Signal → insight latency | p95 ≤ 900 ms | Page on-call |
| Insight API read | p95 ≤ 200 ms at edge | Page on-call |
| Stream delivery | p95 ≤ 400 ms from compute | Page on-call |
| Widget first meaningful render | ≤ 1.2 s on 4G | Blocks release |
| Stream availability | 99.9% monthly | SLA credit |
| Ingest durability | No acknowledged signal lost | Incident review |
What happens when each part breaks
| Failure | Required behaviour | Traveller sees |
|---|---|---|
| CDN down | Tag never loads; host page unaffected | Nothing |
| Ingest unavailable | Collector buffers, backs off, drops buffer rather than degrading the page | Nothing |
| Engine degraded | Serve last known insight with a staleness marker; suppress low-confidence keys | Slightly older guidance |
| Stream drops | Browser auto-reconnects and resumes from Last-Event-ID; snapshot if too far behind | Brief stale indicator |
| Widget origin blocked | Host renders its own fallback; no error surfaced by us | Nothing |
| Consent withdrawn mid-session | Frame clears immediately, stream closes, deletion propagates | Widget disappears cleanly |
| Model drift detected | Automatic rollback to last known-good version, alert raised | Nothing |
Each phase ships something usable and has an explicit exit test
Exit: stakeholders agree the insight-not-messages boundary.
Exit: a real host site produces validated signals for seven days with zero page-performance regressions.
Exit: p95 read latency under 200 ms and every score renders its evidence.
retry: (RT-1, RT-4).Last-Event-ID plus bounded replay and snapshot fallback (RT-3, RT-5).BroadcastChannel leader election (RT-2).Exit: six tabs open on the host site consume one connection, and a forced network drop resumes with no duplicate or missing sequence.
frame-ancestors and scoped embed tokens (SEC-4, SEC-6).Exit: penetration test passes with no finding above low, a team embed pasted onto a public page renders nothing, and the copy lint blocks a build containing the word “margin” in traveller strings.
Exit: a measured conversion lift published from a holdout, reconciled against the customer's own reporting.
Needed before Phase 2 closes
| Question | Why it matters | Needed by |
|---|---|---|
| Who owns the traveller ID?Our identifier, or the host's? | Determines whether profiles are portable between hosts and how erasure propagates | Phase 1 |
| Per-key change thresholds | Sets how chatty the stream is and therefore the entire cost model | Phase 2 |
| Banned-term list for traveller copy | EMB-11 needs a concrete list before the copy lint can be built | Phase 4 |
| Structured intent taxonomy | The constraint that keeps us out of free text; must be expressive enough to be useful | Phase 4 |
| Multi-host traveller identity | Commercially attractive, privately fraught. Needs a legal position, not just a technical one | Phase 5 |
| Long-polling fallback | Only worth building if we see real EventSource failures in the field | Phase 5 |