Skip to content

Identity Webhook Dispatch

How the two client-facing identification webhooks — account_identified and visitor_identified — get from "a provider recognised someone" to "the client's endpoint received a POST".

The short version: producers only write rows, and one job publishes.

Why the split exists

Five different code paths can identify a visitor's company or person, in four packages. Publishing from each of them would mean five send points for two events, and every future change to webhook behaviour — a new guard, a payload field, a retry rule — would have to be reasoned about five times.

There is also a delivery problem. Cloud Run throttles CPU once a response is written, and Vector's ingest returns 204 and defers to a FastAPI BackgroundTask. A detached asyncio.create_task holding a retry loop with sleep(1/2/4) inside that would deliver intermittently in production and reliably on a laptop — the worst possible failure shape.

So producers write, and identity-sweep reads and publishes. Selection is producer-agnostic: it reads two marker columns on public.visitors, so a company Vector delivers five hours after the visit is dispatched exactly like one found inline.

The producers

Producer Package Timing Writes
Impression enrichment ixchat during the widget impression accounts, visitors
Chat enrichment node ixchat first message of a conversation accounts, visitors
Snitcher resolved by the sweep itself ~40s after the visit, via a queue accounts, visitors
Vector ixvector hours after the visit, inbound webhook accounts, visitors, visitor_identifications
RB2B waterfall ixchat during an engaged session visitors, visitor_identifications

Snitcher is the only one needing a queue, because it is the only provider we pull: its Radar sessions ingest asynchronously, so the impression cannot ask "which company is this?" and get an answer. Everything else either pushes to us or resolves in-request.

End to end

sequenceDiagram autonumber participant V as Visitor participant W as Widget / Chat API participant DB as Supabase participant X as Vector (inbound webhook) participant J as identity-sweep job participant S as Snitcher API participant C as Client endpoint V->>W: page load W->>DB: write visitors / accounts (IP enrichment) W->>DB: queue snitcher_pending_lookups Note over W,DB: no webhook is sent here rect rgba(128,128,128,0.08) Note over J: every 5 min — phase 1, resolve J->>DB: claim_snitcher_lookups (FOR UPDATE SKIP LOCKED) J->>S: company/find alt company found S-->>J: company J->>DB: persist_identified_account (store only) else still processing (404) S-->>J: not ready J->>DB: reschedule +10 min, attempts += 1 end end rect rgba(128,128,128,0.08) Note over J: same run — phase 2, dispatch J->>DB: select candidates where marker IS NULL J->>DB: UPDATE ... WHERE marker IS NULL RETURNING * J->>C: POST account_identified / visitor_identified C-->>J: 200 end Note over X,DB: hours later, independently X->>DB: contact.visited → visitors + visitor_identifications Note over J: next tick picks it up the same way

Phase 2 runs whatever phase 1 did. A Snitcher outage, or missing credentials, must not stop the client feed; exit codes are judged per phase so the Cloud Run failure alert still fires for a broken resolve phase.

Running both in one job rather than two is what lets a company Snitcher resolves at 10:00 go out at 10:00 instead of waiting for the next tick.

What one dispatch pass decides

flowchart TD A[Candidates: marker IS NULL<br/>and updated_at within 24h] --> B{Kind?} B -->|account| C[account_id IS NOT NULL] B -->|visitor| D[email, name or linkedin_url<br/>IS NOT NULL] C --> E[Group by site_domain] D --> E E --> F{Webhook enabled<br/>for this site?} F -->|no| G[Claim and stamp<br/>send nothing] F -->|yes| H{only_with_conversation?<br/>accounts only} H -->|on, no conversation yet| I[Leave marker NULL<br/>still eligible] H -->|otherwise| J[Claim: UPDATE ... WHERE marker IS NULL] J --> K{Won the claim?} K -->|no| L[Another run owns it] K -->|yes| M{HTTPS URL?<br/>named person?<br/>not anonymized?} M -->|fails a guard| N[Stamped, not sent] M -->|passes| O[POST, HMAC-signed]

Two decisions in that graph are worth spelling out, because both encode a trap.

The kind-specific predicate is not an optimisation. A visitor with no company and no name must never be selected, because selecting leads to claiming, claiming stamps the marker, and the company that arrives five minutes later would then never be dispatched. The row has to stay invisible until it has something to say.

only_with_conversation is applied before the claim. A visitor who has not chatted yet may chat within the hour, so leaving the marker NULL keeps them eligible. expire_stale retires them once they fall out of the 24-hour window, which is also what stops the partial index growing without bound.

The marker columns

public.visitors.account_webhook_sent_at and public.visitors.visitor_webhook_sent_at, both nullable timestamptz, each with a partial index on (updated_at) WHERE ... IS NULL so the sweep reads only the undispatched tail.

They live on visitors rather than accounts because both events are per-visit, not per-company: account_identified is already keyed on the visitor (sha256(site_domain:person_id or company_domain)) and carries that visitor's person fields. Keeping both on one row also means the sweep scans one table.

The semantics are "the dispatch pass considered this row", not "we sent it". Rows belonging to a client with the webhook switched off are stamped too. That is deliberate: it means turning a webhook on starts the feed from that moment instead of replaying every visitor since the account was created.

The claim is a single conditional statement, which is the whole concurrency story:

UPDATE public.visitors
   SET account_webhook_sent_at = now()
 WHERE id = ANY($1) AND account_webhook_sent_at IS NULL
 RETURNING *;

Two overlapping runs serialize on the row lock; the second re-evaluates the predicate under READ COMMITTED and gets nothing back. Only returned rows are published. PostgREST sends Prefer: return=representation, so response.data really is the set of rows this run won.

A Redis claim (claim_once / confirm_claim) still wraps each publish as a second line of defence, keyed on the event's stable id. It cannot replace the column — Redis can answer "has this been delivered?" but not "which rows still need sending?".

Backfill without a table rewrite

The migration adds each column DEFAULT NOW() and then immediately drops the default:

ALTER TABLE public.visitors
    ADD COLUMN IF NOT EXISTS account_webhook_sent_at TIMESTAMPTZ DEFAULT NOW();
ALTER TABLE public.visitors
    ALTER COLUMN account_webhook_sent_at DROP DEFAULT;

On PostgreSQL 11+ ADD COLUMN ... DEFAULT is catalog-only — existing rows read the value from attmissingval — so every row that existed before the migration reads as stamped without the table being rewritten and without trg_visitor_first_touch firing once per row. Dropping the default then leaves new rows NULL. That is the entire backfill, and it also means the partial indexes are empty at creation and build instantly.

Guards on the payload

  • Named people only. A visitor_identifications row can be status='matched' on company_domain alone; that is an account identification. Publishing it as a person would ship an empty card and burn the single delivery that visitor ever gets. At least one of email, name, linkedin_url is required.
  • Canonical row, never raw evidence. Payloads are built from the visitors row, so an email a provider returned but persist_personal_email never promoted cannot leak.
  • HTTPS, checked in the backend. The schema pattern only binds the backoffice write path; a URL written by rose-config, a migration or direct SQL is unconstrained, so the publisher re-checks with .strip().lower().startswith("https://").
  • Anonymized visitors are skipped.
  • Signing secrets come from the encrypted store, and failing to read one skips the delivery. Both feeds sign with integrations.site_integration_secrets (<feed>_secret), never from the config row get_widget_config serves to browsers (IX-4501). The signature is optional, so "no secret stored" and "we could not read the secret" must not collapse into the same answer: a store failure skips the send rather than silently downgrading a signed webhook to an unsigned one. The accounts feed still falls back to the legacy plaintext identified_accounts_webhook.secret until scripts/backfill/ix4501_scrub_config_secrets.sql has been run; the visitors feed never had one.

Operating it

# One local run against whatever SUPABASE_URL the env resolves to.
cd backend
IX_ENVIRONMENT=<env> DISPATCH_LIMIT=5 poetry run python apps/jobs/identity_sweep/run_identity_sweep.py

# Deploy (production-only by policy).
just job deploy identity-sweep

SWEEP_LIMIT bounds phase 1 (default 200 Snitcher lookups), DISPATCH_LIMIT bounds phase 2 per event kind (default 500).

The job is production-only for the same reason posthog-batch is: the queue and the marker columns both live in the shared Supabase project, so a staging sweeper would claim production rows — spending Snitcher credits twice and POSTing real clients' webhooks a second time.

When a client says nothing arrives

  1. Is the webhook enabled, with an https:// URL, in integrations.identified_*_webhook?
  2. Are their rows even candidates? account_id IS NULL and no name means never selected.
  3. only_with_conversation on, and the visitor never chatted? Then it is working as configured.
  4. Check the job logs for the delivery line. Each feature labels itself (Identified-account webhook / Identified-visitor webhook), so a 4xx from their endpoint is attributable.

Person-level events are much rarer than account-level ones by nature — naming an individual is far harder than recognising a company, so only a fraction of identified visits produce one. That asymmetry is expected, not a bug.