Supabase Workflow¶
End-to-end procedures for Rose developers and coding agents. Read the relevant
section before acting; agent-specific permissions remain in supabase/AGENTS.md.
Production boundary: test, staging, and production environment labels share Supabase rows. Only local databases and explicitly provisioned preview branches isolate writes. Verify the database target before applying migrations or seeding.
Pick your environment¶
| Target | Command to start | When |
|---|---|---|
| Local Supabase (Docker) | cd supabase && just dev |
Frontend dev, isolated tests, no prod credentials needed. |
| Preview branch (remote) | ./bootstrap.py --branch |
Migrations / RLS / RPC / MV work. Real Postgres, isolated DB, auto-deleted on PR close. |
| Staging | Already wired via frontend/.env.staging |
Final integration testing before merge. |
You never need to develop against production directly.
Git ↔ Supabase branch mapping¶
| Git branch | Supabase branch | Project ref | Role |
|---|---|---|---|
main |
main (default) |
drtzxyuvppalvgczwhne |
Production. Git-triggered migration deployment depends on the Supabase GitHub integration. |
develop |
develop (persistent) |
ziptmcxknuouujrjzanz |
Test database. Has its own migration history; persistent branches are never reaped. Never delete and recreate it: keep its project ref and rebuild it in place (see IX-5331). No running service connects to it; the test backend and R4A test services use production. |
feature/* |
per-PR preview | per branch | Isolated scratch DB, deleted when the PR closes. |
Automatic migration deployment on pushes to main or develop is configured
outside this repository, through the Supabase GitHub integration. Verify its branch
mapping, deployment status, and the target database's migration history before
relying on it. Its Working directory must be . (the folder that contains
supabase/): set to supabase, it finds no migration files, logs
All migrations are up to date. and applies nothing from git (IX-5333). A run's
workdir is in GET /v1/projects/<ref>/actions/<id>.
Two gates stop a broken migration. PR Checks - Supabase applies every file in
supabase/migrations/ to a fresh local database and runs the SQL tests
(supabase/scripts/test_sql.sh) on every develop PR that touches supabase/; the
release workflow reruns it. The main ruleset requires the integration's
Supabase Preview check, so a release PR cannot merge while the latest migration run
on develop failed (repository admins can bypass it for an emergency). Neither
applies migrations to production. A successful GitHub release alone is not evidence
that the schema was deployed.
A develop merge never writes production: .github/workflows/supabase-branch-cleanup.yml
only deletes the PR's preview. Production receives migrations with the main release,
or through a deliberate operator push. IX-5148 adds a
gated release step in front of it.
supabase db push records successfully applied versions in the target database's
supabase_migrations.schema_migrations; subsequent pushes skip those versions.
SQL run directly through the SQL editor or psql does not record that history.
See operator production pushes for the supported
direct-push procedure.
Supabase initially provisions a preview from production's recorded migration SQL.
Our scripts/supabase_branch.py ensure helper then rebuilds fresh previews from
the checkout's supabase/migrations/ files, replacing that initial schema and
history. Production-only changes absent from those files do not survive this
rebuild. Keep applied migration files unchanged and commit them; use new migrations
for corrections.
Standard workflows¶
A. Pure frontend or backend change (no schema work)¶
# Start once
cd supabase && just dev # Boots local Postgres + Auth + Studio at 127.0.0.1:54323
just seed-local # Synthetic data (or seed-local-copy for real)
# Develop
cd ../frontend && just dev
cd ../backend
just dev test
Login as admin@admin.com / admin. See Supabase Seeding for the data set you'll see.
B. Schema change (migration, RLS, RPC, materialized view)¶
# 1. Spin up an isolated branch DB
./bootstrap.py --branch
# 2. Write the migration
date +%Y%m%d%H%M%S # generate timestamp
$EDITOR supabase/migrations/20260520143000_my_change.sql
# 3. Apply it to the branch DB
cd supabase
set -a; . ../backend/.env.local; set +a
DB_URL="postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/${POSTGRES_DB}"
supabase db push --db-url "$DB_URL" --include-all --yes
# OR shortcut that also re-seeds + refreshes MVs:
just seed-branch
# 4. Regenerate types from this preview's project ref
# Follow "Generated types after migrations" below, then commit the generated file.
# 5. Type-check + test
cd ../backend && just mypy <changed-files>
cd ../frontend && just lint
After merging into develop, verify the test database's migration deployment and
history through the configured Supabase integration. The PR's preview branch is
deleted; production receives the migration only with the main release (see
Git ↔ Supabase branch mapping). Do not infer database
deployment from the Git merge or application release alone.
After a main release, confirm production applied it: the Supabase integration's run log
for branch main (GET /v1/projects/drtzxyuvppalvgczwhne/actions, then
/actions/<id>/logs) shows Applying migration <version> or
All migrations are up to date., and production's supabase_migrations.schema_migrations
records the version.
C. Reproducing a real client bug¶
The synthetic data won't match real production shapes. Use the *-copy seed path:
# On a preview branch (recommended — isolated, won't pollute local DB)
./bootstrap.py --branch
cd supabase && just seed-branch-copy --clear
# OR against local Supabase
cd supabase && just seed-local-copy --clear
Requires backend/.env.production (downloaded by just download-env all). The copy is ~60 s and contains real visitor IPs / emails / chat content — treat the resulting DB as sensitive.
D. Editing a materialized view or RPC¶
Before replacing an existing object, inspect its live definition:
-- Inspect the CURRENT shape on develop (the source of truth):
SELECT pg_get_viewdef('public.mv_client_stats_30d'::regclass);
SELECT pg_get_functiondef('public.get_account_stats_with_intent'::regproc);
Then write a migration that drops + recreates (or uses CREATE OR REPLACE only if the signature is identical — Postgres rejects return-type changes via CREATE OR REPLACE alone, see 20260318103855 for the fix pattern).
MVs are populated by triggers / cron in production. On a fresh branch, the synthetic seed refreshes them in dependency order (mv_widget_visitors → mv_conversation_visitors → mv_client_stats_30d). If you add a new MV, append its REFRESH to supabase/justfile in the seed-local and seed-branch targets.
E. Operator production push¶
A deliberate human supabase db push from local migration files is compatible
with later automated pushes: the same versions are already recorded in production
and are skipped there. This is an operator exception to the usual preview-first
workflow; it does not authorize an agent to apply production migrations.
- Verify that the linked project is production (
drtzxyuvppalvgczwhne) and that the migration works with the application currently running there. Avoid overlapping production migration deployments. - From the repository root, inspect
supabase migration list --linkedandsupabase db push --linked --dry-run. The pending list must contain only the intended changes. Reconcile missing local files before proceeding; do not rewrite production history to bypass a mismatch. - The operator runs
supabase db push --linked, then checkssupabase migration list --linkedand verifies the affected database objects. - Commit the exact files, preserving their timestamp versions and SQL, and merge
them into
developandmainbefore subsequent automated deployments. Verify they apply successfully to test and fresh previews: a production push does not update those databases. Include generated types when needed. - Make any correction in a new migration. Editing or renaming an applied file makes production history diverge from what test and previews execute.
For versions already in production but missing locally, follow externally applied migration history.
Don'ts¶
Don't make untracked production schema changes
- Use the preview workflow by default. Deliberate human production pushes must follow operator production pushes; agents remain bound by
supabase/AGENTS.md. - Do not substitute SQL editor or psql execution for
supabase db push: they do not record migration history. - Never
supabase migration repairagainst prod from your laptop. - Never edit an existing migration without permission, including changes intended only to fix fresh replays. Prefer a new migration. Edits to a migration production has run are refused in CI (see Baseline and archived migrations).
Don't seed against the wrong target
The 5 seed targets are easy to confuse. Quick reference:
| Command | Hits |
|---|---|
just seed-local |
Local Supabase :54322 |
just seed-local-fresh |
Local Supabase :54322 (--clear first) |
just seed-local-copy |
Local Supabase :54322 |
just seed-branch |
Whatever's in backend/.env.local (your preview branch) |
just seed-branch-copy |
Whatever's in backend/.env.local (your preview branch) |
Debugging¶
"Invalid login credentials" on the backoffice¶
seed.sql didn't run on this DB. Either you skipped just seed / just seed-branch, or the branch is stuck in CREATING_PROJECT / MIGRATIONS_FAILED. See Common errors.
Magic link emails are rate-limited or not delivered¶
For hosted Supabase Auth, Rose uses Cloudflare Email SMTP, not Supabase's
built-in email service. Verify Supabase Dashboard → Authentication →
SMTP has custom SMTP enabled with the Cloudflare credentials, and
Authentication → Rate Limits has Emails sent set to 30/hour.
For local Docker Supabase, emails go to Inbucket at http://localhost:54324. Local Inbucket delivery does not prove that hosted Cloudflare SMTP is configured correctly.
"Access Restricted: admin@admin.com does not have access"¶
Browser holds a stale JWT signed by a previous (deleted) branch's secret. Clear site data for the backoffice origin and log in again.
Backoffice pages show zero data¶
- Conversations / Visitors / Accounts empty → check the
environmentfilter (defaultproduction) and workspace selector. Synthetic data is on the 4 fake domains (acme.com,contoso.com,fabrikam.com,demo.local). - Home dashboard tiles show 0 → materialized views aren't refreshed. Re-run
just seed-branch(orREFRESH MATERIALIZED VIEWmanually via Studio in the dependency order documented in seeding).
RLS denying queries you expect to work¶
Most tables gate on has_domain_access(site_domain). Verify:
Admins bypass via is_backoffice_admin(). Non-admins need a row in backoffice_user_domains. The synthetic seed grants user@user.com access to all 4 fake domains.
Cheat sheet¶
# Create / refresh isolated branch DB
./bootstrap.py --branch
# Seed (all idempotent)
cd supabase
just seed-local # local synthetic
just seed-local-fresh # local synthetic + --clear
just seed-local-copy # local prod copy
just seed-branch # branch synthetic (also pushes missing migrations, refreshes MVs)
just seed-branch --clear # branch synthetic + --clear
just seed-branch-copy # branch prod copy
# Branch lifecycle CLI
python3.12 scripts/supabase_branch.py ensure
python3.12 scripts/supabase_branch.py delete <git-branch>
python3.12 scripts/supabase_branch.py reap --dry-run
# Inspect current branch DB
set -a; . backend/.env.local; set +a
DB_URL="postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/${POSTGRES_DB}"
psql "$DB_URL"
# After any migration, follow "Generated types after migrations" below.
See also¶
- Supabase Preview Branches — branch lifecycle + troubleshooting reference.
- Supabase Seeding — seed data layout + how to add a new table.
- Supabase Setup — auth flow, RLS architecture,
before_user_createdhook. - Development Workflow — overall git + PR + merge process.
supabase/AGENTS.md— essential agent safeguards and task links into these shared procedures.
Inspect existing objects¶
- When recreating an existing object (materialized view, function, table), the live remote schema is the only source of truth — query it via the Supabase MCP before drafting the new definition. Migration files lie: an object can be reshaped by
ALTERmigrations that don't mention the original name, by hotfixes applied directly, by migrations not yet merged, or by ones that drop+recreate it across several files. Rolling back to a stale shape silently breaks dependent functions/views (e.g. dropping a column the function still references). MCP project:drtzxyuvppalvgczwhne. - Inspect applied migration history:
mcp__supabase__list_migrations. - Function source:
SELECT pg_get_functiondef(oid) FROM pg_proc WHERE proname = '<name>'; - Table/view/MV columns:
SELECT attname, format_type(atttypid, atttypmod) FROM pg_attribute WHERE attrelid = '<schema.name>'::regclass AND attnum > 0 AND NOT attisdropped ORDER BY attnum; - Materialized view body:
SELECT pg_get_viewdef('<schema.name>'::regclass); - Trigger / index / constraint:
pg_trigger,pg_indexes,pg_constraint. - No MCP access? Block — don't reconstruct from migration files. Authenticate first (
mcp__supabase__authenticate) or ask the operator to paste the live definition. A name grep oversupabase/migrationsis an unreliable last resort only: it missesALTERreshapes, direct prod hotfixes, and unmerged migrations, and the file named after the object is usually its first, stalest definition (this rolled back a current experiments-table-sourced RPC to a config-sourced one whose source row had been deleted).
Use migration files as templates after you know the current shape from the live DB — never as the authority on it.
Don't hand-retype the body either: hand-reproduction strips historical comments and invites transcription errors. Verify the latest migration file matches live (md5(pg_get_functiondef(oid)) vs the file's function block), then assemble the new migration as targeted string replacements on that verified text — the diff against source is then exactly your intended change.
Migration authoring and replay¶
Before editing SQL, inspect existing objects above and the
security rules. Write migration files locally;
apply/test only on an isolated local or preview database. Run the
supabase-security skill on every changed migration, fix P1/P2 findings, and
record any remaining P3/info findings in the review summary or authorized PR.
-
Do not delete production data in migrations. Avoid
DELETE, destructiveUPDATE,TRUNCATE,DROP TABLE, andDROP COLUMNagainst tables that can contain customer or analytics data. If data removal is genuinely required, put the destructive SQL in a separate operator-run script or PR note for a human to apply deliberately, not in the automatic migration path. -
Back up rows before any unavoidable destructive SQL. If a migration must move, dedupe, or remove rows to satisfy a constraint, first copy the affected rows into a durable backup table in the same migration, then operate from that backup. Use a specific table name that includes the migration/version and source table, for example:
The backup predicate must match the destructive predicate exactly, and the migration/PR must state how to restore from the backup. Prefer reversibleCREATE SCHEMA IF NOT EXISTS migration_backups; CREATE TABLE IF NOT EXISTS migration_backups.m20260528_visitors_before_dedupe AS SELECT v.*, NOW() AS backed_up_at FROM public.visitors v WHERE <exact destructive predicate>;UPDATE ... FROMrepairs over deleting rows. -
Migrations must be idempotent. Re-running the same migration on a DB where it already applied must succeed without errors. Required patterns:
- Tables:
CREATE TABLE IF NOT EXISTS …(orDROP TABLE IF EXISTSfirst when shape changed — only when wiping data is acceptable). - Functions:
CREATE OR REPLACE FUNCTION …. IfRETURNS TABLEsignature changes, prefix withDROP FUNCTION IF EXISTS <name>(<args>);. - Indexes:
CREATE INDEX IF NOT EXISTS …. - Materialized views:
DROP MATERIALIZED VIEW IF EXISTS … CASCADE; CREATE MATERIALIZED VIEW …. - Triggers:
DROP TRIGGER IF EXISTS … ON …; CREATE TRIGGER …. - Policies:
DROP POLICY IF EXISTS … ON …; CREATE POLICY …. - Cron jobs: wrap in
DO $$ BEGIN IF EXISTS (SELECT 1 FROM cron.job WHERE jobname='…') THEN PERFORM cron.unschedule('…'); END IF; PERFORM cron.schedule(…); END $$;. - Constraints / columns: guard with
DO+information_schemacheck orALTER TABLE … ADD COLUMN IF NOT EXISTS …. -
Idempotency is also what makes branch DBs + nightly seeding tractable; non-idempotent migrations break
just seed-branchreruns. -
A migration that uses an extension's functions must enable the extension itself —
create extension if not exists <ext> with schema extensions;(orpublicwhen a function'ssearch_pathexcludesextensions), before first use. Fresh DBs — preview branches, CI,supabase db reset— replay every migration on an empty Postgres, so an extension only enabled out-of-band on prod (a migration just assuming it exists) aborts mid-migration withfunction … does not existand fails the whole seed. A later-timestamped migration can't fix it:language sqlbodies validate atCREATE FUNCTION, so thecreate extensionmust sit in an earlier-named migration. (IX-3551:pg_trgm/similarity()in…4244_add_resolve_brand_functionbroke every factorybootstrap.py --branchrun until…4243_enable_pg_trgmwas added.) -
A migration that
CREATE OR REPLACEs an existing object must sort AFTER every committed migration touching that object. Fresh DBs (preview branches, CI,db reset) replay in timestamp order — a fix file named earlier than an already-committed redefinition of the same function is applied, then silently overwritten by it, so prod (incremental) and preview (full replay) end up with different definitions and preview "validation" tests the reverted code. Before finalizing, grepsupabase/migrationsfor other files defining the same object and confirm yours has the latest timestamp (date +%Y%m%d%H%M%S); re-check after rebases. (IX-4055: fix stamped 08:31 was overwritten on replay by the pre-existing 09:21 perf migration redefining the same function.) -
Dropping or replacing a named constraint/index? Grep earlier migrations for that name first. Fresh-DB replay aborts when their
ON CONFLICT (cols)can no longer name it, or their unique index no longer holds against the data your change permits — invisible on prod, which never re-runs them. Guard the historical file; worked example20260729171531_add_website_rescan_source_control_plane.sql. -
A migration that reads a materialized view breaks every fresh DB. Matviews replay
WITH NO DATAon preview branches / CI /db reset, so a baseline capture (CREATE TABLE … AS SELECT * FROM mv_*) errors and aborts the whole seed — guard onrelispopulated. Worked example: section 2 of20260828165954_ix4498_incremental_mv_widget_visitors.sql. -
Statements that depend on an out-of-band role or superuser privilege can fail fresh replay. Supabase's initial provisioning replays production's stored
supabase_migrations.schema_migrations.statements; editing a git file does not change those recorded statements. A bareGRANT … TO n8n_service, for example, can fail there if the role exists only in production (IX-5081). Our preview helper subsequently resets the application schemas and replays the git files as the branch's Postgres role, even if initial provisioning reportedMIGRATIONS_FAILED. Its final schema therefore follows Git, not the stored production SQL. Guard role-specific statements from the start withDO $$ BEGIN IF EXISTS (SELECT 1 FROM pg_roles WHERE rolname='…') THEN … END IF; END $$;. Keep applied files unchanged; any exceptional historical replay edit still requires permission and does not repair production's recorded SQL.
Generated types after migrations¶
Generate frontend/shared/src/types/database.generated.ts from the database where
the migration was applied, then commit it alongside the migration. Missing schema
types can make Supabase client calls resolve to never.
Production is the default. From frontend/, use the usual command or select
an isolated database containing your unapplied migration. For a preview, obtain
that preview's project ref from the Supabase dashboard:
# Usual workflow: production schema
just gen-types
# Docker-local database
just gen-types local
# Hosted preview database (replace the placeholder)
just gen-types '<preview-project-ref>'
The default project is drtzxyuvppalvgczwhne; .env.local does not override the
selected target. The recipe generates and formats into a temporary directory;
failed or empty output leaves the existing
types intact. Only successful, nonempty formatted output replaces the checked-in file.
- Stale branch: a database ahead of your checkout can introduce unrelated schema
drift. If the diff dwarfs your migration, update from
developand regenerate against the intended target. If generation must be deferred, record that explicitly. - Preview behind production: a preview branch replays migrations only, so objects that exist in production out of band are absent and regenerating from it deletes their types. Keep the checked-in file and copy in only your migration's new entries from the preview output (IX-5063).
- Unapplied migration: generation cannot see a change absent from the target. Prefer applying it to an isolated local/preview database first. For a small exact addition such as an enum member, a temporary hand-patch must match the migration and be reconciled with the next regeneration.
- After rebasing a hand-patch: check for duplicate schema keys even if Git reports no conflict. Reconcile with develop's generated version while preserving your migration's additions; regenerate from the correct database when available.
Alias views over renamed concepts¶
When a product term replaces a physical name that live code depends on, do not rename
the table. Add the new name as a view and move readers to it one at a time
(public.workspaces / public.workspace_roots over public.sites / public.site_roots,
20260917181817_ix5063_workspace_alias_views.sql). The expand step is then purely
additive, and a later physical rename is invisible to application code.
WITH (security_invoker = true), or the view bypasses the base table's RLS.- One base table, no aggregates or
DISTINCT: the view stays auto-updatable, so inserts, updates and deletes reach the base table and fire its triggers. REVOKE ALL … FROM PUBLIC, anon, authenticated, then grant what the base table grants.- A view's column list is fixed at creation. A migration that adds a column to the
base table must
CREATE OR REPLACE VIEWthe alias, appending the column last.supabase/tests/ix5063_workspace_alias_views.test.sqlfails on drift. - A view cannot share its table's name, so a renamed column on a table whose own name
is fine (
knowledge.ingestion_operations.site_id) is aliased in the application model. - Generated types mark every view column nullable. Restore the base table's NOT NULLs at
the call site with
.overrideTypes<Row[], { merge: false }>(). - PostgREST embeds resolve through the views using the base tables' foreign-key names
(
workspaces?select=…,clients!sites_client_id_fkey(…),workspace_roots(…)).
Baseline and archived migrations¶
Branches replay production's recorded schema_migrations.statements for every version
production has, not the git files. IX-5159 collapsed everything up to 20260928172748
into supabase/migrations/20260928180000_baseline.sql, a dump of production's schema
(roles, schema, default privileges, cron jobs) plus the reference rows the old
migrations seeded (config definitions, the testfeatures fixture sites). The superseded files live in
supabase/migrations_archive/ as history only: neither the CLI nor any test reads them. Plan and evidence: docs/plans/2026-09-29-ix-5159-baseline-migration.md.
- Never edit or delete a migration production has run. Branches would keep replaying
the recorded SQL while git says otherwise.
supabase/scripts/check_applied_migrations.sh(injust lint-migrationsand PR checks) refuses such diffs againstorigin/main. - Never stub a missing version. A stub pushed anywhere becomes the SQL every future
branch replays. When production has a version with no local file (pushed from a
worktree that has not merged yet),
just pushborrows its recorded SQL for that push and deletes it afterwards. The worktree that pushed it still lands the file, unchanged. - An existing local volume keeps its old schema. When
just devprintsStarting database from backup, it reused the Docker volume and applied no pending migrations, the baseline included. Check withselect max(version) from supabase_migrations.schema_migrations;. To try one new migration without losing local data, apply that file alone:psql "postgresql://postgres:postgres@127.0.0.1:54322/postgres" -v ON_ERROR_STOP=1 -f <file>.supabase db resetrebuilds from the baseline but deletes local data.
Compare a database's schema with production¶
Run supabase/scripts/schema_fingerprint.sql on both databases and diff the sorted
kind|name|md5 lines. Branch or local:
psql "$DB_URL" -At -F '|' -f supabase/scripts/schema_fingerprint.sql. Production
(read-only): POST the file's SQL to the Management API
/v1/projects/drtzxyuvppalvgczwhne/database/query with "read_only": true.
Expected differences: comments inside function bodies, platform-managed extension
versions, and Supabase's own event triggers. Anything else is drift.
Recover externally applied migration history¶
When production records a version with no file on any branch, recover the real SQL
rather than creating a stub: read its name and statements from
supabase_migrations.schema_migrations with mcp__supabase__execute_sql (project
drtzxyuvppalvgczwhne), save it as <version>_<name>.sql, and land it on develop.
If MCP is unavailable, a read-only metadata dump works:
supabase db dump --schema supabase_migrations --data-only --file .context/supabase-debug/supabase_migrations_dump.sql.
Do not run production migration repair to hide an orphaned version: the schema
change remains, and rewriting the ledger loses its history.
Resolve metadata access through the sanctioned MCP or the operator; the metadata
dump above is not permission to acquire direct production credentials.
Heavy refreshes and index builds¶
These are deliberate operator procedures on the identified database. Coding
agents must not use direct production credentials as a fallback for unavailable
MCP access. For an already-open psql session, provide SQL and \i / \timing /
\set ON_ERROR_STOP on, not another shell connection command. Give absolute
\i paths or state the directory psql was started from.
-
Inline full-table rebuilds in migrations: bound
lock_timeout+statement_timeout, never disable them. A migration that calls a mart-rebuild inline (SELECT analytics.refresh_experiment_results();,REFRESH MATERIALIZED VIEW …, anyCREATE TABLE ASscanningsession_events) exceeds the connection'sstatement_timeouton prod-sized data →SQLSTATE 57014. Do NOT fix it withstatement_timeout = 0— the rebuild's blue-green swap takesACCESS EXCLUSIVEon the liveexperiment_*marts, and unbounded means it can queue behind a live reader and block everything behind it. Instead, at the top of the file (plainSET, notSET LOCAL— must survive whetherdb pushuses one transaction or one per statement):SET lock_timeout = '5s';(swap fails fast under contention, migration aborts cleanly, re-run when idle) +SET statement_timeout = '15min';(generous ceiling for the scan, not infinite). ASETinside the function's ownCREATE FUNCTION … SET …clause does not help — the timer is armed by the caller's statement, so the guard must precede theSELECT. Better still: keep the heavy rebuild OUT of the migration — DDL only (sub-second locks), rebuild via the cron / out-of-band — unless a same-migration RPC depends on the new mart columns (IX-3534'sget_experiment_sanity()does, so it stays inline but bounded). -
PostgREST runs every request under
authenticator(statement_timeout=8s;anon3s) —service_rolecarries no override. A57014from a job at volume is usually contention, not an oversized batch: the mart refresh crons starve a normally 1-3s request past the cap. Current worst offenders (7d,cron.job_run_details):refresh-mv-device-stats-dailyat04:15(avg 443s, max 7m41s) andrefresh-mv-conversation-visitorsat:00(avg 46s).refresh_mv_widget_visitors_safeused to be the big one — a fullREFRESHmeasured at 248s in IX-4199 and 17min by IX-4498 — but it is incremental since IX-4498 and now runs ~12s, so a:30collision is no longer a plausible cause. Check what else was running before shrinking the batch — a job whose chunk sizes are constants gets a shorter run from a lower batch limit, not a shorter statement. (IX-4199 replay: passes died at:00, survived off the hour.) -
CREATE INDEX CONCURRENTLYcannot be built through the Supabase SQL editor / API on a large table — the ~60s platform gateway kills it mid-build and leaves an INVALID index shell. Run thescripts/backfill/*_concurrently.sqloperator script viapsqlover the Direct connection string (Dashboard → Project Settings → Database → Connection string → Direct,db.<ref>.supabase.co:5432, reveal/reset the password there) — raw Postgres, no HTTP gateway,statement_timeout='60min'. The pooler creds inbackend/.env.productiondon't authenticate from a dev box. The scripts already drop any INVALID shell before rebuilding. Hand the operator the expected wall-clock andpg_stat_progress_create_indexwith the script. -
A materialized view created
WITH NO DATAreads as empty (all-zero cards, blank charts) until its firstREFRESH— and that first full build cannot run through the Supabase SQL editor / MCP (~60s gateway cap kills it). A migration that shipsCREATE MATERIALIZED VIEW … WITH NO DATA(lock-light DDL) leaves the populate as an operator step; merging the migration to production does not fill the mart. The 4h refresh cron does not self-heal it fast either: the first build is a full scan, andCONCURRENTLYis invalid on a never-populated MV (falls back to a plainREFRESHbounded by the cron'sstatement_timeout, e.g. 15min — if the build exceeds that it fails silently every cycle and the mart stays empty forever). Symptom seen: IX-3593 —mv_conversation_facts(#1416) shipped empty, Home Conversion funnel + journey returned zeros. Operator runs the first build viapsqlover the Direct connection (same string as the CONCURRENTLY note above —db.<ref>.supabase.co:5432, password from Dashboard → Project Settings → Database → Connection string → Direct):Verify withPGPASSWORD='<direct-db-password>' psql \ "host=db.<ref>.supabase.co port=5432 dbname=postgres user=postgres sslmode=require" \ -c "SET statement_timeout = '60min';" \ -c "REFRESH MATERIALIZED VIEW public.<mv_name>;"SELECT count(*) FROM public.<mv_name>;(non-zero = populated). Then confirm the recurring cron's first auto-refresh finishes under its ownstatement_timeout; if not, raise that timeout or move to an incremental refresh. -
To trigger a heavy cron-managed refresh function on demand (
analytics.refresh_experiment_results(), mart rebuilds, etc.), run it directly viapsqlover the Direct connection — NOT the SQL editor, NOT an every-minute cron. The function isSECURITY DEFINERwith EXECUTE revoked fromanon/authenticated/PUBLIC, so only the owner runs it; the multi-minute run also blows the SQL editor / MCP ~60s gateway. Two wrong instincts to avoid: (a) a plainSELECT analytics.refresh_experiment_results();in the SQL editor → gateway kills it mid-run; (b) acron.schedule('oneshot','* * * * *', …)to "fire it once" → pg_cron does not guard against overlap, and a job that runs minutes will have the next minute's tick stack a second concurrent rebuild →ACCESS EXCLUSIVEswap contention / lock pile-up (the exact storm a rebuild causes). Right way — operator runs as owner over the Direct connection (db.<ref>.supabase.co:5432, password from Dashboard → Project Settings → Database → Connection string → Direct), synchronous, full budget, no gateway, no overlap:PGPASSWORD='<direct-db-password>' psql \ "host=db.<ref>.supabase.co port=5432 dbname=postgres user=postgres sslmode=require" \ -c "SET statement_timeout = '45min';" \ -c "SELECT analytics.refresh_experiment_results();"psqlreturning the result row with no error = the run finished; the wall-clock it blocked is the real run time. If a one-shot cron is genuinely the only option (no Direct access), schedule a SINGLE specific upcoming minute (MM HH * * *), let it fire once, thencron.unscheduleit before it can repeat — never* * * * *.