Testing knowledge changes¶

Use this procedure to choose the smallest test environment that proves a change to knowledge ingestion, retrieval, or its backoffice surfaces. Local processes are the fastest feedback loop; a cloud preview is required when the code needs the cloud network path or isolated retrieval stores. Neither replaces the final diagnostic and retrieval evidence when stored knowledge can change.

Agents should use knowledge-ingestion-control-plane when starting or debugging the local ingestion services and rose-knowledge-diagnostics when inspecting RAG storage. Those skills add orchestration and safety checks; this page is the shared procedure for developers and agents.

Choose the test path¶

Change or question	First feedback loop	When cloud is required	Pre-merge proof
Pure function, contract, repository, or handler logic	Focused backend tests	Only if the changed boundary has no local execution path	Relevant suites from backend testing
Backoffice knowledge UI or Admin/Knowledge API route	Whole local stack against local or preview Supabase	Only for a cloud-only dependency or callback	Frontend checks plus the owning backend suites
Knowledge API/content-worker dispatch or callback behavior	Local API/worker first; add tunnels only when delivery itself matters	Use a cloud preview when Cloud Tasks, scheduler recovery, IAM, or stable callback URLs are part of the claim	Operation state and logs, plus focused control-plane tests
Document-loader, chunking, embedding, storage adapter, or full ingestion path	Unit/integration suites	Use a cloud preview for any trial that runs the loader or writes retrieval storage. An adapter-only fix may stop at units when the verification checklist requires no rebuild	Knowledge verification checklist
Stored corpus integrity, tenant isolation, cleanup, or deletion safety	Read-only `diagnose-tenant` baseline	Use the environment whose stores are in scope; use preview storage for a write trial	Comparable before/after diagnostics with coverage and findings
“Does the chatbot use the changed knowledge?”	Passage capture and `rose-chat` against the same target as the trial	Use the preview search API when the candidate corpus lives in preview retrieval storage	Retrieved passages, sources, and a traced chat result

Use more than one row when the change crosses boundaries. For example, a new document-loader handler needs focused tests, a cloud-preview job trial, matching before/after diagnostics, and retrieval/chat evidence. A UI-only display change does not need a loader run.

Choose the database before starting services¶

IX_ENVIRONMENT=test does not select an isolated Supabase database. Test, staging, and production services share the production Supabase project unless .env.local overrides it.

Choose one target and make every local reader and writer agree:

Target	Setup	Use it for	Boundary
Local Supabase	From `supabase/`, run `just dev`	Fast UI/API/handler iteration with synthetic data	Laptop-only Postgres; no isolated MongoDB, Neo4j, Redis, or LightRAG
Hosted preview Supabase	From the repository root, run `./bootstrap.py --branch`	Branch migrations, hosted callbacks, and local services that need an isolated Postgres target	Postgres only until paired with cloud-preview retrieval prefixes
Shared production Supabase	No override; the host is `drtzxyuvppalvgczwhne.supabase.co`	Read-only inspection or an explicitly authorized production operation	`test` and `staging` labels do not isolate its rows

Both local Supabase and ./bootstrap.py --branch write the effective credentials to backend/.env.local and frontend/.env.local. Start or restart services only after choosing the target; do not edit those files by hand. Verify the host that the launcher prints before using any mutation flow.

For local Supabase, use the repository recipe rather than a bare supabase start:

cd supabase
just dev

The recipe starts Supabase and runs gen-env; without that second step, the application services continue loading .env.test and can silently reach shared production Supabase. Local knowledge services also need Application Default Credentials for GCP-backed providers and OIDC:

gcloud auth application-default login

Choose a test client and workspace shape¶

Use the fictional TestFeatures client for local or preview backoffice/API smoke tests. Its migration-created root workspace is registered at testfeatures.com/, and local Supabase auto-login gives the admin access to it. Confirm the fixture before testing:

select w.id, w.config_key, r.host, r.path_prefix, r.enabled
from public.workspaces w
join public.workspace_roots r on r.workspace_id = w.id
where r.host = 'testfeatures.com'
order by r.path_prefix;

Choose the workspace shape that matches the change:

Question	Fixture
Does the local backoffice/Admin/Knowledge API path work for an ordinary workspace?	Select the existing `TestFeatures` workspace at `/`
Does host/path resolution keep sibling workspaces isolated?	Run `just test-knowledge-ingestion-control-plane-migration` from `supabase/`; its disposable `routing.example` fixture creates `/fr`, `/uk`, and `/fr/pricing` workspaces and rolls them back
Does workspace-keyed configuration preserve the host sibling?	Run `supabase/tests/ix5071_client_configs_workspace_key.test.sql` against isolated local/preview Postgres; it creates a host workspace plus `/fr` and rolls back
Does an interactive UI flow specifically require a path workspace?	Create the sibling under `testfeatures.com` only in local/preview Supabase, record its ID, and remove it by resetting/tearing down that isolated target

For the focused workspace-key test, run from the repository root:

psql postgresql://postgres:postgres@127.0.0.1:54322/postgres \
  -v ON_ERROR_STOP=1 \
  -f supabase/tests/ix5071_client_configs_workspace_key.test.sql

Run the workspace non-regression checks when the implementation changes workspace/root selection, routing, or workspace_id ownership. Keep the host workspace and path sibling distinct in assertions; seeing rows for testfeatures.com alone does not prove that the selected workspace owns them.

TestFeatures currently has no seeded rows in config.knowledge_faqs. It proves workspace selection and control-plane routing, not retrieval quality. Do not add a curated FAQ merely to make the smoke test pass: use the rolled-back SQL/unit fixtures for ownership behavior and an authorized preview corpus for actual loader, retrieval, and chat evidence.

Start the local application stack¶

For backoffice → Admin API → Knowledge API → content-worker development, run from backend/:

just dev-knowledge-stack test

The command starts and supervises:

Process	Address
Client backoffice	`http://localhost:3002`
Admin API	`http://localhost:8080`
Knowledge API	`http://localhost:8081`
Content worker	`http://localhost:8082`

It refuses to reuse occupied ports, prints the effective Supabase host, and records each owned PID, working directory, port, and log path under .context/dev-knowledge-stack/. Ctrl-C stops only the process groups started by that invocation. The local Supabase env generated by just dev configures backoffice auto-login as admin@admin.com / admin.

The equivalent commands, each in its own terminal, are:

# backend/
just dev-knowledge-api test

# backend/
just dev-knowledge-content-worker test

# backend/
export KNOWLEDGE_API_URL=http://localhost:8081
export KNOWLEDGE_API_OIDC_AUDIENCE=http://localhost:8081
export KNOWLEDGE_API_IMPERSONATE_SA="$(gcloud run services describe admin-api-test --region=europe-west9 --project=inboundx --format='value(spec.template.spec.serviceAccountName)')"
just dev-admin test

# frontend/client-backoffice/
just dev-local-admin http://localhost:8080

Use the separate commands when one service needs a custom port, tunnel URL, or OIDC audience. Follow the local control-plane reference for those arguments and authentication requirements.

Before any local write, verify the environment of the processes that actually hold the backend ports. Each command must print a local API URL containing 127.0.0.1:54321, not the shared Supabase host:

for port in 8080 8081 8082; do
  pid="$(lsof -nP -iTCP:"$port" -sTCP:LISTEN -t)"
  ps eww -p "$pid" | tr ' ' '\n' | grep '^SUPABASE_URL='
done

Then check process health without creating knowledge state:

curl --fail http://localhost:8080/health
curl --fail http://localhost:8081/ready
curl --fail http://localhost:8082/ready

Open the backoffice knowledge page and confirm its authenticated operations request returns 200 before testing a form or handler. A 502 from the Admin API usually means the local Knowledge API URL was supplied without the matching OIDC audience or Admin API impersonation identity; the aggregate launcher sets all three.

Starting four processes does not make the ingestion path fully local:

Cloud Tasks cannot deliver to localhost; a real dispatch needs authenticated HTTPS tunnels or a cloud-preview job.
GCS, Firecrawl, model providers, and Cloud Tasks remain remote.
A deployed document-loader cannot inherit the laptop's .env.local and cannot safely complete work created only in an isolated local database.
The document-loader write path needs the cloud NAT IP accepted by MongoDB Atlas.

Use the local stack for endpoint, UI, contract, and callback iteration. Use tunnels only when the local callback path itself is under test; otherwise a cloud preview is simpler and closer to the merge gate.

Use a cloud preview for ingestion and retrieval writes¶

From backend/, deploy the branch's ingestion job (previews are job-only — a shared -test service is never redeployed):

just cloud-preview job:document-loader   # or just: `just cloud-preview` (default)

Verify the exact callback URLs, OIDC audiences, queues, and preview job name used by the operation: the preview job does not automatically reroute the shared test scheduler or queues. If those connections are not preview-specific, the run proves the job in isolation, not end-to-end orchestration.

Deploy and run the preview document-loader once, passing the tenant for this run:

just cloud-preview-run document-loader TENANT_ID=testfeatures.com LIMIT=1

TENANT_ID is required; LIMIT bounds the legacy source pass; add IGNORE_TIMESTAMPS=true to force a reload. Before trusting the result, confirm in the execution's logs that storage_manager set both tenant IDs to <id>__<tenant> and that ixmongo used database <id>_test. A bare tenant ID or rose_test means the run touched shared test data. Then read the final Document update completed counts. See the cloud-preview procedure for teardown and isolation details.

The preview uses one branch-derived identity for an isolated Supabase branch, its own Mongo database, and a tenant-ID prefix for Mongo, Neo4j and the loader's Redis state. It never shifts live test traffic.

just cloud-preview ensures the current branch's Supabase preview and rewrites the generated .env.local files to that branch. Local services started after that point therefore see the same Supabase rows as the preview. This is useful for a local UI or Admin API while inspecting a preview operation.

Do not assume that this makes the local processes part of the preview runtime. The cloud services also receive the preview-only Mongo database and tenant prefix through their secret; the local stack does not. Do not split one operation across local and preview API/worker/loader processes unless the control-plane tunnel guide has made every callback URL, OIDC audience, queue, and database target agree. For retrieval and loader proof, call the preview service or job itself.

When finished, tear down locally created preview resources:

just cloud-preview-down

Teardown deletes the Supabase branch but leaves .env.local pointing at it. From the repository root, point local services back at the shared project:

python3.12 scripts/supabase_branch.py unwire

Clean up local state¶

When the chosen database was local Supabase, stop it from supabase/:

just stop

This also removes the generated Supabase entries from the backend and frontend .env.local files so a later development session does not silently keep using the stopped local target.

Apply the merge gate¶

The runtime choice answers “where can this code execute?” The knowledge verification checklist answers “what evidence is required before merge?” Follow its relevant stages:

Run the combined owning unit suites and any guarded FAQ integrations.
Capture a read-only diagnose-tenant baseline when storage can change.
Run an authorized ingestion trial in the selected isolated target only when the change needs one.
Capture matching after-state diagnostics and raw retrieved passages.
Run a traced rose-chat question against the same retrieval environment when answer behavior is part of the claim.
Report target, commit/image/execution, commands and exits, coverage, findings, retrieved sources, remaining unknowns, and preview teardown.

A successful service health check proves startup. A successful loader execution proves completion. Neither proves corpus integrity, tenant isolation, retrieval relevance, or answer quality; retain the corresponding diagnostic and retrieval evidence.