GEO Agent¶

The GEO (Generative Engine Optimization) Agent transforms real visitor questions from the Website Agent into published FAQ pages that get cited by AI search engines (ChatGPT, Perplexity, Google AI Overviews) and improve the Website Agent's own answers.

Pipeline¶

A run targets one workspace. Start it from the backoffice (see Generate suggestions from the backoffice) or from a terminal with rose-geo run <domain | config_key> (e.g. sundayapp-com-fr).

Collect visitor questions from Supabase messages × conversations (last 30 days, production only) for the workspace's host.
Filter noise (support requests, navigation queries, bots, spam).
Group by page_url and pick top-N pages by question volume.
Cluster questions per page (DBSCAN, cosine distance) + a site-wide global pool.
Generate FAQ pairs grounded in the client's knowledge base (LightRAG hybrid retrieval).
Diff them against the published entries in geo.faqs and write proposals to geo.faq_proposals for review (see Review before publishing).

A run never writes geo.faqs and never publishes. Publish now in the backoffice (/admin/geo/republish) renders the accepted entries to static HTML + JSON-LD (FAQPage + Article + BreadcrumbList), uploads HTML + sitemap + metadata to Cloudflare R2 and notifies search engines. geo.faqs is publishing content, separate from config.knowledge_faqs: nothing here reaches chat retrieval.

See docs/plans/2026-04-10-ix-2453-geo-content-agent-design.md for the full architecture.

Sections¶

A workspace's FAQ can be organized into sections a buyer browses (pricing, integrations, comparisons…), stored in geo.sections and geo.faqs.section_id (IX-5745). One section list serves every language page; titles are stored per language. Sections are optional: a workspace without them works as before.

Suggest. With 12 or more published questions and no sections, the FAQ page's Organize step offers Suggest sections: one LLM call groups every question into 4 to 8 sections, and code rejects a plan that leaves out or repeats a question, has a section of fewer than 3 questions, or uses a slug shorter than 3 characters (two-letter path segments are language pages). The reviewer renames sections, moves questions and removes possible duplicates, then applies; nothing is stored before Apply.
Own. After Apply, people edit a section's title (per language) and address (its slug, shared by every language), add, reorder and delete sections (a deleted section's questions become Unsorted) and move questions with their section menu. Only staff can suggest a whole new list again, since it changes section page addresses.
Unsorted questions (a language page sorted later, a hand-written entry, a suggestion that fits no section) get a Sort them prompt, which places them in the existing sections only.
New suggestions. A run places each new question in a section (payload.section_id); the reviewer can change it before accepting.

All reads and writes go through /admin/geo/sections… and /admin/geo/faqs/{id}/section; geo.sections is service_role-only. rose-geo sections <workspace> prints a fresh suggestion without storing it. Tune the grouping with rose-eval geo-sections --n-runs 6 --verbose: a frozen copy of Rose's published questions with a hand-made grouping, in seconds. Publishing one page per section is planned in docs/plans/2026-10-26-ix-5745-geo-site-sections.md (B3).

Review before publishing¶

Everything on the page is human-approved (IX-4657). A run emits two kinds of proposal:

New question: no published entry covers it.
Updated answer: a new answer for a published entry, in two cases only:
- The workspace's knowledge changed since the answer was written. Each accepted answer records the knowledge revision it was written against: the sum of the revisions retrieval has loaded for the run's environment (knowledge.ingestion_corpus_freshness.loaded_revisions; staging reads test). The run regenerates every entry whose revision is older, sampled or not, and even when there is too little traffic for new questions. A workspace without freshness rows gets no automatic refresh.
- A person pressed Refresh answer (geo.faqs.refresh_requested_at). The new answer always reaches review, and the decision clears the request.
A hand-written or hand-edited answer (human_edited_at) is only refreshed on request. An automatic refresh is proposed when the answer materially changed: a different number (price, date, count), or word overlap below 60%. Otherwise it is recorded as unchanged, and like a rejected refresh it is not run again until the knowledge changes. A refresh stops when the run's generation budget or deadline is spent; the rest wait for the next run. A sampled question that matches an entry never rewrites its answer: different wording from the model is noise.

The page accumulates. Each run samples a different set of questions, long-tail ones above all, so an entry missing from one run is not evidence that it stopped mattering: a run never proposes removing anything. A human removes a question that is no longer relevant, before publishing.

A generated question matches an entry by the long-tail dedup rule (substring, or question token overlap of 80% or more). The inbox accumulates like the page: a suggestion stays until a reviewer decides it. A run never proposes a question already waiting for review or regenerates an answer already waiting at the current knowledge revision; a newer answer for an entry supersedes its older pending one. A rejected proposal is not proposed again, and a question a human removed (geo.faqs.retired_at) never comes back as a suggestion.

The GEO Agent opens on the FAQ page editor (/config-studio/content-agent, IX-5775); its settings (on/off, Custom domain, Pipeline Settings) are behind the header gear (/config-studio/content-agent/setup). The editor has four steps: Review (suggestions and published questions in one list; accept, reject, edit before accepting, bulk decisions, add, delete), Organize (sections and order, drag and drop), Style (the page style on the client's own questions) and Publish (changes since the last publish, hosting status, translations, export, the only Publish button). ?step= selects the step; the old /faqs and Page Style routes redirect to it. A Visitor questions link in the header opens the observatory (/config-studio/content-agent/observatory). Every write goes through the admin API (/admin/geo/proposals, /admin/geo/faqs), which authorizes the workspace; both tables are service_role-write only. Accepted entries record the run and proposal in generation_metadata; hand-written entries have none. Changes reach the public page on the next Publish now.

There is no auto-accept yet: every suggestion waits for a human.

Generate suggestions from the backoffice¶

Generate suggestions on /config-studio/content-agent/faqs starts a run for the selected workspace (IX-5691). It is safe for any reviewer: a run only writes proposals.

POST /admin/geo/runs authorizes the workspace, refuses it without the geo feature (403), inserts a queued geo.runs row and executes the Cloud Run job geo-run-<env> with a GEO_RUN_ID override. The job claims that queued row and runs its workspace; a run it cannot claim (queued in another database, or already failed) runs nothing, so a local admin API never starts work against shared Supabase. The admin API cannot run the pipeline itself: it takes about 10 minutes and needs ixchat. A partial unique index allows one queued or running run per workspace; a second start gets 409.
The job reports each stage to its row: collect, cluster, generate (candidates done / at most), refresh (published answers done / stale), propose, for the default-language page only (other pages are translated on accept). It ends succeeded with the new-question and updated-answer counts, or failed with an error the reviewer sees: a hosting conflict verbatim, anything else as a generic message.
The page polls GET /admin/geo/runs/latest every 3 seconds while a run is active, and refreshes the inbox when it ends.
A queued or running row that has not changed for 30 minutes is a dead job (task timeout, crash, failed start): the admin API marks it failed on the next read or start, so it never blocks the button.

The job runs in every environment, without retry (a retry repeats every LLM call). Staging and test answer from test knowledge but write to the shared Supabase like rose-geo run --env staging. Logs of one run: gcloud run jobs executions list --job geo-run-production --region europe-west9, then the execution's logs, which hold the traceback.

Test a run on a local copy¶

Test a pipeline or review change on a client's real data without writing shared Supabase. Pick a client with recent visitor traffic: cactusinbound.com is a test site with no visitors, so a default 30-day run collects nothing (--lookback widens the window).

Start an isolated stack from a copy of supabase/ under .context/<name>/ with its own project_id (see the SQL-test note in supabase/AGENTS.md). Use supabase start --workdir .context/<name> -x studio,imgproxy,inbucket,edge-runtime,logflare,vector,supavisor,realtime,storage-api, not supabase db start: the pipeline and admin API need PostgREST, and a later supabase start on a stack started with db start keeps it database-only.
scripts/seed_local_db.py --domains <host> --tables clients,domains,config.client_configs with --target-url / --target-key from the isolated stack's supabase status (the seeder defaults to the shared local stack), then copy the host's conversations, their messages and visitors, and geo.faqs yourself: the seeder's --since drops visitors older conversations reference, so their inserts fail. Set the copied visitors' account_id and person_id to null: accounts and people are not copied, and their foreign keys refuse the insert.
The copied workspace starts with corpus_status = 'absent' and storage_key = 'site:<id>', so every answer fails or reads an empty tenant. Set corpus_status = 'ready', and set the production storage_key with sites_assign_storage_key disabled for that one update. Copy the workspace's knowledge.ingestion_corpus_freshness rows to exercise refreshes.
Export the stack's SUPABASE_URL, SUPABASE_SERVICE_ROLE_KEY and SUPABASE_ANON_KEY in the shell (they win over env files; without the anon key the admin API's config resolver falls back to Secret Manager and fails on GCP_PROJECT_ID), then run rose-geo run <host> --env staging, ixadmin_api.uvicorn_server and the backoffice with VITE_DIRECT_SUPABASE_* and VITE_ADMIN_API_BASE_URL overrides. Never press Publish now on the client's workspace: it uploads to its live page. Test publishing as in step 6.
To test Generate suggestions, press it, then run the local run yourself: the deployed geo-run-<env> job finds no such row in shared Supabase and runs nothing, so the local row stays queued (or fails when the job cannot be started). Run apps/jobs/geo_run/run_geo_run.py from backend/ with GEO_RUN_ID set to the queued run's id (insert a new queued geo.runs row only if it failed), IX_ENVIRONMENT=staging IX_IS_LOCAL=true and the stack's exports. The open page shows its progress.
To test Publish, publish a throwaway workspace instead. Add a domain such as <ticket>-e2e.test for the same client. Copy the host's config.client_configs rows to it with geo.publishing.public_base set to https://<that domain>/faq and public_host removed (subdirectory hosting skips IndexNow). Copy a few geo.faqs rows with the new workspace_id and site_domain: a row whose site_domain names another host is refused. Publishing still writes the shared R2 bucket, so delete the objects under <domain>/ afterwards.

Workspaces on one host (sunday: sundayapp.com, /fr, /en-gb) each get their own page and share their questions, never their answers (IX-5504):

Each question is attributed to the workspace whose canonical root is the longest prefix of its page_url path; no page URL means the bare-host workspace. conversations.workspace_id is not used: a client without explicit routing records every widget turn under its bare-host workspace.
Questions are clustered for the whole host. A workspace's page keeps the clusters asked on its own pages plus the generic ones asked in at least two workspaces; per-page sections use only its own pages. A single-workspace host is unaffected.
Answers come from the workspace's own corpus and config: ixchat is called with the workspace's location (preview routing), and its pages are in its own languages (see Pages in several languages).
geo.faqs, geo.faq_proposals and geo.question_clusters are keyed by workspace_id; site_domain stays the registered host.
The page is published under the workspace config_key (faq.userose.ai/sundayapp-com-fr/frequently-asked-questions). A bare-host workspace's key is its domain, so its URL does not change.
Each page lists hreflang alternates for every language page of its workspace and of every published sibling. The tag is the page language, plus the workspace's geo.publishing.hreflang_region when set (en-GB). Tags that still collide (US and UK both en, no region) are dropped. Alternates converge as each sibling republishes.
A path workspace may not publish to a URL a sibling uses (public_base / public_host): a workspace created from its host's config inherits the host page's URL, and the run and the republish refuse it (HTTP 409) until it gets its own URL or none (Rose-hosted).
Custom subdomain pointers stay keyed by the registered host, so the backoffice domain page is not open to path workspaces; subdirectory hosting (public_base) works per workspace.

Pages in several languages¶

A workspace publishes one page in its identity.default_language and one in each language of geo.page_languages (IX-5730). It is an explicit list, not identity.additional_languages: that one drives chat buttons and CTAs, and often names a language a sibling workspace already publishes in (sundayapp.com lists fr, served by /fr). Unset means one page, as before.

Generation. A run generates, selects and proposes the default-language page only (IX-5775). The other pages are translations of its accepted entries: nothing is generated or reviewed per language.
Translation. Accepting a proposal, adding or editing a default-language entry translates it into every page language (ixgeo/translation.py, use case geo_faq_translator). The write returns at once; the backoffice then calls POST /admin/geo/translations/sync?faq_id= without waiting, a separate request so Cloud Run keeps CPU for it. A translation must keep every number of its source; if it changes one, it is not written and the entry shows as missing. Deleting an entry retires its translations. Section titles missing in a language are translated when sections are applied or renamed.
Storage. geo.faqs.language and geo.faq_proposals.language name the page; NULL is the default-language page (rows written before IX-5730). A translation row has source_faq_id (its default-language entry, same workspace) and generation_metadata.source_hash (the source text it was made from). A translation page takes its order and sections from the source entries.
Status. GET /admin/geo/translations reports each entry per language: up to date, out of sync (its source changed and the re-translation failed, so the page keeps the previous version) or missing. A changed source is always re-translated: the editor has no screen for editing a translation. POST /admin/geo/translations/sync fills a page (missing and out-of-sync translations, section titles), or re-translates one entry with faq_id.
Review. Questions are added and reviewed on the default page only; the admin API refuses POST /faqs on another language page (422).
URLs. The default page keeps its URL; another language lives under /{lang} of the same base: faq.userose.ai/{config_key}/de/frequently-asked-questions, faq.client.com/de/frequently-asked-questions, {public_base}/de/. A subdirectory proxy that forwards {public_base}/* needs no change.
Publishing. Publishing first finishes the missing or stale translations, then renders every language page that has entries. A translation that still fails is never silent: the source keeps the reason (generation_metadata.translation_errors), the Publish step lists each such question with its language and reason, and the publish response (untranslated) makes the backoffice warn after publishing. A new question stays off that page; an updated one keeps its previous translation. A language page without entries is not rendered (the default page must have some). Each page has its own canonical URL and hreflang alternates; one sitemap lists them all with xhtml:link alternates; IndexNow pings every URL. The Worker serves a language page only when meta.json language_urls lists it, so removing a language takes its page down at the next publish. Each publish records what every page held in geo.publications; GET /admin/geo/changes lists the default page's changes since the last one (none before the first).

Publishing & Delivery¶

Rendered FAQ HTML lives in Cloudflare R2 (rose-geo-pages bucket) and is served by the cloudflare-geo-pages Worker at faq.userose.ai. Clients can optionally point a CNAME faq.client.com → faq.userose.ai; the Worker resolves the host to a tenant via per-tenant meta.json stored alongside the HTML.

Canonical URL strategy¶

Each tenant has exactly one canonical URL to avoid SEO duplicate-content penalties. The canonical is baked into the static HTML at render time:

Tenant config	Canonical URL
No custom domain	`https://faq.userose.ai/{config_key}/frequently-asked-questions` (`config_key` = domain for a bare-host workspace)
Custom CNAME configured (`publishing.public_host`)	`https://{public_host}/frequently-asked-questions`
Subdirectory via client reverse proxy (`publishing.public_base`)	`{public_base}/`

A page in another language than the default one appends /{lang} to the base (see Pages in several languages).

The Worker enforces canonicalization: any request whose Host does not match the tenant's public_host is 301-redirected to the canonical URL. The HTML body itself is identical at both hosts (same R2 object), but only one URL is indexable. With public_base, direct hits on faq.userose.ai/{site_domain}/... are 301-redirected to {public_base}/, except requests whose X-Rose-Public-Base header equals public_base exactly: those come from the client's proxy and are served.

Set up custom hosting for a client (staff)¶

The client follows the hosting guides; staff finish the setup. The two custom options need different work on our side, and public_host and public_base are mutually exclusive.

	Subdomain (`faq.client.com`)	Subdirectory (`client.com/faq`)
Client does	`CNAME faq.client.com → faq.userose.ai`	Reverse-proxy rule on their CDN or host (five requirements)
Requests reach our Cloudflare with `Host`	`faq.client.com`	`faq.userose.ai`
Our Cloudflare	Add a custom hostname (below)	Nothing: no custom hostname, IP or allowlist
Config	`geo.publishing.public_host`	`geo.publishing.public_base`

Subdomain:

Check the client's CNAME resolves: dig +short faq.client.com CNAME returns faq.userose.ai.
In the Cloudflare dashboard, userose.ai zone → SSL/TLS → Custom Hostnames → Add Custom Hostname faq.client.com (fallback origin faq.userose.ai, already active). Wait for the certificate and hostname to show Active.
Backoffice → Config Studio → Content agent → Domain → Custom subdomain: enter faq.client.com and save. The status checklist on that card walks through DNS, Cloudflare, publication and canonical.

Subdirectory:

Wait until the proxy works. Open https://client.com/faq/ in a browser; it must show the FAQ. Use a browser, not curl: managed hosts often return 403 to command-line tools. Do not set public_base before this: the 301 would send every visitor of faq.userose.ai/{site_domain}/... to a broken proxy.
Note the exact host the site serves (follow its redirect: www.client.com or client.com). public_base must equal the client's X-Rose-Public-Base header character for character, with no trailing slash.
Backoffice → Config Studio → Content agent → Domain → Subdirectory hosting (advanced): enter the public base URL and save. Or use rose-config for geo.publishing.public_base.
Republish: Config Studio → GEO Agent → step Publish → Publish (re-renders stored FAQs, no regeneration). Republishing rebases the canonical, OpenGraph and JSON-LD URLs, the sitemap and robots.txt, and enables the worker's 301.
Verify: the subdirectory card's proxy check shows Proxy live — canonical matches; curl -sI https://faq.userose.ai/{site_domain}/frequently-asked-questions returns 301 to {public_base}/; the proxied page's <link rel="canonical"> points at {public_base}/.

Symptom	Cause	Fix
`502` on the client's `/faq` before step 3	Proxy calls an IP or sends the client's `Host`/SNI; Cloudflare refuses	Client proxies to the hostname `faq.userose.ai` with `Host` and SNI `faq.userose.ai`
Host asks Rose for an IP or `X-Real-IP` value	`faq.userose.ai` is on shared Cloudflare anycast IPs that can change	There is none to give; proxy to the hostname
Redirect loop after step 4	`X-Rose-Public-Base` missing or not equal to `public_base`	Client fixes the header, or correct `public_base` and republish
Proxy check shows Proxy serves wrong canonical	Not republished, or the proxy maps to another tenant	Republish; check the path mapping `/faq/<path>` → `/{site_domain}/<path>`

To revert, clear the field, republish, then let the client remove the proxy rule.

Search Engine Discoverability¶

After each pipeline run the Worker (and the pipeline's IndexNow ping) make the new content discoverable across the major engines used by AI search.

Engine	Mechanism	Indexing latency	Why it matters
Google	`sitemap.xml` with `<lastmod>` + JSON-LD `dateModified`	Hours → days	Largest crawl footprint. Powers Google AI Overviews.
Bing	IndexNow protocol (real-time push)	Minutes	Powers ChatGPT search — direct path into AI citations.
Yandex	IndexNow protocol	Minutes	Same protocol as Bing.
Perplexity	Inherits Bing + Google indexes	Inherits both	No dedicated submission API.
All bots	`robots.txt` + per-tenant `sitemap.xml`	Continuous	Allows discovery without prior knowledge of the URL.

Sitemap¶

Each tenant gets one sitemap.xml listing the canonical URL of each of its language pages, with <lastmod> = publish timestamp and, when there are alternates, xhtml:link rel="alternate" hreflang entries. The Worker serves it at:

https://faq.userose.ai/{site_domain}/sitemap.xml
https://faq.client.com/sitemap.xml (custom domains)

The <lastmod> value is the only freshness signal Google reliably honors after they removed the ping endpoint in January 2024.

IndexNow (Bing / Yandex / ChatGPT)¶

IndexNow is a push protocol supported by Bing, Yandex, Naver, Seznam, and Yep. A single POST to https://api.indexnow.org/indexnow notifies all participating engines that a URL changed; they recrawl within minutes instead of days.

Why it's worth wiring up: Bing's index is the primary source for ChatGPT search results. Faster Bing indexing → faster AI citations for newly generated FAQs.

Domain ownership verification: IndexNow requires a static key file (e.g. /{indexnow_key}.txt) served at the same domain as the submitted URL, containing the key string as its body. The Worker serves this file from configuration.

Failure mode: The pipeline's IndexNow ping is best-effort — failures are logged but do not fail the pipeline. The sitemap remains the canonical fallback.

Google: no equivalent push API¶

Google deprecated google.com/ping?sitemap=... in January 2024 and offers no general-purpose real-time indexing API:

Indexing API: documented as JobPosting and BroadcastEvent only. Using it for other content is undocumented and may be ignored or penalized — not used here.
Search Console URL Inspection API: ~2000 requests/day per property, too restrictive for automated use.

The supported path for Google is sitemap + accurate <lastmod>, plus a one-time Search Console verification per domain after Worker deployment.

Response Headers¶

The Worker sets the following headers on FAQ HTML responses:

Content-Type: text/html; charset=utf-8
Cache-Control: public, max-age=86400
X-Robots-Tag: index, follow

Edge KV caches the R2 object for 60s so a fresh pipeline run propagates quickly; the downstream Cache-Control is the longer 24h browser/CDN TTL.

Schema Stacking¶

Every rendered page contains three JSON-LD blocks for maximum AI citation coverage:

FAQPage — primary structured-data schema for Q&A content.
Article — wraps the page as a publishable article with datePublished and dateModified set to the pipeline run timestamp.
BreadcrumbList — navigation context.

Research consistently shows that stacked schemas roughly double AI citation rates versus a single FAQPage block. dateModified is set at pipeline render time (not at every HTML construction) so cached HTML stays consistent with what search engines see.

Configuration¶

The publishing block of the geo config slug controls page styling and delivery:

Field	Type	Purpose
`public_host`	`str \\| None`	Custom delivery host. When set, canonical URL is `https://{public_host}/frequently-asked-questions`. When unset, defaults to `https://faq.userose.ai/{site_domain}/frequently-asked-questions`.
`hreflang_region`	`str \\| None`	Staff-only. ISO country code added to every page's `hreflang` tag (`GB` → `en-GB`), for same-language siblings.
`title`, `subtitle`, `logo_url`, `custom_css`, `theme_preset`, color fields, font fields	various	Visual styling — see `GeoPublishing` model.