Skill System¶

The skill system provides modular, composable capabilities that the AI agent can use to generate responses. Skills are organized by category and can be enabled/disabled per client.

Architecture¶

Skill Definition¶

Each skill is defined in a SKILL.md file with YAML frontmatter:

---
name: skill_name
default_enabled: true
description: Brief description for LLM selection
category: response_ending
required: false
emoji: "🎯"
always_on: false
---

# Instructions

Detailed instructions for the LLM...

Skill Examples¶

Skills use two types of examples to guide LLM behavior:

1. Triggers (Frontmatter)¶

trigger_examples in the YAML frontmatter help the skill selector LLM decide WHEN to select a skill. They appear in the Triggers column of the selector prompt. Only the first 5 are shown. These can be example visitor messages or semantic trigger descriptions.

trigger_examples:
  - "We're a fintech company"    # Industry signal
  - "What ROI can I expect?"     # Results signal
  - "Do you have case studies?"  # Proof signal

Best practices for trigger_examples:

Use semantic variety (different ways to express the same intent)
Avoid language-specific phrases; prefer patterns the LLM can generalize
Put the most important trigger type first (first 5 are shown to LLM)
Match the categories described in the description field

1b. Thread Continuations (Frontmatter, `response_handling` only)¶

A description states when the visitor's message is on topic, so the selector drops the skill on follow-up turns where the thread is still open — a bare "51-250 employees" answering the sizing question you asked on a pricing turn no longer looks like a pricing message. Measured on the pricing guardrail, the skill survived only ~58% of those follow-up turns, and on the turns it dropped the answer quoted a price figure the guardrail exists to suppress.

Every response_handling row therefore gets a thread clause rendered into its description at selector-prompt build time, derived from the skill's own name. Nothing to write, and it cannot go stale when the description is edited:

| competitors | …comparisons with other solutions, OR while a competitors discussion is already
                open and the visitor is still on it. |

thread_continuations replaces the generic wording with what continuing this topic actually looks like:

thread_continuations:
  - "answering a sizing or scope question you asked on a pricing turn"
  - "narrowing what they need"
  - "pushing for a range"

| pricing | …subscription fees, OR when a pricing discussion is already open and the visitor is
            continuing it — answering a sizing or scope question you asked on a pricing turn,
            narrowing what they need, pushing for a range. A pricing thread stays open until the
            visitor changes subject. NOT for questions about… |

Best practices for thread_continuations:

Add it only where the topic genuinely spans turns; the one-line default covers the rest. The long form repeated on every guardrail row grew the selector table 20% and measurably degraded selection — identical text on every row carries no discriminative signal.
Describe the visitor's move ("answering the sizing question you asked"), not the mechanism ("was selected last turn"). The selector matches stated scope and ignores instructions about turn history.
A continuation-shaped trigger_examples entry reinforces it, e.g. "(after you asked their size on a pricing turn) we're a team of about 40".

2. Instruction Examples (Body)¶

The # examples section in the skill body shows the answer LLM HOW to use the skill correctly once selected. Use GOOD/BAD patterns:

# examples

**GOOD - Natural transition to demo:**
**Visitor:** "We're looking to automate our onboarding"
**Agent:** "Automating onboarding can reduce manual work...

👇 Feel free to connect with a Sales expert using the button below."

**BAD - Missing emoji:**
**Agent:** "Feel free to connect with our Sales team."
**Why bad:** Missing 👇 emoji at the beginning of the last sentence.

Best practices for instruction examples:

Always explain Why bad for BAD examples
Show realistic visitor/agent exchanges
Cover edge cases and common mistakes
Keep examples generic (not client-specific) in global skills
Shared prompts (main.md, global skills, intent-classifier.md, redirect/template.md) carry principles only: no client names, real product names or wording copied from a production case. When a shared rule only holds with examples that mirror the failing cases, fix those cases in the client's knowledge or client skill instead.

Skill Categories¶

Category	Purpose	Examples
`response_ending`	How to end responses	`demo_offer`, `content_gating`, `content_recommendation`, `context_gathering`, `clean_ending`
`response_handling`	Content generation	`pricing`, `competitors`, `support`
`content_enhancement`	Client-specific content extras	Client-specific: `case_studies`, `positioning`, `product_knowledge`
`system`	Auto-injected context	`knowledge_retrieval`, `visitor_profile`, `conversation_history`

Category Definition (CATEGORY.md)¶

Each category folder contains a CATEGORY.md file that configures how the category behaves in both the skill selector and the answer writer.

See backend/apps/shared_data/prompts/website-agent/skills/response_ending/CATEGORY.md for a full example.

CATEGORY.md Fields¶

Fields are consumed by two different LLMs:

Field	Consumer	Purpose
`display_name`	Answer Writer	Heading text in assembled prompt
`answer_instructions`	Answer Writer	Instructions shown before skills in main prompt
`uncertainty_guidance`	Answer Writer	Shown when multiple skills from category are selected
`selector_description`	Skill Selector	Brief description to help LLM understand category
`selector_guidance`	Skill Selector	Detailed rules for when to select skills
`order`	Both	Sort order for categories (lower = first)
`hidden_from_llm`	Both	If true, skills in category are auto-injected by code

How Categories Affect Skill Selection¶

The selector_guidance field is critical for LLM decision-making. Use it to:

Define defaults: "clean_ending is the DEFAULT for most responses"
Create decision frameworks: List conditions for each skill in priority order
Set exclusivity rules: "content_gating > context_gathering when visitor states industry"
Handle uncertainty: "When uncertain, select BOTH and let the answer LLM decide"

Always-On Skills¶

Skills with always_on: true in their YAML frontmatter are always injected into the answer prompt, regardless of whether the LLM skill selector picked them. This is useful when the skill selector cannot determine if a skill is needed (e.g., because the relevant context isn't available yet at selection time), but the answer LLM can decide at generation time.

When to use always_on:

The skill depends on RAG context that isn't available during skill selection
The answer LLM should conditionally apply the skill based on its own judgment
The skill adds a fallback behavior (e.g., offering a CTA when the bot can't answer)

Example: The book_a_call skill for augment.org uses always_on: true because the skill selector runs before RAG retrieval and cannot know if the knowledge base lacks information. The answer LLM sees the RAG context and decides whether to include the advisor booking link.

Source exclusions on always-on skills¶

An always-on skill can exclude existing retrieval sources from the answer context. Its native conditions gate both the instructions and the exclusions:

always_on: true
conditions:
  - starts_with_any: ["https://example.com/fr"]
retrieval_source_exclusions:
  - source_uri: "supabase://knowledge_faqs/other-market-faq-id"

source_uri matches an exact source in retrieved provenance. When the skill's conditions match, the source is excluded regardless of the visitor's question or resolved graph keywords. There are no query exceptions. Keep the skill's instructions consistent with that restriction, including explicit requests for another market. Selector-dependent exclusions and conditions other than native URL/language gates are rejected at load time; availability gates run too late. For market rules, use positive URL gates so a missing page URL does not imply a market.

IXChat applies these exclusions after LightRAG returns and before formatting the answer context and reported sources. It removes matching chunks and graph facts whose known sources are all excluded. Complete mixed-source graph facts retain their allowed contributions; their merged text cannot be split safely. Graph facts with truncated provenance are omitted when exclusions are active. Unattributed legacy facts cannot be assigned to a source by this feature, so this is answer-context routing rather than a separate knowledge boundary. Remaining records and their order are unchanged. Filtering does not refill removed slots; configured chunk and token budgets remain upper limits. A staff FAQ replacement preview inherits the original FAQ's source scope. This does not change the search query, tenant, index or stored knowledge; skills without this field keep their existing behavior.

Bindings use existing source URIs. Update FAQ rows in place; recreating one requires updating its binding and rerunning the market checks.

Per-Locale Skill Loading: `conditions`¶

The conditions frontmatter field lets a skill declare hard-gate predicates that must match before the skill is shown to the LLM selector at all. Skills whose conditions don't match are filtered out before the selector LLM sees the candidate list, so the wrong-region or wrong-language skill text never enters the prompt — eliminating the "lost in the middle" failure mode where RAG context drowns out branching prose inside a single skill.

Supported condition keys (evaluated in _condition_matches in backend/packages/ixskills/ixskills/skill_registry.py):

Key	Value	Match semantics
`starts_with_any`	list of URL prefixes	page URL starts with one of them at a `/` boundary (trailing-slash tolerant). Regex-free; preferred for page targeting
`does_not_start_with_any`	list of URL prefixes	page URL starts with NONE of them. Catch-all counterpart of `starts_with_any`
`page_url_matches`	regex	`re.search(value, condition_values["page_url"])` must hit. Legacy — prefer `starts_with_any`
`page_url_not_matches`	regex	`re.search(value, condition_values["page_url"])` must NOT hit. Legacy — prefer `does_not_start_with_any`
`conversation_language`	language tag	must equal `condition_values["conversation_language"]`
`*_availability` (e.g. `content_gating_availability: allowed`)	typically `allowed`	must equal resolved availability

Prefix keys take a list: conditions: [{ starts_with_any: ["/en-gb"] }]. A prefix of /en-gb matches /en-gb and /en-gb/compare but not /en-gbexit (anchored at a path-segment boundary). Path prefixes (start with /) are matched against the URL path only; absolute prefixes (start with http) against the whole URL.

Multiple conditions on one skill use AND semantics — every condition must match. A skill with no conditions: block always passes the filter (catch-all).

Always-on interaction: the same filter applies to always-on injection (get_always_on_skills in skill_registry.py), so per-locale always-on skills like regulatory_framing-uk only inject for visitors whose page_url matches. Client augmenters are filtered the same way (resolve_skill_augmentations): an augmenter whose conditions don't match is not added with its parent (e.g. sundayapp.com's sunday_us_plans stays off /fr and /en-gb).

Example layout (ibanfirst.com pricing split):

clients/ibanfirst.com/response_handling/
├── pricing-uk/SKILL.md       # conditions: [{ page_url_matches: "uk\\.ibanfirst\\.com|/en-gb/" }]
├── pricing-fr/SKILL.md       # conditions: [{ page_url_matches: "fr\\.ibanfirst\\.com|/fr/" }]
└── pricing/SKILL.md          # conditions: [{ page_url_not_matches: "uk\\.ibanfirst\\.com|/en-gb/|fr\\.ibanfirst\\.com|/fr/" }]

The EU/default catch-all keeps the canonical name (pricing) so it overrides the global pricing skill via base-name lookup. UK and FR variants use suffixed names with explicit page_url_matches. Result: each visitor sees exactly one matching skill per category, with no wrong-region content in the prompt.

Failure modes:

Bad regex: re.error at request time → condition treated as not matched → skill skipped (fail-closed). Logs at warn.
Missing field (e.g. visitor metadata lacks page_url): same as bad regex — skill skipped, implicit catch-all loads instead.
All region skills mutually exclusive (no catch-all): a visitor with an unmatched URL gets no skill in that category — design the catch-all explicitly via page_url_not_matches or by leaving conditions off the fallback skill.

Skill description guidance: don't repeat the region in the skill's description. The hard-gate filter already routes by URL — the LLM doesn't need "UK visitors only" framing because it never sees the wrong-region variants. Mirror the description across regional variants so the LLM selector picks consistently regardless of which one is shown.

See backend/packages/ixskills/tests/test_skill_registry_region_filter.py for the full matrix of region-filter tests.

Per-Path Client Skills: `clients/<host>/<segment>/`¶

A client folder may hold path folders next to its category folders. A folder that is not a category but contains category folders is a path layer:

clients/sundayapp.com/
├── response_handling/redirects/SKILL.md          # every page on the host
├── en-gb/response_handling/redirects/SKILL.md    # pages under /en-gb
└── fr/response_handling/redirects/SKILL.md       # pages under /fr

The layer loads as client sundayapp.com/en-gb; its skills keep their plain names.
Per turn, SkillRegistry.resolve_skills_client(domain_id, page_url, workspace_prefix) picks the longest layer whose /<segment> prefixes the page path (segment-safe, the starts_with_any matcher). A page under no layer, or no page, falls back to the routed workspace's canonical root path (ResolvedWorkspace.canonical_prefix), then to the host. The ixchat nodes get it from utils.node_helpers.skills_client_id.
Lookup is layer → host → global: a layer skill replaces the same-name host skill, and every other host skill is inherited. Config and the other domain_id uses are unaffected.
It keys on the URL path first, so a root that is not a routed workspace works too. The workspace fallback covers a workspace's extra roots elsewhere: welcome.payfit.com/fr-signup routes to the payfit.com/fr workspace and gets the payfit.com/fr layer. One path segment per layer.

Prefer a path folder over <skill>-<root> siblings gated with conditions:: a same-name client override hides the generic skill even when its conditions exclude the page, so a gated override leaves the excluded roots with nothing.

Config-Gated Variants: `competitors` (Tier 1 / Tier 2 / Tier 3)¶

The competitors skill resolves to one of three sibling files at lookup time, driven by per-client config fields rather than per-client SKILL.md duplication. Used for clients with ingested competitive battlecards (e.g. hyperline.co).

Files:

Path	Skill `name:`	`hidden_from_llm:`	Role
`response_handling/competitors/SKILL.md`	`competitors`	false	Tier 1. Generic defer. Unchanged from pre-IX-3166.
`response_handling/competitors-aware/SKILL.md`	`competitors_aware`	true	Tier 2. Canonical naming + roster-acknowledged defer.
`response_handling/competitors-battlecard/SKILL.md`	`competitors_battlecard`	true	Tier 3. Rep-voice fighting / battlecard-grounded.

The Tier 2/3 files use unique names + hidden_from_llm: true so they do not appear in the skill selector menu. The LLM selects competitors; skill_registry.get_skill("competitors", client_id, site_flags) returns the right variant.

Config slug (global scope): competitors — schemas/configs/global/competitors.schema.json

Field	Default	Effect
`names: list[str]`	`[]`	Canonical competitor roster. Non-empty triggers Tier 2. Used by `{db_competitors_names}` placeholder for canonical naming.
`battlecard_mode: bool`	`false`	Client-visible opt-in. Non-empty `names` + `battlecard_mode=true` → Tier 3. Requires battlecards ingested in the KB; safety fallback in the Tier 3 body defers if retrieval surfaces no chunks.

Resolution order in skill_registry.get_skill:

1. clients/<client_id>/competitors          (per-client override — wins, current 15 clients unaffected)
2. if site_flags.battlecard_mode AND site_flags.competitors_names non-empty:
       competitors_battlecard               (Tier 3)
3. if site_flags.competitors_names non-empty:
       competitors_aware                    (Tier 2)
4. competitors                              (Tier 1 fallback — current default)

Variant logic is scoped to name == "competitors". Other skill names take the unchanged code path.

Tier matrix (for clients without an override):

`names`	`battlecard_mode`	Tier	Behavior
`[]`	`false`	1	Generic defer (identical to today)
`[]`	`true`	1	Mode no-op without names
non-empty	`false`	2	Canonical naming + roster-acknowledged defer
non-empty	`true`	3	Rep-voice fighting

Wiring:

backend/packages/ixchat/ixchat/nodes/skill_applier.py:_build_skill_instructions reads resolver.competitors.{names,battlecard_mode} and passes as site_flags to aggregate_skill_instructions.
aggregate_skill_instructions(..., site_flags) propagates to registry.get_skill(name, client_id, site_flags) for each skill in the answer prompt.
db_competitors_names placeholder populated in build_prompt_parameters (answer.py) from resolver.competitors.names — used inside Tier 2/3 skill bodies for canonical-name guidance.

Tests: backend/packages/ixskills/tests/test_skill_registry_variant.py covers all four branches + per-client override precedence + variant-scoped-to-competitors safety property.

Onboarding a new battlecard client:

Ingest battlecards via rose-scrape-internal-docs.
Verify chunks retrieve via rose-chat.
Set competitors.names via the backoffice (tags input) or rose-config.
Verify Tier 2 — canonical naming, defer answers.
Flip competitors.battlecard_mode = true.
Verify Tier 3 — rep-voice grounded answers.

Declarative Skill Composition: `dependencies`, `augments`, and `blocks`¶

Three frontmatter fields let a skill declaratively shape the final skill set when it is selected:

Field	Effect
`dependencies: [skill_a, skill_b]`	When this skill is selected, add these other skills to the selection.
`augments: [skill_a]`	Link this skill with another skill. If either side is selected, the other side is included.
`blocks: [skill_x, skill_y]`	When this skill is selected, remove these other skills from the selection.

Use augments when a client-specific skill should add posture or facts to a generic policy without replacing that generic policy. Same-name client skills still override global skills; augments is the safer pattern when the shared behavior must remain active.

augments is bidirectional:

LLM selects competitors                 → client_competitor_posture is added
LLM selects client_competitor_posture   → competitors is added

There are two operating modes:

Hidden augmenter — one selector-facing direction

Use this when the generic parent should be the only skill shown to the LLM, and the client skill is purely companion instructions:

---
name: client_competitor_posture
category: response_handling
hidden_from_llm: true
augments: [competitors]
---

The LLM cannot select client_competitor_posture directly. Runtime behavior is effectively: competitors selected → client_competitor_posture added.

Visible augmenter — both selector-facing directions

Use this when the client skill has useful selector metadata that may catch client-specific phrasings the generic parent misses:

---
name: client_competitor_posture
category: response_handling
hidden_from_llm: false
augments: [competitors]
---

Now both paths are possible: if the LLM selects competitors, the client posture is added; if the LLM selects client_competitor_posture, the generic competitors policy is added. This is the right mode when the client skill's description / trigger_examples are intended to help selection, not just answer generation.

Saved pricing guidance. A client skill that augments pricing disables the workspace's saved pricing guidance (backoffice Knowledge > Pricing) by default, so a code-owned policy cannot be overridden by text saved later. Add keeps_saved_pricing_guidance: true to an augmenter that only carries behaviour and triggers: the guidance then still replaces the generic default policy and holds the facts. A same-name pricing override always disables the guidance.

blocks is the inverse of dependencies. Use it for declarative exclusivity — e.g. an inline-link skill that must suppress the standard CTA button because the link captures the conversion instead:

---
name: b2b_request
category: content_enhancement
blocks: [demo_offer]   # Inline Typeform link replaces the Enroll Now CTA
---

blocks is enforced by _rule_apply_skill_blocks in skill_applier_rules.py, which runs as a final pass after both ENDING_RULES and SYSTEM_RULES. This means a blocks directive overrides the priority-based exclusivity in the ending rules — even if demo_offer would normally win its priority race, a selected skill with blocks: [demo_offer] removes it.

The skill applier enforces blocks again after dependency and augmentation resolution. This second pass matters when composition adds the blocker itself. For example, a client can replace the generic demo CTA with its own routing instructions without adding a new runtime flag:

---
name: contact_routing
category: response_handling
augments: [demo_offer]
blocks: [demo_offer]
---

When demo_offer is selected, augments adds contact_routing; the final blocks pass then removes demo_offer. The body of contact_routing remains responsible for choosing the destination from the conversation context. This composition runs on the normal answer path and does not involve the off-topic redirect handler.

When to prefer blocks over a custom rule in skill_applier_rules.py:

The exclusion is intent-driven, not state-driven (no need to inspect context or state)
The exclusion is specific to one or two skills, not a pattern across many
The exclusion lives naturally with the skill that triggers it (easier to reason about than a far-away rule)

If the exclusion depends on runtime state (e.g. visitor profile, turn number, feature flags), write a custom rule instead.

Skill Selection Pipeline¶

The skill_selector node uses a two-phase pipeline:

LLM Selection → Post-Processing Rules → Final Skills

Phase 1: LLM Selection

The LLM selects skills based on conversation context and skill descriptions.

Phase 2: Post-Processing Rules

Mechanical rules enforce priority and exclusivity:

Ending Rules (priority-based, first match wins):
demo_offer - Priority 1, removes all other ending skills
content_gating - Priority 2, removes context_gathering/clean_ending
context_gathering - Priority 3, can add clean_ending
System Rules (all apply additively):
visitor_profile - Injected if valid company data exists
conversation_history - Injected if turn > 0
knowledge_retrieval - Always injected (RAG context)
Always-on skills - Any skill with always_on: true for the current client
Final Rules (run after ending + system rules; never short-circuit):
_rule_apply_skill_blocks - Removes any skill listed in a selected skill's blocks frontmatter

Skill Toggles¶

Skills can be enabled/disabled per client via the skill_toggles column in the agent_config table.

Default Behavior¶

Skills are ENABLED by default unless they have default_enabled: false in their SKILL.md metadata.

Configuration¶

-- Enable content_gating for a client
UPDATE agent_config
SET skill_toggles = '{"content_gating": {"enabled": true}}'
WHERE site_domain = 'example.com';

Lookup Cascade¶

1. agent_config.skill_toggles (explicit toggle)
2. SKILL_DEFAULT_ENABLED (from SKILL.md metadata)
3. True (default - skills enabled unless explicitly disabled)

Content Gating Skill¶

The content_gating skill enables email capture by offering valuable content in exchange.

How It Works¶

2-Step Flow:

Turn N (Offer): Agent answers question + offers content with 💌 emoji
Turn N+1 (Capture): Agent asks for email if visitor accepts

Availability States¶

State	Condition	Behavior
`allowed`	Enabled, no restrictions	Skill can be selected
`discouraged`	Within cooldown period	LLM discouraged from selecting
`forbidden`	Disabled OR email captured	Skill removed from selection

Cooldown Logic¶

After offering content (Turn N): - Turn N+1: Allowed (must handle visitor response) - Turn N+2 to N+3: Discouraged (cooldown active) - Turn N+4+: Allowed (cooldown passed)

Qualification Criteria¶

Optional criteria that trigger content gating offers:

UPDATE agent_config
SET qualification_criteria = '["number of support agents", "current tool"]'
WHERE site_domain = 'example.com';

When visitors mention these criteria, the LLM is more likely to offer relevant content.

No-repeat guards¶

Code, not prompt text, keeps the agent from repeating itself across turns:

Links. dialog_state_extractor records every URL the assistant shares (📎 card, markdown and bare links) as a page key (host without www., path; no scheme, query, fragment or trailing slash) in curated_content_recommended_urls. Curated cards skip those pages. Links written in a skill are not filtered, which is why catalogs belong in curated content.
Link catalogs belong in curated content. Case studies, articles and resource pages go in curated content (rose-config content add-item), not in a client skill: cards are linked once per conversation. just validate-skills (run by just dev, just deploy and the backend tests) rejects a client content_enhancement skill that links more than 3 pages; older catalogs are listed in LINK_CATALOG_ALLOWLIST until migrated.
Questions. dialog_state_extractor keeps every 👉 question of the session (asked_follow_up_questions); the answer writer's turn instruction lists them and forbids asking one again, even reworded. A required qualification question may come back, never worded exactly like the previous question. Question pressure (context_gathering_frequency) is unchanged.
Acknowledgements. The intent classifier sets acknowledgement_only when a message only acknowledges, agrees or thanks (an acknowledged open question is not answered). skill_applier then adds an "ACKNOWLEDGEMENT ONLY" turn instruction: an offer in the previous message is accepted; otherwise the reply moves to the next step without repeating any fact, benefit or link already given.
Chips. Suggested answers already shown are not offered again (see Suggested answers).

The agent can still repeat a link copied from conversation history or retrieved knowledge.

Adding New Skills¶

1. Create SKILL.md¶

mkdir -p backend/apps/shared_data/prompts/website-agent/skills/{category}/{skill_name}/

---
name: my_skill
default_enabled: true
description: When to use this skill
category: response_ending
emoji: "🎯"
always_on: false              # Set to true to bypass LLM selector and always inject
dependencies: []              # Optional: skill names auto-included with this one
augments: []                  # Optional: bidirectional pairing with another skill
blocks: []                    # Optional: skill names removed when this one is selected
---

# Instructions

Your skill instructions here...

2. Add Post-Processing Rules (if needed)¶

For ending skills with priority/exclusivity rules, update skill_selector.py:

ENDING_RULES = [
    _rule_demo_offer,
    _rule_content_gating,
    _rule_my_skill,  # Add new rule
    _rule_context_gathering,
]

3. Add Default Toggle (if disabled by default)¶

# In agent_config.py
SKILL_DEFAULT_ENABLED: dict[str, bool] = {
    "content_gating": False,
    "my_skill": False,  # Requires explicit enablement
}

4. Test¶

poetry run pytest packages/ixchat/tests/test_skill_selector.py -v

Preview a skill or prompt edit on the deployed agent¶

rose-chat reads skills and prompts from your working tree but needs every backend secret. Without them (a cloud agent session), send the edited files to the deployed agent with the rose CLI (backend/apps/rose_cli, installed in backend/.venv by ./bootstrap.py). It answers one Playground turn with those files layered over its own, from the client's real knowledge, and keeps the turn out of client analytics:

ROSE_CLI_TOKEN=... IX_API_KEY=... backend/.venv/bin/rose ask \
    --env staging --site acme.com --page-url https://acme.com/pricing "How much is it?"
backend/.venv/bin/rose ask --baseline ...   # same question, deployed skill files (--faq still applies)
backend/.venv/bin/rose ask --json ...       # full completion event, incl. `intent`

rose ask sends every .md under website-agent/ changed since your branch forked from origin/develop (--base; files changed only upstream are not sent), or the --file paths (relative to backend/apps/shared_data/prompts/): skills, intent-classifier client instructions (website-agent/intent-classifier-instructions/<domain>.md, new files included), main.md and the other agent prompts. They go in promptOverrides on /api/lightrag/query/stream, which the agent API honours only with the X-Rose-CLI-Token header matching the ROSE_CLI_TOKEN secret, and refuses with 403 otherwise, as is a wrong token; without the secret the feature is off. The API rereads the secret every 5 minutes, so a rotation takes effect within that time. The token lets its holder run preview turns on any client, so keep it in staff or routine environments only. Code changes and deleted files cannot be previewed.

The same token reads and tests a client's knowledge FAQs, without saving anything:

backend/.venv/bin/rose faqs --site acme.com   # enabled FAQs, with ids
backend/.venv/bin/rose ask --site acme.com \
    --faq "Is there a free plan?" "No, a 14-day free trial." "Is it free?"
backend/.venv/bin/rose ask --site acme.com \
    --replace-faq <faq_id> "Is there a free plan?" "No, a 14-day free trial." "Is it free?"

rose faqs calls GET /api/prompt-preview/knowledge-faqs (--workspace-id for one of the host's workspaces). --faq and --replace-faq send faqOverrides: the retriever adds each as an FAQ chunk (supabase://knowledge_faqs/preview-<n> in the knowledge used) after LightRAG, so it leads the context like a curated FAQ, and --replace-faq leaves out the chunks of the FAQ it edits. A preview proves the answer once the FAQ is retrieved, not that retrieval picks it: after applying the FAQ, ask the question again to check that. A client gets the same FAQ test from the Rose config MCP: ask_agent with a preview_faq change token.

The same token reads what a diagnosis needs, read-only:

backend/.venv/bin/rose ask -v --language fr ...           # + graph facts, intent, skills, retrieval query, why
backend/.venv/bin/rose ask --site acme.com --replay <session_id> "And the price?"  # earlier turns first
backend/.venv/bin/rose kb search --site acme.com "free trial"           # ingested chunks with the text
backend/.venv/bin/rose kb read --site acme.com https://acme.com/pricing # one document, as its chunks
backend/.venv/bin/rose conversation --site acme.com <session_id>        # a reported transcript and page
backend/.venv/bin/rose config --site acme.com ctas                      # config that shapes answers
backend/.venv/bin/rose eval --site acme.com --list                      # the client's Evaluations set
backend/.venv/bin/rose eval --site acme.com --run [--baseline] --json   # each question, fresh, with your files

--replay asks a reported conversation's earlier visitor turns in a new preview session (its page and language), then the question: a preview turn is stored under its session id, so the agent API refuses any session id that does not start with prompt-preview_. rose eval is the client's own Evaluations set (backoffice dataset_questions, approved rows only), each question asked as a fresh Playground turn in its language, printed next to its expected answer; the caller judges. It is not rose-eval, the staff eval on the cross-client Langfuse datasets. rose kb reads the tenant's LightRAG text chunks (what retrieval sees, environment-specific: MongoDB is isolated per environment), bounded and literal. rose conversation reads the stored conversations/messages rows (Supabase, shared). rose config serves only the slugs that shape answers (PREVIEW_CONFIG_SLUGS in routes/prompt_preview.py), never integrations.

The rose CLI and the client MCP (backend/apps/rose_config_mcp) share one implementation, backend/packages/ixrose_client: a rule changes there, never in one of the two layers.

Prompt & Skill Development Workflow - How a prompt or skill edit ships
IXChat Package - Full chatbot architecture