Skip to content

Prompt & Skill Development Workflow

Prompts and skills live in the repo, under backend/apps/shared_data/prompts/. Those files are the runtime source in every environment, production included. Nothing reads prompts back from Langfuse at runtime.

An edit reaches production the same way code does:

flowchart LR A[Edit the .md file] --> B[Test locally] B --> C{Tests pass?} C -->|No| A C -->|Yes| D[Commit] D --> E[PR merged to develop] E --> F[Deploy: baked into the API image]

There is no prompt label to flip, and no push to make. rose-langfuse prompt push maintains a backup mirror in Langfuse for audit and recovery; it is never a deployment step and never changes what production serves.

Local Development

Start the Development Server

cd backend
just dev development

Prompts and skills are read from the repo:

backend/apps/shared_data/prompts/website-agent/
├── main.md                          # Main system prompt
├── skills/
│   ├── response_handling/
│   │   └── pricing/
│   │       └── SKILL.md
│   └── clients/
│       └── abtasty.com/
│           └── skills/
│               └── pricing/
│                   └── SKILL.md
└── ...

rose-chat reads the same files from your working tree, so a run reflects an uncommitted edit immediately.

AI-Assisted Development

Because prompts and skills are plain Markdown files in the codebase, you can use AI assistants (like Claude Code) to:

  1. Modify prompts and code together - Make coherent changes across the entire system
  2. Refactor skill definitions - Update SKILL.md files while adjusting related Python code
  3. Add new skills - Create new skill files with proper structure and metadata
  4. Test changes - Run unit tests to verify behavior before committing

Testing

cd backend
poetry run pytest -m unit packages/ixskills/

Tests verify skill selection logic, prompt rendering, metadata parsing, and the integration between skills and the router.

For behavioural checks against a real client, run rose-chat or rose-eval; both read the prompts from the repo.

Shipping a Change

# 1. Edit the prompt or SKILL.md file
# 2. Run the tests
cd backend
poetry run pytest -m unit packages/ixskills/

# 3. Commit prompt/skill changes together with the code that depends on them
git add backend/apps/shared_data/prompts/ backend/packages/ixskills/
git commit -m "feat: add enterprise pricing handling to pricing skill"

# 4. Open a PR against develop. On merge, the change deploys with the API image.

The Langfuse Backup Mirror

.github/workflows/deploy-prompts.yml pushes the repo prompt files to Langfuse on merge to develop and on release. This is a one-way mirror: repo → Langfuse. It gives an audit trail and a recovery point; it has no runtime effect.

You rarely need to run it by hand. When you do:

rose-langfuse prompt status   # what the mirror is missing
rose-langfuse prompt diff     # the actual differences
rose-langfuse prompt push     # refresh the mirror

rose-langfuse prompt pull goes the other way — it restores repo files from the mirror. It is a recovery tool, not part of any normal workflow.

langfuse_version frontmatter

Every mirrored file carries a langfuse_version counter in its frontmatter:

---
langfuse_version: 42
type: skill
name: Pricing
---

It is a backup-mirror version counter, bumped by push so the mirror can detect drift. It selects nothing at runtime and changing it changes no behaviour.

Troubleshooting

A prompt or skill edit isn't taking effect

In order of likelihood:

  1. Not deployed — the file only reaches production through commit → merge → deploy. Check that the change is on the deployed commit.
  2. Selector miss — the skill was not selected for that turn. Check the skill's description/triggers, and the selector's reasoning in the trace.
  3. Load error — the file failed to parse (invalid YAML frontmatter, missing required field). A missing prompt file raises at load time; check the service logs.
  4. Retrieved content overrides it — the answer came from the knowledge base rather than the skill. Check the retrieved chunks in the trace.

The Langfuse mirror is never a cause: nothing reads from it.

Tests failing after skill changes

  1. Check skill metadata is valid YAML
  2. Verify skill category matches expected values
  3. Run the specific test with verbose output:
poetry run pytest -v packages/ixskills/tests/test_skill_selector.py