CLI Langfuse¶

The cli-langfuse command provides unified access to Langfuse functionality: prompt synchronization, dataset management, and trace inspection.

Installation¶

The CLI tool is available after activating the Poetry shell in the backend directory:

cd backend
poetry shell

Once in the Poetry shell, the cli-langfuse command is available.

Required Environment Variables¶

Ensure these environment variables are set (typically via .env):

Variable	Description
`LANGFUSE_SECRET_KEY`	Langfuse API secret key
`LANGFUSE_PUBLIC_KEY`	Langfuse API public key
`LANGFUSE_HOST`	Langfuse API host URL

Command Structure¶

cli-langfuse <subcommand> <command> [OPTIONS]

Subcommands¶

Subcommand	Description
`prompt`	Manage prompt synchronization between local files and Langfuse
`dataset`	Manage Langfuse datasets
`trace`	Inspect and manage Langfuse traces

Prompt Commands¶

Manage bidirectional synchronization between local Markdown files and Langfuse prompts/skills.

Commands¶

Command	Description
`pull`	Fetch prompts from Langfuse to local files
`push`	Upload local changes to Langfuse
`status`	Check sync status of tracked items
`diff`	Compare local files against remote versions
`list`	List all prompts with a given prefix
`bump`	Apply a label to latest versions without creating new versions
`promote`	Promote prompts between labels (e.g., development → production)

pull¶

Fetch prompts from Langfuse and save them as local Markdown files.

cli-langfuse prompt pull [OPTIONS]

Option	Default	Description
`--prefix`	`website-agent/`	Prefix filter for prompts
`--label`	`latest`	Label to pull (development, production, latest)
`--force`	`false`	Overwrite local changes without conflict detection
`--slug`	-	Specific slug(s) to pull (can be repeated)

push¶

Upload local Markdown files to Langfuse as prompts.

cli-langfuse prompt push [OPTIONS]

Option	Default	Description
`--prefix`	`website-agent/`	Prefix for prompt slugs
`--label`	`development`	Label to assign to pushed prompts
`--force`	`false`	Push even if remote has newer version
`--yes`	`false`	Skip confirmation prompt

status¶

Check the synchronization status of all tracked items.

cli-langfuse prompt status [OPTIONS]

Option	Default	Description
`--prefix`	`website-agent/`	Prefix filter for status check

Status Values:

Status	Meaning
`SYNCED`	Local matches Langfuse
`LOCAL_MODIFIED`	Local has unpushed changes
`REMOTE_MODIFIED`	Langfuse has unpulled changes
`CONFLICT`	Both local and remote changed
`NEW_LOCAL`	Exists locally, not in Langfuse
`NEW_REMOTE`	Exists in Langfuse, not locally

diff¶

Show differences between local files and Langfuse versions.

cli-langfuse prompt diff [OPTIONS]

Option	Default	Description
`--prefix`	`website-agent/`	Prefix filter
`--all`	`false`	Show all items, including synced ones

list¶

List all prompts in Langfuse matching the prefix.

cli-langfuse prompt list [OPTIONS]

Option	Default	Description
`--prefix`	`website-agent/`	Prefix filter for listing

bump¶

Apply a label to the latest versions of all prompts/skills without creating new versions.

cli-langfuse prompt bump [OPTIONS]

Option	Default	Description
`--prefix`	`website-agent/`	Prefix filter for prompts
`--label`	`development`	Label to apply to latest versions
`--yes`	`false`	Skip confirmation prompt

Typical Usage:

# After pushing changes, apply development label for testing
cli-langfuse prompt bump --label development

# After validation, apply production label
cli-langfuse prompt bump --label production

promote¶

Promote prompts from one label to another.

cli-langfuse prompt promote --from <LABEL> --to <LABEL> [OPTIONS]

Option	Default	Description
`--from`	(required)	Source label
`--to`	(required)	Target label
`--prefix`	`website-agent/`	Prefix filter

Dataset Commands¶

Manage Langfuse datasets for evaluation and testing.

Commands¶

Command	Description
`list`	List all datasets
`items`	List items in a dataset
`create`	Create a new dataset
`add-item`	Add an item to a dataset

list¶

List all datasets in Langfuse.

cli-langfuse dataset list

items¶

List items in a specific dataset.

cli-langfuse dataset items <DATASET_NAME>

create¶

Create a new dataset.

cli-langfuse dataset create <NAME> [OPTIONS]

Option	Default	Description
`--description`	-	Dataset description

add-item¶

Add an item to a dataset.

cli-langfuse dataset add-item <DATASET_NAME> [OPTIONS]

Option	Default	Description
`--input`	(required)	Input JSON
`--expected-output`	-	Expected output JSON
`--metadata`	-	Metadata JSON

Trace Commands¶

Inspect and manage Langfuse traces.

Commands¶

Command	Description
`list`	List recent traces
`get`	Get trace details
`add-to-dataset`	Add a trace to a dataset

list¶

List recent traces.

cli-langfuse trace list [OPTIONS]

Option	Default	Description
`--limit`	`10`	Number of traces to show

get¶

Get detailed information about a specific trace.

cli-langfuse trace get <TRACE_ID>

add-to-dataset¶

Add a trace to a dataset for evaluation.

cli-langfuse trace add-to-dataset <TRACE_ID> <DATASET_NAME>

Prompt Sync Examples¶

# Pull all prompts from Langfuse
cli-langfuse prompt pull

# Pull specific prompts
cli-langfuse prompt pull --slug skills/pricing --slug main

# Push all local changes
cli-langfuse prompt push --yes

# Check sync status
cli-langfuse prompt status

# View differences
cli-langfuse prompt diff

# Promote from development to production
cli-langfuse prompt promote --from development --to production

# Bump latest versions to development label
cli-langfuse prompt bump --label development

# Bump to production after validation
cli-langfuse prompt bump --label production --yes

# Work with a different prefix
cli-langfuse prompt pull --prefix rose-internal/

Dataset & Trace Examples¶

# List all datasets
cli-langfuse dataset list

# Create a new dataset
cli-langfuse dataset create "evaluation-set-v1" --description "Q1 2026 evaluation"

# List items in a dataset
cli-langfuse dataset items "evaluation-set-v1"

# List recent traces
cli-langfuse trace list --limit 20

# Get trace details
cli-langfuse trace get "trace-abc123"

# Add a trace to a dataset
cli-langfuse trace add-to-dataset "trace-abc123" "evaluation-set-v1"

Key Concepts¶

Frontmatter Handling¶

The tool handles frontmatter differently based on file type:

Skills (files in /skills/ or named SKILL.md):

Frontmatter remains in the content sent to Langfuse
Only langfuse_version is stripped before push
This allows skills to be self-contained with their metadata

Prompts (all other files):

Entire frontmatter is extracted and stored in Langfuse's config field
Content sent to Langfuse contains only the prompt text
Frontmatter is restored on pull

Version Tracking¶

Each file includes a langfuse_version field in its frontmatter:

---
langfuse_version: 42
type: skill
name: Pricing
---

This version number:

Is automatically updated after successful push
Enables conflict detection between local and remote changes
Is stripped before sending content to Langfuse

Labels¶

Langfuse supports labels for versioning:

Label	Purpose
`development`	Active development (default for push)
`production`	Production-ready prompts
`latest`	Most recent version regardless of label

Tags¶

Tags are automatically derived from metadata and file paths:

type:skill, type:system_prompt, type:rag, type:test
category:<category> from frontmatter
client:<client_id> from frontmatter or path pattern clients/<id>/

File Structure¶

Local Files¶

Prompts and skills are stored in:

backend/apps/shared_data/
├── website-agent/           # Default prefix
│   ├── main.md              # Main system prompt
│   ├── skills/              # Skill definitions
│   │   ├── pricing/
│   │   │   └── SKILL.md
│   │   └── demo_offer/
│   │       └── SKILL.md
│   └── clients/             # Client-specific overrides
│       └── abtasty.com/
│           └── skills/
│               └── pricing/
│                   └── SKILL.md
└── rose-internal/           # Alternative prefix
    └── ...

Metadata File¶

Sync state is tracked in .lf-sync/metadata.json:

{
  "default_prefix": "website-agent/",
  "default_label": "development",
  "items": {
    "website-agent/main": {
      "langfuse_version": 42,
      "label": "development"
    }
  }
}

Conflict Resolution¶

When both local and remote have changes, the tool inserts Git-style conflict markers:

<<<<<<< LOCAL
Your local content here
=======
Remote content from Langfuse
>>>>>>> REMOTE

Resolving Conflicts¶

Open the file with conflict markers
Choose the correct content (or merge manually)
Remove the conflict markers (<<<<<<<, =======, >>>>>>>)
Run cli-langfuse prompt push to upload the resolved version

Blocked Operations

Files with unresolved conflict markers cannot be pushed. Resolve all conflicts first.

Workflow Integration¶

The typical development workflow:

flowchart TD A[Edit local files] --> B{Ready to sync?} B -->|Yes| C[cli-langfuse prompt status] C --> D{Status?} D -->|LOCAL_MODIFIED| E[cli-langfuse prompt push] D -->|REMOTE_MODIFIED| F[cli-langfuse prompt pull] D -->|CONFLICT| G[cli-langfuse prompt pull] G --> H[Resolve conflicts] H --> E E --> I[Changes in Langfuse] F --> J[Local files updated]

Why Use This Tool?¶

Version Control: Track prompt changes in Git alongside code
Collaboration: Multiple developers can work on prompts with conflict detection
Environments: Use labels to manage development vs production prompts
Auditability: Full history of prompt changes in both Git and Langfuse
Local Development: Edit prompts in your IDE with syntax highlighting

Troubleshooting¶

"Conflict detected"¶

The file has been modified both locally and in Langfuse:

# Pull remote changes (creates conflict markers)
cli-langfuse prompt pull

# Manually resolve conflicts in the file
# Remove <<<<<<< LOCAL, =======, >>>>>>> REMOTE markers

# Push resolved version
cli-langfuse prompt push

"Version mismatch - remote is newer"¶

Someone else pushed changes to Langfuse:

# Pull the latest version first
cli-langfuse prompt pull

# Then push your changes
cli-langfuse prompt push

"Authentication failed"¶

Check your environment variables:

# Verify variables are set
echo $LANGFUSE_SECRET_KEY
echo $LANGFUSE_PUBLIC_KEY
echo $LANGFUSE_HOST

# Reload environment if needed
source .env

"File has unresolved conflict markers"¶

The file contains <<<<<<<, =======, or >>>>>>> markers:

Open the file and search for these markers
Resolve the conflict by choosing the correct content
Remove all conflict markers
Save and retry the push

"Prompt not found in Langfuse"¶

The prompt may not exist yet:

# For new local files, just push them
cli-langfuse prompt push

# Verify it was created
cli-langfuse prompt list