Skip to content

CLI Chat

The cli-chat command is a CLI tool for rapid chatbot iteration. It calls the chatbot service directly (bypassing HTTP), maintains session state via Redis, and logs to files for easy inspection.

Installation

The CLI tool is available after activating the Poetry shell in the backend directory:

cd backend
poetry shell

Once in the Poetry shell, the cli-chat command is available.

Usage

cli-chat "message" --site <site-name> [OPTIONS]

Modes

Mode Usage Description
Single query cli-chat "message" --site foo.com Send one message, get a response
Continue cli-chat "message" --continue Continue the last conversation
Interactive cli-chat --interactive --site foo.com REPL mode for multi-turn testing
Info cli-chat --info Show last session info

Options

Option Short Default Description
message - - The message to send (positional, optional in interactive/info mode)
--site -s - Site name (required for new conversations)
--continue -c false Continue the last conversation
--session - - Use a specific session ID
--interactive -i false Enter interactive REPL mode
--info - false Show last session info
--env - development Environment: development, staging, production, test
--trace - false Enable Langfuse tracing
--add-to-dataset - - Add trace to dataset after chat (requires --trace). Defaults to main-dataset if no name given.
--dataset-type - e2e Type of dataset item to create: e2e, intent-classifier, skill-selector
--feature - - Override feature flags (repeatable, e.g. --feature enable_new_agentic_system=true)
--click-cta - - Simulate a CTA button click to trigger booking flow
--debug - false Enable debug logging to console

Examples

New conversation

cli-chat "What does Rose do?" --site mayday.fr

Continue the last conversation

cli-chat "Tell me more about pricing" --continue

Interactive REPL mode

cli-chat --interactive --site mayday.fr

In interactive mode, type messages at the prompt. Special commands:

  • quit / exit / q — End the session
  • new — Start a new session (same site)

Force agentic system via feature flags

# Force NEW agentic system
cli-chat "Hello" --site mayday.fr --feature enable_new_agentic_system=true

# Force LEGACY system
cli-chat "Hello" --site mayday.fr --feature enable_new_agentic_system=false

Simulate CTA click (booking flow)

cli-chat "I want a demo" --site mayday.fr --click-cta 1
cli-chat "john@example.com" --continue

Enable Langfuse tracing

cli-chat "Can rose help with pricing?" --site userose.ai --trace

Add conversation to a dataset

# Add to default dataset (main-dataset)
cli-chat "What does your product do?" --site mayday.fr --trace --add-to-dataset

# Add to a named dataset
cli-chat "What does your product do?" --site mayday.fr --trace --add-to-dataset mayday.fr-v2

# Add as intent-classifier item
cli-chat "I need help" --site mayday.fr --trace --add-to-dataset intent-classifier --dataset-type intent-classifier

# Add as skill-selector item
cli-chat "Show me pricing" --site mayday.fr --trace --add-to-dataset skill-selector --dataset-type skill-selector

Session Management

cli-chat persists session state to backend/.chat_session.json. This enables --continue to resume the last conversation.

Session state includes:

  • session_id: Unique ID (cli_{site}_{timestamp}_{random})
  • site_name: The site for this session
  • created_at: When the session was created
  • turn_count: Number of messages exchanged
  • last_message_at: Timestamp of the last message

Session Lifecycle

  1. --site foo.com — Creates a new session
  2. --continue — Reuses the last session (same site and session ID)
  3. --session <id> --site foo.com — Uses a specific session ID
  4. In interactive mode, new — Creates a fresh session

Dataset Population

cli-chat can populate Langfuse datasets for use with cli-eval. This replaces the old shell-script-based dataset creation.

Workflow

  1. Run a conversation with --trace to create a Langfuse trace
  2. Add --add-to-dataset <name> to extract the trace into a dataset item
  3. The tool waits for the trace to be available in Langfuse (retries up to 6 times)
  4. The trace is converted to evaluator-compatible format based on --dataset-type

Dataset Types

Type Format Used By
e2e (default) {"query": "..."} / {"response": "..."} E2EAPIEvaluator
intent-classifier Extracts intent from trace ClassificationEvaluator
skill-selector Extracts selected skills from trace MultiLabelEvaluator

Environment

By default, cli-chat runs in development environment, which:

  • Loads .env.development
  • Sets IX_PROMPT_SOURCE=local (loads prompts from local files instead of Langfuse)
  • Sets IX_IS_LOCAL=true

Use --env to override:

cli-chat "Hello" --site mayday.fr --env staging

Log Files

All sessions are logged to backend/logs/:

  • chat-{session_id}.log — Full log for each session
  • chat-latest.log — Symlink to the most recent log file

The log file path is displayed at the start of each run.

Exit Codes

Code Meaning
0 Success
1 Error (query failure, missing arguments)
130 Cancelled by user (Ctrl+C)