Skip to content

Latest commit

 

History

History
1843 lines (1277 loc) · 33.9 KB

File metadata and controls

1843 lines (1277 loc) · 33.9 KB

CLI Reference

Complete reference for the Phlo command-line interface.

Installation

# Using pip
pip install -e .

# Using uv (recommended)
uv pip install -e .

Verify:

phlo --version

Global Options

phlo --help              # Show help
phlo --version           # Show version

Command Overview

Core Commands:

phlo init                # Initialize new project
phlo services            # Manage infrastructure services
phlo plugin              # Manage plugins
phlo dev                 # Start development server
phlo test                # Run tests
phlo config              # Configuration management
phlo env                 # Environment exports

Workflow Commands:

phlo workflow create     # Create new workflow
phlo validate-workflow   # Validate workflow configuration

Asset Commands:

phlo materialize         # Materialize assets
phlo backfill            # Backfill partitioned assets
phlo status              # Show asset status
phlo logs                # Query structured logs

Data Catalog Commands:

phlo branch              # Manage Nessie branches
phlo catalog             # Catalog operations (tables, describe, history)
phlo lineage             # Asset lineage (show, export)

Quality Commands:

phlo schema              # Manage Pandera schemas
phlo validate-schema     # Validate Pandera schemas
phlo schema-migrate      # Diff, plan, apply, and scaffold table schema migrations
phlo migrate             # Run declarative data migration specs

Optional Plugin Commands (requires installation):

phlo postgrest           # PostgREST management
phlo hasura              # Hasura GraphQL management
phlo publishing          # dbt publishing layer
phlo metrics             # Metrics summary (built-in)
phlo alerts              # Alerting rules
phlo openmetadata        # OpenMetadata catalog

Services Commands

Manage Docker infrastructure services.

Services are provided by installed service packages (e.g., phlo-dagster, phlo-trino). Install core services with:

uv pip install -e ".[core-services]"

Optional services can be added later:

phlo plugin install phlo-observatory

phlo services init

Initialize infrastructure directory and configuration.

phlo services init [OPTIONS]

What it does:

  • Creates .phlo/ directory
  • Generates Docker Compose configurations
  • Sets up network and volume definitions
  • Creates phlo.yaml config file

Options:

--force              # Overwrite existing configuration
--name NAME          # Project name (default: directory name)
--dev                # Development mode: mount local phlo source
--no-dev             # Explicitly disable dev mode
--phlo-source PATH   # Path to phlo repo or src/phlo for dev mode

Examples:

# Basic initialization
phlo services init

# Force overwrite existing
phlo services init --force

# With custom project name
phlo services init --name my-lakehouse

# Development mode with local source
phlo services init --dev --phlo-source /path/to/phlo

phlo services start

Start all infrastructure services.

phlo services start [OPTIONS]

Options:

--native                 # Run native dev services (phlo-api, Observatory) as subprocesses
--profile PROFILE        # Additional service profiles
--service SERVICE        # Start only specific service(s)
--detach, -d             # Run in background
--build                  # Rebuild containers before starting

Profiles:

  • observability: Prometheus, Grafana, Loki
  • api: PostgREST, Hasura
  • catalog: OpenMetadata

Examples:

# Start core services
phlo services start

# Start with observability
phlo services start --profile observability

# Run Observatory/phlo-api without Docker (useful on ARM Macs)
phlo services start --native

# Develop the phlo framework from a local monorepo (Dagster container installs editable from source)
phlo services init --force --dev --phlo-source /path/to/phlo
phlo services start

# Multiple profiles
phlo services start --profile observability --profile api

# Rebuild and start
phlo services start --build

Validation behavior:

  • --profile values are validated before Docker commands are executed.
  • Unknown profiles fail fast with a ClickException that includes valid options.
  • --service targets include required dependencies and matching *-setup bootstrap services.
phlo services start --profile not-a-profile
# Error: Invalid profile: not-a-profile. Valid profile options: api, observability

Services started:

  • PostgreSQL (port 10000)
  • MinIO (ports 10001-10002)
  • Nessie (port 10003)
  • Trino (port 10005)
  • Dagster webserver (port 10006)
  • Dagster daemon

phlo services stop

Stop all running services.

phlo services stop [OPTIONS]

Options:

--volumes, -v        # Remove volumes (deletes all data)
--profile PROFILE    # Stop only services in specified profile
--service SERVICE    # Stop specific service(s)
--native             # Also stop native subprocess services

Examples:

# Stop all services (preserve data)
phlo services stop

# Stop and delete all data
phlo services stop --volumes

# Stop specific profile
phlo services stop --profile observability

# Stop specific services
phlo services stop --service postgres,minio

phlo services list

List available services with status.

phlo services list [OPTIONS]

Options:

--all                # Show all services including optional
--json               # Output as JSON

Examples:

phlo services list
phlo services list --all
phlo services list --json

Error behavior:

  • Invalid phlo.yaml is surfaced with a targeted message:
phlo services list --json
# Error: Failed to read /path/to/phlo.yaml. Check YAML syntax and file permissions, then retry.
  • Discovery failures include next-step diagnostics:
phlo services list --json
# Error: Failed to discover services. Verify service plugins are installed and run `phlo plugins list` for diagnostics.

phlo services ports

Show port mappings for all services.

phlo services ports [OPTIONS]

Options:

--json               # Output as JSON
--all                # Include stopped services with defaults

Source values:

  • env: Port set via compose env resolution, including shell vars, phlo.yaml env:, or .phlo/.env*
  • compose: Literal host port from the compose mapping
  • default: Using the service.yaml default
  • runtime: Live port detected from the running container

Examples:

phlo services ports
phlo services ports --json
phlo services ports --all

Output:

Service            Host Port   Container Port   Source     Status
----------------------------------------------------------------------
postgres           10000       5432             env       Running
minio-api          10001       9000             env       Running
dagster-webserver  10006       3000             env       Running
clickhouse-http    8123        8123             default   Stopped

Conflict detection:

When two services resolve to the same host port, both rows are highlighted with a prefix:

⚠ Port conflict: trino and hasura both map to host port 8080

phlo services add

Add an optional service to the project.

phlo services add SERVICE_NAME [OPTIONS]

Options:

--no-start           # Don't start the service after adding

Examples:

phlo services add prometheus
phlo services add grafana --no-start

State semantics:

  • add ensures the service is present in services.enabled.
  • add removes that service from services.disabled if present.
  • If already enabled and not disabled, command exits without changing config.
# Before
services:
  enabled: []
  disabled:
    - prometheus
phlo services add prometheus --no-start
# After
services:
  enabled:
    - prometheus
  disabled: []

phlo services remove

Remove a service from the project.

phlo services remove SERVICE_NAME [OPTIONS]

Options:

--keep-running       # Don't stop the service

Examples:

phlo services remove prometheus
phlo services remove grafana --keep-running

State semantics:

  • remove removes the service from services.enabled.
  • remove adds the service to services.disabled.
  • By default it stops the service first; --keep-running skips the stop step.
# Before
services:
  enabled:
    - prometheus
  disabled: []
phlo services remove prometheus --keep-running
# After
services:
  enabled: []
  disabled:
    - prometheus

phlo services reset

Reset infrastructure by stopping services and deleting volumes.

phlo services reset [OPTIONS]

Options:

--service SERVICE    # Reset only specific service(s)
-y, --yes            # Skip confirmation

Examples:

phlo services reset
phlo services reset --service postgres
phlo services reset -y

phlo services restart

Restart services (stop + start).

phlo services restart [OPTIONS]

Options:

--build              # Rebuild containers before starting
--profile PROFILE    # Restart services in profile
--service SERVICE    # Restart specific service(s)
--dev                # Enable dev mode when restarting

Examples:

phlo services restart
phlo services restart --build
phlo services restart --service dagster

phlo services status

Show status of all services.

phlo services status

Output:

SERVICE              STATUS    PORTS
postgres             running   10000
minio                running   10001-10002
nessie               running   10003
trino                running   10005
dagster-webserver    running   10006
dagster-daemon       running

phlo services logs

View service logs.

phlo services logs [OPTIONS] [SERVICE]

Options:

--follow, -f         # Follow log output
--tail N             # Show last N lines

Examples:

# All logs
phlo services logs

# Follow specific service
phlo services logs -f dagster-webserver

# Last 100 lines
phlo services logs --tail 100 trino

Plugin Commands

Manage Phlo plugins for extending functionality.

phlo plugin list

List all installed plugins.

phlo plugin list [OPTIONS]

Options:

--type TYPE          # Filter by plugin type (sources, quality, transforms, services, hooks, assets, resources, orchestrators, catalogs, all)
--all                # Include registry plugins in output
--json               # Output as JSON

Examples:

# List all plugins
phlo plugin list

# List only source connectors
phlo plugin list --type sources

# List hook plugins
phlo plugin list --type hooks

# JSON output
phlo plugin list --json

Output:

Services:
  NAME              VERSION    DESCRIPTION
  dagster           0.1.0      Dagster orchestration engine
  postgres          0.1.0      PostgreSQL database
  trino             0.1.0      Distributed SQL query engine

Sources:
  NAME              VERSION    DESCRIPTION
  rest_api          1.0.0      REST API connector
  jsonplaceholder   1.0.0      JSONPlaceholder example source

Quality Checks:
  NAME              VERSION    DESCRIPTION
  null_check        1.0.0      Null value validation
  threshold_check   1.0.0      Threshold validation

phlo plugin search

Search available plugins in the registry.

phlo plugin search [QUERY] [OPTIONS]

Options:

--type TYPE          # Filter by plugin type
--json               # Output as JSON
--tag TAG            # Filter by tag

Examples:

# Search for PostgreSQL-related plugins
phlo plugin search postgres

# Search for quality check plugins
phlo plugin search --type quality

# Search for hook plugins
phlo plugin search --type hooks

# Search by tag
phlo plugin search --tag observability

phlo plugin install

Install a plugin from the registry.

phlo plugin install PLUGIN_NAME [OPTIONS]

Options: none

Examples:

# Install a plugin
phlo plugin install phlo-superset

# Install specific version
phlo plugin install phlo-superset==0.2.0

phlo plugin info

Show detailed information about a plugin.

phlo plugin info PLUGIN_NAME [OPTIONS]

Options:

--type TYPE          # Plugin type (auto-detected if omitted)
--json               # Output as JSON

Examples:

# Get plugin info (auto-detect type)
phlo plugin info dagster

# Specify type
phlo plugin info rest_api --type sources

# JSON output
phlo plugin info dagster --json

Output:

dagster
Type: services
Version: 0.1.0
Author: Phlo Team
Description: Dagster orchestration engine for workflow management
License: MIT
Homepage: https://github.com/iamgp/phlo
Tags: orchestration, core, service

Service Details:
  Container: dagster-webserver
  Ports: 10006
  Dependencies: postgres

phlo plugin update

Update plugins to latest versions.

phlo plugin update [OPTIONS]

Options:

--dry-run            # Show available updates without applying
--json               # Output available updates as JSON

Examples:

# Update available plugins
phlo plugin update

# Check for updates without installing
phlo plugin update --dry-run

# JSON output
phlo plugin update --json

phlo plugin create

Create a new plugin scaffold (for plugin development).

phlo plugin create PLUGIN_NAME --type TYPE [OPTIONS]

Options:

--type TYPE          # Plugin type: sources, quality, transforms, services, hooks, catalogs, assets, resources, orchestrators
--path PATH          # Custom output path

Examples:

# Create source connector plugin
phlo plugin create my-api-source --type source

# Create quality check plugin
phlo plugin create my-validation --type quality

# Create hook plugin
phlo plugin create my-hooks --type hook

# Create with custom path
phlo plugin create my-plugin --type source --path ./plugins/my-plugin

Creates:

phlo-plugin-my-api-source/
├── pyproject.toml           # Package config with entry points
├── README.md                # Documentation
├── src/
│   └── phlo_my_api_source/
│       ├── __init__.py
│       └── plugin.py        # Plugin implementation
└── tests/
    └── test_plugin.py       # Test suite

phlo plugin check

Validate installed plugins.

phlo plugin check [OPTIONS]

Options:

--json               # Output as JSON

Examples:

# Validate all plugins
phlo plugin check

# JSON output
phlo plugin check --json

Project Commands

phlo init

Initialize a new Phlo project.

phlo init [PROJECT_NAME] [OPTIONS]

Options:

--template TEMPLATE      # Project template (default: basic)
--force                  # Initialize in non-empty directory

Templates:

  • basic: Ingestion + dbt transforms (requires phlo-dbt)
  • minimal: Minimal project structure (no transforms)

Example:

phlo init my-lakehouse --template basic
cd my-lakehouse

Creates:

my-lakehouse/
├── .env.example          # Local secrets template (.phlo/.env.local)
├── workflows/
│   ├── ingestion/
│   ├── schemas/
│   └── transforms/       # basic template only
│       └── dbt/
├── tests/
└── phlo.yaml

phlo dev

Start Dagster development server.

phlo dev [OPTIONS]

Options:

--port PORT          # Port for webserver (default: 3000)
--host HOST          # Host to bind (default: 127.0.0.1)
--workflows-path PATH # Path to workflows directory

Example:

phlo dev --port 3000

Opens Dagster UI at http://localhost:3000

Workflow Commands

phlo workflow create

Interactive workflow creation wizard.

phlo workflow create [OPTIONS]

Options:

--type TYPE          # Workflow type (currently: ingestion)
--domain DOMAIN      # Domain/namespace (e.g., api, files)
--table TABLE        # Table name
--unique-key KEY     # Unique key column
--cron CRON          # Cron schedule expression
--api-base-url URL   # REST API base URL (optional)
--field NAME:TYPE    # Additional schema field (repeatable)

Interactive prompts:

  1. Workflow type (ingestion/quality/transform)
  2. Domain name
  3. Table name
  4. Unique key column
  5. Schedule (cron expression)
  6. API base URL (optional)

Example (interactive):

phlo workflow create

Example (non-interactive):

phlo workflow create \
  --type ingestion \
  --domain github \
  --table events \
  --unique-key id

Creates:

workflows/
├── ingestion/
│   └── github/
│       └── events.py
└── schemas/
    └── github.py

Asset Commands

phlo materialize

Materialize Dagster assets.

phlo materialize ASSET_NAME [OPTIONS]

Options:

--select SELECTOR        # Asset selection query
--partition PARTITION    # Specific partition to materialize
--no-contract-refresh    # Skip automatic schema-contract refresh
--dry-run                # Show command without executing

Selection Syntax:

asset_name               # Single asset
asset_name+              # Asset and downstream
+asset_name              # Asset and upstream
asset_name*              # Asset and all dependencies
tag:group_name           # All assets with tag
*                        # All assets

Examples:

# Single asset
phlo materialize dlt_glucose_entries

# Asset and downstream
phlo materialize dlt_glucose_entries+

# Specific partition
phlo materialize dlt_glucose_entries --partition 2025-01-15

# By tag
phlo materialize --select "tag:nightscout"

# Dry run
phlo materialize dlt_glucose_entries --dry-run

# Skip automatic contract refresh
phlo materialize dlt_glucose_entries --no-contract-refresh

Testing Commands

Schema Migration Commands

phlo schema-migrate export-contract

Export a Phlo contract snapshot for a table.

phlo schema-migrate export-contract TABLE_NAME [OPTIONS]

Options:

--schema-class NAME      # Explicit Pandera schema class name
--output PATH            # Custom contract output path
--force                  # Overwrite existing output file

Examples:

phlo schema-migrate export-contract warehouse.customers
phlo schema-migrate export-contract warehouse.customers --output .phlo/contracts/customers.json

phlo schema-migrate scaffold-yaml

Generate migration scaffold YAML from a previously exported contract.

phlo schema-migrate scaffold-yaml TABLE_NAME [OPTIONS]

Options:

--from-contract PATH     # Contract path (default: .phlo/contracts/<table>.json)
--output PATH            # YAML output path (default: .phlo/migrations/<table>.yaml)
--force                  # Overwrite existing output file

Examples:

phlo schema-migrate scaffold-yaml warehouse.customers
phlo schema-migrate scaffold-yaml warehouse.customers --from-contract .phlo/contracts/customers.json

phlo schema-migrate scaffold-yaml-recent

Generate migration scaffold YAML files for recently added/updated contracts.

phlo schema-migrate scaffold-yaml-recent [OPTIONS]

Options:

--since-hours INTEGER    # Only include contracts modified in last N hours (default: 24)
--limit INTEGER          # Max recent contracts to scaffold
--force                  # Overwrite existing output files

Examples:

phlo schema-migrate scaffold-yaml-recent
phlo schema-migrate scaffold-yaml-recent --since-hours 1 --limit 5

phlo schema-migrate plan/apply/history

Existing migration lifecycle commands remain available:

phlo schema-migrate plan warehouse.customers
phlo schema-migrate apply warehouse.customers
phlo schema-migrate history warehouse.customers --limit 5

Data Migration Commands

For end-to-end spec authoring and execution workflow, see Data Migrations Guide.

phlo migrate validate

Validate a migration YAML spec without executing writes.

phlo migrate validate SPEC_FILE

phlo migrate run

Run a migration YAML spec.

phlo migrate run SPEC_FILE [OPTIONS]

Options:

--dry-run               # Read and validate without writes
--format table|json     # Output format (default: table)

phlo migrate list

List migration specs from default directories (migrations/, workflows/migrations/).

phlo migrate list [--directory PATH]

phlo migrate status

Show recent migration run history.

phlo migrate status [OPTIONS]

Options:

--limit INTEGER         # Number of entries to show (default: 10)
--format table|json     # Output format (default: table)

phlo test

Run tests.

phlo test [TEST_PATH] [OPTIONS]

Options:

--local              # Skip Docker integration tests
--verbose, -v        # Verbose output
--marker, -m MARKER  # Run tests with marker
--coverage           # Generate coverage report

Markers:

  • integration: Integration tests requiring Docker
  • unit: Fast unit tests
  • slow: Slow-running tests

Examples:

# All tests
phlo test

# Specific test file
phlo test tests/test_ingestion.py

# Unit tests only
phlo test -m unit

# Skip integration tests
phlo test --local

# Specific test
phlo test -k test_glucose_ingestion

# With coverage
phlo test --coverage

Logging & Monitoring Commands

phlo logs

Query and filter structured logs from Dagster runs.

phlo logs [OPTIONS]

Options:

--asset NAME         # Filter by asset name
--job NAME           # Filter by job name
--level LEVEL        # Filter by log level: DEBUG, INFO, WARNING, ERROR
--since TIME         # Filter by time (e.g., 1h, 30m, 2d)
--run-id ID          # Get logs for specific run
--follow, -f         # Tail mode - follow new logs in real-time
--full               # Don't truncate long messages
--limit N            # Number of logs to retrieve (default: 100)
--json               # JSON output for scripting

Examples:

# View recent logs
phlo logs

# Filter by asset
phlo logs --asset dlt_glucose_entries

# Filter by log level
phlo logs --level ERROR

# Logs from last hour
phlo logs --since 1h

# Follow logs in real-time
phlo logs --follow

# Logs for specific run
phlo logs --run-id abc-123-def

# JSON output for processing
phlo logs --json --limit 500

phlo backfill

Backfill partitioned assets over a date range.

phlo backfill [ASSET_NAME] [OPTIONS]

Arguments:

  • ASSET_NAME (optional): Asset to backfill

Options:

--start-date DATE    # Start date (YYYY-MM-DD)
--end-date DATE      # End date (YYYY-MM-DD)
--partitions DATES   # Comma-separated partition dates (YYYY-MM-DD,...)
--parallel N         # Number of concurrent partitions (default: 1)
--resume             # Resume last backfill, skipping completed partitions
--dry-run            # Show what would be executed without running
--delay SECS         # Delay between parallel executions in seconds (default: 0.0)

Examples:

# Backfill for date range
phlo backfill dlt_glucose_entries --start-date 2025-01-01 --end-date 2025-01-31

# Specific partitions
phlo backfill dlt_glucose_entries --partitions 2025-01-15,2025-01-16,2025-01-17

# Parallel backfill (5 concurrent)
phlo backfill dlt_glucose_entries --start-date 2025-01-01 --end-date 2025-01-31 --parallel 5

# Resume interrupted backfill
phlo backfill dlt_glucose_entries --resume

# Dry run to see what would execute
phlo backfill dlt_glucose_entries --start-date 2025-01-01 --end-date 2025-01-07 --dry-run

# With delay between executions
phlo backfill dlt_glucose_entries --start-date 2025-01-01 --end-date 2025-01-31 --parallel 3 --delay 2.5

Branch Commands

Manage Nessie catalog branches.

phlo branch create

Create a new branch.

phlo branch create BRANCH_NAME [OPTIONS]

Options:

--from REF           # Create from reference (default: main)
--description DESC   # Branch description

Examples:

# Create from main
phlo branch create dev

# Create from specific commit
phlo branch create feature --from abc123

# With description
phlo branch create dev --description "Development branch"

phlo branch list

List all branches.

phlo branch list [OPTIONS]

Options:

--pattern PATTERN    # Filter by pattern
--show-hashes        # Show commit hashes

Example:

phlo branch list
phlo branch list --pattern "pipeline/*"

phlo branch merge

Merge branches.

phlo branch merge SOURCE TARGET [OPTIONS]

Options:

--strategy STRATEGY  # Merge strategy (default: normal)
--no-ff              # Create merge commit even if fast-forward

Examples:

# Merge dev to main
phlo branch merge dev main

# Force merge commit
phlo branch merge dev main --no-ff

phlo branch delete

Delete a branch.

phlo branch delete BRANCH_NAME [OPTIONS]

Options:

--force, -f          # Force delete even if not merged

Examples:

phlo branch delete old-feature
phlo branch delete old-feature --force

phlo branch diff

Show differences between two branches.

phlo branch diff SOURCE_BRANCH [TARGET_BRANCH] [OPTIONS]

Arguments:

  • SOURCE_BRANCH: Source branch to compare
  • TARGET_BRANCH: Target branch (default: main)

Options:

--format FORMAT      # Output format: table, json (default: table)

Examples:

# Compare dev to main
phlo branch diff dev

# Compare two branches
phlo branch diff feature-a feature-b

# JSON output
phlo branch diff dev main --format json

Catalog Commands

Manage the Iceberg catalog (Nessie-backed).

phlo catalog tables

List all Iceberg tables in the catalog.

phlo catalog tables [OPTIONS]

Options:

--namespace NS       # Filter by namespace (e.g., bronze, silver)
--ref REF            # Nessie branch/tag reference (default: main)
--format FORMAT      # Output format: table, json (default: table)

Examples:

# List all tables
phlo catalog tables

# List tables in specific namespace
phlo catalog tables --namespace bronze

# List tables on specific branch
phlo catalog tables --ref feature-branch

# JSON output
phlo catalog tables --format json

phlo catalog describe

Show detailed table metadata.

phlo catalog describe TABLE_NAME [OPTIONS]

Arguments:

  • TABLE_NAME: Table to describe (e.g., bronze.events)

Options:

--ref REF            # Nessie branch/tag reference (default: main)

Examples:

# Describe table
phlo catalog describe bronze.events

# Describe on specific branch
phlo catalog describe bronze.events --ref dev

Output:

Shows table metadata including:

  • Location
  • Current snapshot ID
  • Format version
  • Schema (columns, types, constraints)
  • Partitioning
  • Properties

phlo catalog history

Show table snapshot history.

phlo catalog history TABLE_NAME [OPTIONS]

Arguments:

  • TABLE_NAME: Table name

Options:

--limit N            # Number of snapshots to show (default: 10)
--ref REF            # Nessie branch/tag reference (default: main)

Examples:

# Show recent snapshots
phlo catalog history bronze.events

# Show last 20 snapshots
phlo catalog history bronze.events --limit 20

phlo lineage show

Display asset lineage in ASCII tree format.

phlo lineage show ASSET_NAME [OPTIONS]

Arguments:

  • ASSET_NAME: Asset name to show lineage for

Options:

--direction DIR      # Direction: upstream, downstream, both (default: both)
--depth N            # Maximum depth to traverse

Examples:

# Show full lineage
phlo lineage show dlt_glucose_entries

# Show only upstream dependencies
phlo lineage show dlt_glucose_entries --direction upstream

# Limit depth
phlo lineage show dlt_glucose_entries --depth 2

phlo lineage export

Export lineage to external formats.

phlo lineage export ASSET_NAME [OPTIONS]

Arguments:

  • ASSET_NAME: Asset name to export lineage for

Options:

--format FORMAT      # Export format: dot, mermaid, json (default: dot)
--output PATH        # Output file path (required)

Examples:

# Export to Graphviz DOT
phlo lineage export dlt_glucose_entries --format dot --output lineage.dot

# Export to Mermaid diagram
phlo lineage export dlt_glucose_entries --format mermaid --output lineage.md

# Export to JSON
phlo lineage export dlt_glucose_entries --format json --output lineage.json

Configuration Commands

phlo config show

Display current configuration.

phlo config show [OPTIONS]

Options:

--format FORMAT      # Output format: yaml, json, env
--secrets            # Show secrets (masked by default)

Examples:

phlo config show
phlo config show --format json
phlo config show --secrets

phlo config validate

Validate configuration files.

phlo config validate [FILE]

Examples:

# Validate phlo.yaml
phlo config validate phlo.yaml

phlo env export

Export the generated environment configuration.

phlo env export [OPTIONS]

Examples:

# Export non-secret defaults
phlo env export

# Export with secrets (from .phlo/.env.local)
phlo env export --include-secrets

# Write to a file
phlo env export --include-secrets --output env.full

Utility Commands

phlo status

Show asset status and freshness.

phlo status [OPTIONS]

Options:

--assets             # Show assets only
--services           # Show services only
--stale              # Show only stale assets
--group GROUP        # Filter by group
--json               # JSON output for scripting

Example:

phlo status
phlo status --stale
phlo status --group nightscout

phlo validate-schema

Validate Pandera schemas.

phlo validate-schema SCHEMA_PATH [OPTIONS]

Options:

--check-constraints  # Check that constraints are defined
--check-descriptions # Check that field descriptions are present

Example:

phlo validate-schema workflows/schemas/events.py
phlo validate-schema workflows/schemas/events.py --no-check-descriptions

phlo validate-workflow

Validate workflow configuration.

phlo validate-workflow WORKFLOW_PATH [OPTIONS]

Arguments:

  • WORKFLOW_PATH: Path to workflow file or directory

Options:

--fix                # Auto-fix issues where possible

Examples:

# Validate single workflow
phlo validate-workflow workflows/ingestion/api/events.py

# Validate directory
phlo validate-workflow workflows/ingestion/

# Auto-fix where possible
phlo validate-workflow workflows/ingestion/weather.py --fix

phlo schema

Manage Pandera schemas.

phlo schema COMMAND [OPTIONS]

Subcommands:

list

List all available Pandera schemas.

phlo schema list [OPTIONS]

Options:

--domain DOMAIN      # Filter by domain
--format FORMAT      # Output format: table, json (default: table)

Examples:

# List all schemas
phlo schema list

# Filter by domain
phlo schema list --domain api

# JSON output
phlo schema list --format json

Optional Plugin Commands

The following commands are provided by optional packages. Install the corresponding package to use these commands.

phlo postgrest

PostgREST configuration and management.

Installation: phlo plugin install phlo-postgrest

phlo hasura

Hasura GraphQL engine management.

Installation: phlo plugin install phlo-hasura

phlo publishing

dbt publishing layer management.

Installation: phlo plugin install phlo-dbt

phlo metrics

Metrics summary and export. Built into core Phlo.

phlo alerts

Alerting rules and notifications.

Installation: phlo plugin install phlo-alerting

phlo openmetadata

OpenMetadata catalog integration.

Installation: phlo plugin install phlo-openmetadata

Run phlo <command> --help after installation for command-specific documentation.

Environment Variables

CLI behavior can be customized with environment variables:

# Dagster home directory
export DAGSTER_HOME=~/.dagster

# Workspace YAML location
export DAGSTER_WORKSPACE=/path/to/workspace.yaml

# Phlo configuration
export PHLO_CONFIG=/path/to/phlo.yaml

# Log level
export PHLO_LOG_LEVEL=DEBUG

Exit Codes

0    # Success
1    # General error
2    # Command not found
3    # Invalid arguments
4    # Configuration error
5    # Service error

Examples Cookbook

Complete Workflow Setup

# 1. Create project
phlo init my-project
cd my-project

# 2. Initialize infrastructure
phlo services init

# 3. Start services
phlo services start

# 4. Create workflow
phlo workflow create

# 5. Run tests
phlo test

# 6. Materialize
phlo materialize dlt_events

Development Workflow

# Start Observatory/phlo-api natively (no Docker)
phlo services start --native

# Create feature branch
phlo branch create feature-new-workflow

# Create workflow
phlo workflow create

# Test workflow
phlo test events

# Materialize to test
phlo materialize dlt_events --partition 2025-01-15

# Merge to main
phlo branch merge feature-new-workflow main

Troubleshooting Workflow

# Check service status
phlo services status

# View logs
phlo services logs -f dagster-webserver

# Check stale assets
phlo status --stale

# Validate configuration
phlo config validate

# Re-materialize failed asset
phlo materialize failed_asset

Next Steps