E2e cli work (#521)

MichaelGHSeg · neelkanth-kaushik · web-flow · commit bee196b761bc · 2026-02-18T16:16:57.000-05:00
* This commit contains code for Analytics Python CLI E2E Testing Tool - LIBRARIES-2434

* Refactor e2e-cli to use unified JSON input format

- Accept --input with JSON containing sequences, config, writeKey, apiHost
- Support time-delayed event sequences for retry testing
- Output JSON result on stdout for test harness integration
- Map config options to SDK parameters (flushAt, flushInterval, maxRetries)

* Add E2E test workflow

Runs sdk-e2e-tests suite against the e2e-cli to verify SDK behavior.

* Add E2E_TEST_SUITES to e2e workflow for selective test execution

* Fix e2e workflow CLI_COMMAND to use console script entry point

Changed from 'python -m e2e_cli' (non-existent module) to 'e2e-cli'
(the actual console_scripts entry point). Removed unnecessary CLI_CWD.

* Rename e2e-test to e2e-cli, update README

Rename directory to match the convention used by all other SDK repos.
Replace the outdated README with one documenting the current unified
input/output format and parameters.

* Add E2E_TESTS_TOKEN for private sdk-e2e-tests repo checkout

* Add e2e-config.json and run-e2e.sh for local E2E testing

Per-SDK config and convenience script for the generic test runner
in sdk-e2e-tests. Run ./e2e-cli/run-e2e.sh to build and test locally.

* Update CI workflow to use run-tests.sh from sdk-e2e-tests

Replace hardcoded env vars and direct npm test call with
./scripts/run-tests.sh which reads e2e-config.json for test
configuration. This ensures CI uses the same config as local runs.

---------

Co-authored-by: neelkanth-kaushik &lt;nkaushik@segment.com&gt;
Co-authored-by: Claude Opus 4.6
diff --git a/.github/workflows/e2e-tests.yml b/.github/workflows/e2e-tests.yml
@@ -0,0 +1,66 @@
+# E2E Tests for analytics-python
+# Copy this file to: analytics-python/.github/workflows/e2e-tests.yml
+#
+# This workflow:
+# 1. Checks out the SDK and sdk-e2e-tests repos
+# 2. Installs the SDK and e2e-cli dependencies
+# 3. Runs the e2e test suite
+
+name: E2E Tests
+
+on:
+  push:
+    branches: [main, master]
+  pull_request:
+    branches: [main, master]
+  workflow_dispatch: # Allow manual trigger
+
+jobs:
+  e2e-tests:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout SDK
+        uses: actions/checkout@v4
+        with:
+          path: sdk
+
+      - name: Checkout sdk-e2e-tests
+        uses: actions/checkout@v4
+        with:
+          repository: segmentio/sdk-e2e-tests
+          token: ${{ secrets.E2E_TESTS_TOKEN }}
+          path: sdk-e2e-tests
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: '20'
+
+      - name: Install Python SDK
+        working-directory: sdk
+        run: pip install -e .
+
+      - name: Install e2e-cli dependencies
+        working-directory: sdk/e2e-cli
+        run: pip install -e .
+
+      - name: Run E2E tests
+        working-directory: sdk-e2e-tests
+        run: |
+          ./scripts/run-tests.sh \
+            --sdk-dir "${{ github.workspace }}/sdk/e2e-cli" \
+            --cli "e2e-cli"
+
+      - name: Upload test results
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: e2e-test-results
+          path: sdk-e2e-tests/test-results/
+          if-no-files-found: ignore
diff --git a/.gitignore b/.gitignore
@@ -10,3 +10,5 @@ build
 .vscode/
 .idea/
 .python-version
+.venv
+.DS_Store
diff --git a/e2e-cli/.env b/e2e-cli/.env
@@ -0,0 +1,4 @@
+APP_NAME='e2e-cli'
+DEBUG_MODE = False
+SEND_EVENTS = True
+LOG_FORMAT='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
diff --git a/e2e-cli/README.md b/e2e-cli/README.md
@@ -0,0 +1,62 @@
+# analytics-python e2e-cli
+
+E2E test CLI for the [analytics-python](https://github.com/segmentio/analytics-python) SDK. Accepts a JSON input describing events and SDK configuration, sends them through the real SDK, and outputs results as JSON.
+
+## Setup
+
+```bash
+cd e2e-cli
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -r requirements.txt
+pip install -e .
+```
+
+## Usage
+
+```bash
+e2e-cli --input '{"writeKey":"...", ...}'
+```
+
+Or without installing:
+
+```bash
+python3 -m src.cli --input '{"writeKey":"...", ...}'
+```
+
+## Input Format
+
+```jsonc
+{
+  "writeKey": "your-write-key",       // required
+  "apiHost": "https://...",           // optional — defaults to https://api.segment.io
+  "sequences": [                      // required — event sequences to send
+    {
+      "delayMs": 0,
+      "events": [
+        { "type": "track", "event": "Test", "userId": "user-1" }
+      ]
+    }
+  ],
+  "config": {                         // optional
+    "flushAt": 100,                   // upload_size in Python SDK
+    "flushInterval": 500,             // ms (auto-converted to seconds if > 100)
+    "maxRetries": 10,
+    "timeout": 15
+  }
+}
+```
+
+Note: Python is a server-side SDK — there is no CDN settings fetch, so `cdnHost` does not apply.
+
+## Output Format
+
+```json
+{ "success": true, "sentBatches": 1 }
+```
+
+On failure:
+
+```json
+{ "success": false, "error": "description", "sentBatches": 0 }
+```
diff --git a/e2e-cli/__init__.py b/e2e-cli/__init__.py
diff --git a/e2e-cli/e2e-config.json b/e2e-cli/e2e-config.json
@@ -0,0 +1,7 @@
+{
+  "sdk": "python",
+  "test_suites": "basic",
+  "auto_settings": false,
+  "patch": null,
+  "env": {}
+}
diff --git a/e2e-cli/main.py b/e2e-cli/main.py
@@ -0,0 +1 @@
+
diff --git a/e2e-cli/requirements.txt b/e2e-cli/requirements.txt
@@ -0,0 +1,6 @@
+click==8.1.8
+python-dotenv==1.0.1
+python-dateutil==2.8.2
+requests==2.32.3
+PyJWT==2.10.1
+backoff==2.2.1
diff --git a/e2e-cli/run-e2e.sh b/e2e-cli/run-e2e.sh
@@ -0,0 +1,37 @@
+#!/bin/bash
+#
+# Run E2E tests for analytics-python
+#
+# Prerequisites: Python 3, pip, Node.js 18+
+#
+# Usage:
+#   ./run-e2e.sh [extra args passed to run-tests.sh]
+#
+# Override sdk-e2e-tests location:
+#   E2E_TESTS_DIR=../my-e2e-tests ./run-e2e.sh
+#
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+SDK_ROOT="$SCRIPT_DIR/.."
+E2E_DIR="${E2E_TESTS_DIR:-$SDK_ROOT/../sdk-e2e-tests}"
+
+echo "=== Building analytics-python e2e-cli ==="
+
+# Install SDK
+cd "$SDK_ROOT"
+pip install -e .
+
+# Install e2e-cli
+cd "$SCRIPT_DIR"
+pip install -e .
+
+echo ""
+
+# Run tests
+cd "$E2E_DIR"
+./scripts/run-tests.sh \
+    --sdk-dir "$SCRIPT_DIR" \
+    --cli "e2e-cli" \
+    "$@"
diff --git a/e2e-cli/setup.py b/e2e-cli/setup.py
@@ -0,0 +1,16 @@
+from setuptools import setup, find_packages
+
+setup(
+    name='e2e-cli',
+    version='0.1.0',
+    packages=find_packages(),
+    include_package_data=True,
+    install_requires=[
+        'click',
+    ],
+    entry_points={
+        'console_scripts': [
+            'e2e-cli = src.cli:run',
+        ],
+    },
+)
diff --git a/e2e-cli/src/__init__.py b/e2e-cli/src/__init__.py
diff --git a/e2e-cli/src/cli.py b/e2e-cli/src/cli.py
@@ -0,0 +1,141 @@
+#!/usr/bin/env python3
+"""
+Analytics Python E2E CLI
+
+Accepts a JSON input with event sequences and SDK configuration,
+sends events through the analytics SDK, and outputs results as JSON.
+"""
+
+import click
+import json
+import sys
+import os
+import time
+import logging
+
+sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "../../")))
+
+from segment.analytics.client import Client  # noqa: E402
+
+
+def setup_logging(debug: bool = False):
+    level = logging.DEBUG if debug else logging.WARNING
+    logging.basicConfig(
+        level=level,
+        format="%(asctime)s %(levelname)s %(name)s: %(message)s",
+        stream=sys.stderr,
+    )
+    return logging.getLogger("e2e-cli")
+
+
+def send_event(client: Client, event: dict, logger: logging.Logger):
+    """Send a single event through the analytics client."""
+    event_type = event.get("type")
+    user_id = event.get("userId", "")
+    anonymous_id = event.get("anonymousId", "")
+    message_id = event.get("messageId")
+    timestamp = event.get("timestamp")
+    context = event.get("context")
+    integrations = event.get("integrations")
+    traits = event.get("traits")
+    properties = event.get("properties")
+    event_name = event.get("event")
+    name = event.get("name")
+    category = event.get("category")
+    group_id = event.get("groupId")
+    previous_id = event.get("previousId")
+
+    logger.debug(f"Sending {event_type} event: {event}")
+
+    if event_type == "identify":
+        client.identify(user_id, traits, context, timestamp, anonymous_id, integrations, message_id)
+    elif event_type == "track":
+        client.track(user_id, event_name, properties, context, timestamp, anonymous_id, integrations, message_id)
+    elif event_type == "page":
+        client.page(user_id, category, name, properties, context, timestamp, anonymous_id, integrations, message_id)
+    elif event_type == "screen":
+        client.screen(user_id, category, name, properties, context, timestamp, anonymous_id, integrations, message_id)
+    elif event_type == "alias":
+        client.alias(previous_id, user_id, context, timestamp, integrations, message_id)
+    elif event_type == "group":
+        client.group(user_id, group_id, traits, context, timestamp, anonymous_id, integrations, message_id)
+    else:
+        raise ValueError(f"Unknown event type: {event_type}")
+
+
+@click.command()
+@click.option("--input", "input_json", type=str, required=True, help="JSON input with sequences and config")
+@click.option("--debug", is_flag=True, help="Enable debug logging")
+def run(input_json: str, debug: bool):
+    """Run the E2E CLI with the given input configuration."""
+    logger = setup_logging(debug)
+    output = {"success": False, "sentBatches": 0, "error": None}
+
+    try:
+        data = json.loads(input_json)
+
+        write_key = data.get("writeKey", "test-key")
+        api_host = data.get("apiHost", "https://api.segment.io")
+        sequences = data.get("sequences", [])
+        config = data.get("config", {})
+
+        # Extract config options
+        flush_at = config.get("flushAt", 100)  # upload_size in Python SDK
+        flush_interval = config.get("flushInterval", 0.5)  # upload_interval (seconds)
+        max_retries = config.get("maxRetries", 10)
+        timeout = config.get("timeout", 15)
+
+        # If flushInterval is in ms (> 100), convert to seconds
+        if flush_interval > 100:
+            flush_interval = flush_interval / 1000.0
+
+        logger.info(f"Creating client with host={api_host}, flush_at={flush_at}, flush_interval={flush_interval}")
+
+        # Create the analytics client
+        client = Client(
+            write_key=write_key,
+            host=api_host,
+            debug=debug,
+            upload_size=flush_at,
+            upload_interval=flush_interval,
+            max_retries=max_retries,
+            timeout=timeout,
+            sync_mode=False,  # Use async mode to test batching
+        )
+
+        # Process event sequences
+        for seq in sequences:
+            delay_ms = seq.get("delayMs", 0)
+            events = seq.get("events", [])
+
+            if delay_ms > 0:
+                logger.debug(f"Waiting {delay_ms}ms before sending next sequence")
+                time.sleep(delay_ms / 1000.0)
+
+            for event in events:
+                send_event(client, event, logger)
+
+        # Flush and shutdown
+        logger.debug("Flushing client...")
+        client.flush()
+        client.join()
+
+        output["success"] = True
+        # Note: We don't have easy access to batch count from the SDK internals
+        # This could be enhanced if needed
+        output["sentBatches"] = 1  # Placeholder
+
+    except json.JSONDecodeError as e:
+        output["error"] = f"Invalid JSON input: {e}"
+        logger.error(output["error"])
+    except Exception as e:
+        output["error"] = str(e)
+        logger.exception("Error running CLI")
+
+    # Output result as JSON (last line of stdout)
+    print(json.dumps(output))
+    sys.exit(0 if output["success"] else 1)
+
+
+if __name__ == "__main__":
+    run()
