diff --git a/changelog.d/elevenlabs-importer.added.md b/changelog.d/elevenlabs-importer.added.md new file mode 100644 index 000000000..27029e09a --- /dev/null +++ b/changelog.d/elevenlabs-importer.added.md @@ -0,0 +1 @@ +- Added the separately packaged ElevenLabs conversation importer, with JSON export and API fetching support for transcripts, tool calls and results, interruptions, usage metadata, and authenticated recording references. diff --git a/plugins/DEVELOPMENT.md b/plugins/DEVELOPMENT.md index cf5259815..34c58864d 100644 --- a/plugins/DEVELOPMENT.md +++ b/plugins/DEVELOPMENT.md @@ -271,6 +271,7 @@ Use the package directory and distribution name from this table: |---|---|---| | `braintrust-importer` | `kitaru-braintrust-importer` | `python/kitaru-braintrust-importer/vX.Y.Z` | | `claude-agent-sdk` | `kitaru-claude-agent-sdk` | `python/kitaru-claude-agent-sdk/vX.Y.Z` | +| `elevenlabs-importer` | `kitaru-elevenlabs-importer` | `python/kitaru-elevenlabs-importer/vX.Y.Z` | | `evaluator` | `kitaru-evaluator` | `python/kitaru-evaluator/vX.Y.Z` | | `jsonl-importer` | `kitaru-jsonl-importer` | `python/kitaru-jsonl-importer/vX.Y.Z` | | `langfuse-importer` | `kitaru-langfuse-importer` | `python/kitaru-langfuse-importer/vX.Y.Z` | diff --git a/plugins/README.md b/plugins/README.md index 3182de490..e9221ac7c 100644 --- a/plugins/README.md +++ b/plugins/README.md @@ -10,6 +10,7 @@ Kitaru keeps the default catalog in `src/kitaru/server/api/bootstrap.py`. At sta |---|---|---| | `braintrust-importer` | `kitaru-braintrust-importer` | Braintrust importer and importer-backed adapter | | `claude-agent-sdk` | `kitaru-claude-agent-sdk` | Claude Agent SDK recording and replay adapter | +| `elevenlabs-importer` | `kitaru-elevenlabs-importer` | ElevenLabs conversation importer (unpublished, non-default); see [package guide](packages/elevenlabs-importer/README.md) | | `evaluator` | `kitaru-evaluator` | All built-in evaluators | | `jsonl-importer` | `kitaru-jsonl-importer` | Kitaru JSONL importer | | `langfuse-importer` | `kitaru-langfuse-importer` | Langfuse importer and importer-backed adapter | diff --git a/plugins/packages/elevenlabs-importer/CHANGELOG.md b/plugins/packages/elevenlabs-importer/CHANGELOG.md new file mode 100644 index 000000000..b7bb79981 --- /dev/null +++ b/plugins/packages/elevenlabs-importer/CHANGELOG.md @@ -0,0 +1,11 @@ +# Changelog + +## Unreleased + +- Add file and API imports for finished ElevenLabs Agents conversations. +- Preserve transcript events, matched tool requests and results, source analysis, usage, billing metadata, and authenticated recording references. +- Represent transcript events as spans without reconstructing underlying model requests or adding a replay runtime. + +### Release context + +Initial package version: `0.1.0`. Requires Kitaru `0.27.0` or later for the existing imported-node and API-fetch contracts. No new core data contract is required. The worker first-party package inventory also gains this distribution so package installation can bypass a stale dependency cutoff; that inventory update will ship in the next applicable core release. This package is intended for the next applicable plugin release and is not a server-default importer. Publication is pending. diff --git a/plugins/packages/elevenlabs-importer/README.md b/plugins/packages/elevenlabs-importer/README.md new file mode 100644 index 000000000..a0f1cb8bc --- /dev/null +++ b/plugins/packages/elevenlabs-importer/README.md @@ -0,0 +1,128 @@ +# Kitaru ElevenLabs importer + +Import finished ElevenLabs Agents conversations as Kitaru sessions, including transcript events, tool calls, source analysis, and recording references. This is an unreleased development package. It has not been published to PyPI or included in the server-default importer catalog. + +Use the file parser when you already retain ElevenLabs conversation responses in your own database. The optional API fetcher reads selected conversations from ElevenLabs. Neither path starts a voice conversation or runs the source agent. + +## Accepted payloads + +The parser accepts a full conversation detail response from `GET /v1/convai/conversations/{conversation_id}`, a JSON array of full detail responses, an object containing `{"conversations": [details]}`, or a `post_call_transcription` webhook envelope with the detail record in `data`. Conversation-list summaries are insufficient because they do not contain the complete transcript. The upload must be one JSON document, not JSONL. + +Only terminal `done` and `failed` conversations are accepted. Import finished records because a later import with the same conversation ID is skipped, not applied as an update. Parser parameters must be empty: `{}`. Conversation grouping and `--join-on` are unsupported. + +A malformed conversation yields an isolated `ImportFailure`, allowing valid neighboring conversations to import. An unreadable JSON document or unsupported parser parameters fail the upload. + +## Preserved data + +Each conversation becomes one session identified by its ElevenLabs conversation ID. Kitaru deduplicates that ID together with the importer provider. Native IDs, agent identity, status, timestamps, selected call metadata, analysis, and raw billing data remain available in the imported payloads and metadata. `metadata.elevenlabs.provenance` records the upload format and the webhook timestamp when supplied. + +Each transcript event becomes a generic span that preserves its role, message, time within the call, interruption information, and available usage. These spans describe recorded conversation events. They do not claim that an utterance is a complete underlying model request or response. Text selectors show each event's message in the Kitaru node thread. Session selectors identify the first user message and the last agent message when those messages exist. + +Tool calls become tool nodes linked to their transcript event. Requests and results are matched by `request_id`. JSON-encoded arguments and results are decoded for inspection, with the originals preserved in metadata. Failed results remain failures. A request without a result remains an in-progress node, even in a finished conversation, and a result without a request records its missing request in metadata. + +Single-model usage can populate model, token, and reported model-cost fields. Usage involving several models stays in metadata rather than being assigned to a fabricated model call. Conversation charges and ElevenLabs credits remain separate billing metadata. Missing model inputs, system prompts, timing, usage, and costs stay missing. + +When the source reports audio availability, `metadata.elevenlabs.recording_url` references `GET /v1/convai/conversations/{conversation_id}/audio`, and `recording_requires_authentication` is `true`. That endpoint requires ElevenLabs authorization. The importer does not download or host audio, produce a public playback URL, or add an audio player to Kitaru. + +## Use the development package + +Run these commands from a Kitaru checkout. The parser entrypoint is `kitaru_elevenlabs_importer:parse`. The `kitaru_elevenlabs_importer:importer` entrypoint supports both file parsing and API fetching. + +```bash +uv sync --frozen --extra cli --extra worker +uv sync --project plugins --frozen --all-packages +uv run --project plugins python - <<'PY' +from pathlib import Path +from kitaru_elevenlabs_importer import parse + +for item in parse(Path("elevenlabs-conversations.json").read_bytes(), {}): + print(item.model_dump_json()) +PY +``` + +For a local server and worker, build and validate a candidate wheel before registration: + +```bash +uv run --no-sync python scripts/smoke_plugin_artifacts.py \ + --package plugins/packages/elevenlabs-importer \ + --candidate-dir /tmp/kitaru-elevenlabs-wheels +``` + +Start a worker that can resolve the local wheel. Configure `ELEVENLABS_API_KEY` in that worker's environment only if you want API imports. File imports need no ElevenLabs credentials. + +```bash +UV_FIND_LINKS=/tmp/kitaru-elevenlabs-wheels \ + uv run --no-sync kitaru worker start \ + --server http://localhost:8000 \ + --name elevenlabs-import-worker \ + --claim importer +``` + +In another terminal, register the exact candidate version and import a file: + +```bash +uv run --no-sync kitaru importer register elevenlabs-local \ + --server http://localhost:8000 \ + --provider elevenlabs \ + --package kitaru-elevenlabs-importer==0.1.0 \ + --entrypoint kitaru_elevenlabs_importer:importer \ + --display-version 0.1.0 + +uv run --no-sync kitaru session import elevenlabs-conversations.json \ + --server http://localhost:8000 \ + --importer elevenlabs-local@latest \ + --agent your-agent@latest \ + --wait +``` + +These commands create state on the selected development server. Registration stores the package requirement and entrypoint, not wheel bytes. The worker resolves the candidate wheel through `UV_FIND_LINKS`. Register a new immutable importer version when the implementation changes. + +## Fetch from the ElevenLabs API + +The API fetcher requires `ELEVENLABS_API_KEY` in the worker environment with **ElevenAgents Read** permission. Keep the key out of uploaded records, parser parameters, query parameters, and registration metadata. The fetcher uses the official ElevenLabs API endpoint and does not accept a custom host. + +Select individual conversations using the standard Kitaru `--trace-id` flag: + +```bash +uv run --no-sync kitaru session import \ + --server http://localhost:8000 \ + --importer elevenlabs-local@latest \ + --agent your-agent@latest \ + --trace-id your-conversation-id \ + --wait +``` + +Alternatively, select a time window and optionally an ElevenLabs agent: + +```bash +uv run --no-sync kitaru session import \ + --server http://localhost:8000 \ + --importer elevenlabs-local@latest \ + --agent your-agent@latest \ + --since 7d \ + --query '{"agent_id":"your-elevenlabs-agent-id","limit":100}' \ + --wait +``` + +An explicit conversation selection cannot be combined with a time window or `agent_id`. For a time window, `since` is required and `until` defaults to the current time. `--until` selects an explicit upper bound. API query settings are: + +| Field | Accepted value | +|---|---| +| `trace_ids` | Conversation IDs, supplied through repeatable `--trace-id` flags or `--query`. | +| `since`, `until` | Timestamps supplied through `--since`, `--until`, or `--query`. | +| `agent_id` | Optional ElevenLabs agent filter for a time window. | +| `concurrency` | Concurrent detail requests, from 1 through 16. Default: 4. | +| `page_size` | Listing page size, from 1 through 100. Default: 100. | +| `limit` | Maximum scanned conversation-list rows or explicit IDs, from 1 through 10,000. Default: 1,000. | + +API IDs contain only ASCII letters, digits, underscores, or hyphens and are at most 256 characters long. Explicit selections reject duplicate IDs. Unknown query fields are rejected. + +The fetcher paginates conversation summaries and retrieves full details before parsing. The same `limit` also bounds listing requests, so an endless sequence of empty pages fails instead of running indefinitely. A time-window import skips active conversations, which still count toward the scan limit. An explicit ID for an active conversation produces an `ImportFailure`. Reimporting a conversation already stored in Kitaru does not refresh its transcript, analysis, or recording reference. + +## Evaluation and replay limits + +Imported sessions can be inspected, annotated, and evaluated in Kitaru. Source analysis remains ElevenLabs metadata. It does not automatically become a Kitaru evaluator result. + +This package supplies no runnable ElevenLabs agent adapter. Importing a transcript does not provide model or prompt overrides, controlled tool substitution, voice replay, or audio-quality evaluation. Those operations require a separate runtime integration and an appropriate evaluator. + +Licensed under Apache-2.0. diff --git a/plugins/packages/elevenlabs-importer/pyproject.toml b/plugins/packages/elevenlabs-importer/pyproject.toml new file mode 100644 index 000000000..791ac8e44 --- /dev/null +++ b/plugins/packages/elevenlabs-importer/pyproject.toml @@ -0,0 +1,45 @@ +[project] +name = "kitaru-elevenlabs-importer" +version = "0.1.0" +description = "ElevenLabs conversation importer for Kitaru." +readme = "README.md" +license = "Apache-2.0" +requires-python = ">=3.11" +authors = [ + { name = "ZenML GmbH", email = "info@zenml.io" }, +] +classifiers = [ + "Development Status :: 3 - Alpha", + "Intended Audience :: Developers", + "Intended Audience :: Science/Research", + "Operating System :: OS Independent", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.11", + "Programming Language :: Python :: 3.12", + "Programming Language :: Python :: 3.13", + "Programming Language :: Python :: 3.14", + "Topic :: Scientific/Engineering :: Artificial Intelligence", + "Topic :: Software Development :: Libraries :: Python Modules", +] +keywords = ["ai-agents", "elevenlabs", "kitaru", "observability", "traces"] +dependencies = ["kitaru>=0.27.0"] + +[project.optional-dependencies] +api = [] + +[project.urls] +Homepage = "https://kitaru.ai" +Documentation = "https://docs.zenml.io/kitaru/guides/importing-sessions" +Repository = "https://github.com/zenml-io/kitaru" +Issues = "https://github.com/zenml-io/kitaru/issues" +Changelog = "https://github.com/zenml-io/kitaru/blob/develop/plugins/packages/elevenlabs-importer/CHANGELOG.md" + +[build-system] +requires = ["hatchling==1.32.0"] +build-backend = "hatchling.build" + +[tool.hatch.build.targets.wheel] +packages = ["src/kitaru_elevenlabs_importer"] + +[tool.kitaru.artifact] +import-module = "kitaru_elevenlabs_importer" diff --git a/plugins/packages/elevenlabs-importer/src/kitaru_elevenlabs_importer/__init__.py b/plugins/packages/elevenlabs-importer/src/kitaru_elevenlabs_importer/__init__.py new file mode 100644 index 000000000..15c5dab3c --- /dev/null +++ b/plugins/packages/elevenlabs-importer/src/kitaru_elevenlabs_importer/__init__.py @@ -0,0 +1,18 @@ +# Copyright (c) ZenML GmbH 2026. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express +# or implied. See the License for the specific language governing +# permissions and limitations under the License. +"""ElevenLabs Agents conversation importer for Kitaru.""" + +from kitaru_elevenlabs_importer.importer import importer, parse + +__all__ = ["importer", "parse"] diff --git a/plugins/packages/elevenlabs-importer/src/kitaru_elevenlabs_importer/api.py b/plugins/packages/elevenlabs-importer/src/kitaru_elevenlabs_importer/api.py new file mode 100644 index 000000000..f56e03373 --- /dev/null +++ b/plugins/packages/elevenlabs-importer/src/kitaru_elevenlabs_importer/api.py @@ -0,0 +1,253 @@ +# Copyright (c) ZenML GmbH 2026. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express +# or implied. See the License for the specific language governing +# permissions and limitations under the License. +"""Read ElevenLabs conversation details with worker-supplied credentials.""" + +import json +import math +import os +from collections.abc import AsyncIterator +from contextlib import aclosing +from datetime import UTC, datetime +from functools import partial +from typing import Any, Self + +import httpx +from pydantic import ConfigDict, Field, field_validator, model_validator + +from kitaru.api_models.v1.imports import ImportQuery +from kitaru.task.importer import retry_rate_limited, stream_bounded + +_BASE_URL = "https://api.elevenlabs.io/v1/convai/conversations" + + +def _validate_identifier(value: str) -> str: + if ( + not value + or len(value) > 256 + or not all( + character.isascii() and (character.isalnum() or character in "_-") + for character in value + ) + ): + raise ValueError( + "ElevenLabs IDs must contain only ASCII letters, digits, " + "underscores or hyphens" + ) + return value + + +class ElevenLabsImportQuery(ImportQuery): + """Bounded conversation selection using standard Kitaru import queries.""" + + model_config = ConfigDict(extra="forbid", hide_input_in_errors=True) + + trace_ids: list[str] | None = Field(default=None, min_length=1, max_length=10000) + concurrency: int = Field(default=4, ge=1, le=16, strict=True) + agent_id: str | None = None + page_size: int = Field(default=100, ge=1, le=100, strict=True) + limit: int = Field(default=1000, ge=1, le=10000, strict=True) + + @field_validator("trace_ids") + @classmethod + def _validate_ids(cls, value: list[str] | None) -> list[str] | None: + if value is not None: + for identifier in value: + _validate_identifier(identifier) + if len(value) != len(set(value)): + raise ValueError("trace_ids must not contain duplicates") + return value + + @field_validator("agent_id") + @classmethod + def _validate_agent(cls, value: str | None) -> str | None: + return _validate_identifier(value) if value is not None else None + + @model_validator(mode="after") + def _validate_selection(self) -> Self: + if self.trace_ids is not None: + if ( + self.since is not None + or self.until is not None + or self.agent_id is not None + ): + raise ValueError( + "trace_ids cannot be combined with time or agent filters" + ) + if len(self.trace_ids) > self.limit: + raise ValueError("trace_ids count exceeds limit") + return self + + +class _RateLimited(ValueError): + def __init__(self, delay: float) -> None: + super().__init__("ElevenLabs rate limit exceeded") + self.delay = delay + + +def _get_retry_after(exc: Exception) -> float | None: + return exc.delay if isinstance(exc, _RateLimited) else None + + +async def _get_response( + client: httpx.AsyncClient, url: str, params: Any = None +) -> httpx.Response: + try: + response = await client.get(url, params=params) + except httpx.HTTPError: + raise ValueError("ElevenLabs request failed") from None + if response.status_code == 429: + try: + delay = float(response.headers.get("retry-after", "1")) + except ValueError: + delay = 1 + if not math.isfinite(delay): + delay = 1 + raise _RateLimited(min(max(delay, 0), 60)) + if not response.is_success: + raise ValueError(f"ElevenLabs request failed (HTTP {response.status_code})") + return response + + +async def _get_json( + client: httpx.AsyncClient, url: str, params: Any = None +) -> dict[str, Any]: + response = await retry_rate_limited( + partial(_get_response, client, url, params), _get_retry_after + ) + try: + document = response.json() + except ValueError: + raise ValueError("ElevenLabs returned invalid JSON") from None + if not isinstance(document, dict): + raise ValueError("ElevenLabs returned a non-object response") + return document + + +async def _fetch_conversation(client: httpx.AsyncClient, conversation_id: str) -> bytes: + document = await _get_json(client, f"{_BASE_URL}/{conversation_id}") + if document.get("conversation_id") != conversation_id: + raise ValueError("ElevenLabs returned a different conversation_id") + return json.dumps(document).encode("utf-8") + + +async def fetch(query: dict[str, Any]) -> AsyncIterator[bytes]: + """Fetch details by conversation IDs or a paginated start-time window. + + Args: + query: Standard trace_ids or since/until selection with optional agent_id, + page_size, limit, and concurrency bounds. + + Yields: + JSON details for each selected conversation. + + Raises: + ValueError: The query, credentials, or provider response is invalid. + """ + parsed = ElevenLabsImportQuery.model_validate(query) + api_key = os.environ.get("ELEVENLABS_API_KEY") + if not api_key or not api_key.strip(): + raise ValueError("ELEVENLABS_API_KEY must be set in the worker environment") + async with httpx.AsyncClient( + headers={"xi-api-key": api_key}, timeout=60, follow_redirects=False + ) as client: + if parsed.trace_ids is not None: + async with aclosing( + stream_bounded( + ( + _fetch_conversation(client, identifier) + for identifier in parsed.trace_ids + ), + parsed.concurrency, + ) + ) as payloads: + async for payload in payloads: + yield payload + return + since, until = parsed.get_window() + params: dict[str, Any] = { + "call_start_after_unix": math.floor(since.timestamp()), + "call_start_before_unix": math.ceil(until.timestamp()), + "page_size": parsed.page_size, + "exclude_statuses": ["initiated", "in-progress", "processing"], + } + if parsed.agent_id is not None: + params["agent_id"] = parsed.agent_id + cursors: set[str] = set() + seen_ids: set[str] = set() + scanned = 0 + pages = 0 + while scanned < parsed.limit: + pages += 1 + params["page_size"] = min(parsed.page_size, parsed.limit - scanned) + page = await _get_json(client, _BASE_URL, params) + rows = page.get("conversations") + if not isinstance(rows, list): + raise ValueError("ElevenLabs listing lacks conversations array") + identifiers: list[str] = [] + for row in rows[: parsed.limit - scanned]: + scanned += 1 + if not isinstance(row, dict): + raise ValueError( + "ElevenLabs listing contains a non-object conversation" + ) + identifier = row.get("conversation_id") + if not isinstance(identifier, str): + raise ValueError("ElevenLabs listing lacks conversation_id") + _validate_identifier(identifier) + if identifier in seen_ids: + continue + seen_ids.add(identifier) + if row.get("status") in ("done", "failed"): + identifiers.append(identifier) + async with aclosing( + stream_bounded( + ( + _fetch_conversation(client, identifier) + for identifier in identifiers + ), + parsed.concurrency, + ) + ) as payloads: + async for payload in payloads: + # Retention or processing may change between listing and detail. + detail = json.loads(payload) + metadata = detail.get("metadata") + start = ( + metadata.get("start_time_unix_secs") + if isinstance(metadata, dict) + else None + ) + if ( + isinstance(start, bool) + or not isinstance(start, int | float) + or not math.isfinite(start) + ): + raise ValueError("ElevenLabs detail lacks a valid start time") + started_at = datetime.fromtimestamp(start, UTC) + if since <= started_at < until and detail.get("status") in ( + "done", + "failed", + ): + yield payload + if scanned >= parsed.limit or page.get("has_more") is not True: + return + if pages >= parsed.limit: + raise ValueError("ElevenLabs pagination exceeded page-request limit") + cursor = page.get("next_cursor") + if not isinstance(cursor, str) or not cursor or cursor in cursors: + raise ValueError( + "ElevenLabs listing has a missing or repeated pagination cursor" + ) + cursors.add(cursor) + params["cursor"] = cursor diff --git a/plugins/packages/elevenlabs-importer/src/kitaru_elevenlabs_importer/importer.py b/plugins/packages/elevenlabs-importer/src/kitaru_elevenlabs_importer/importer.py new file mode 100644 index 000000000..9df45c8e9 --- /dev/null +++ b/plugins/packages/elevenlabs-importer/src/kitaru_elevenlabs_importer/importer.py @@ -0,0 +1,461 @@ +# Copyright (c) ZenML GmbH 2026. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express +# or implied. See the License for the specific language governing +# permissions and limitations under the License. +"""Normalize ElevenLabs Agents conversations without inventing model requests.""" + +import json +import math +from collections.abc import AsyncIterator, Iterator +from datetime import UTC, datetime, timedelta +from decimal import Decimal +from typing import Any +from urllib.parse import quote + +from kitaru.api_models.v1.imports import ImportFailure +from kitaru.api_models.v1.session import SessionStatus, TokenUsage +from kitaru.api_models.v1.session_node import NodeStatus, NodeType +from kitaru.task.importer import ImportedNode, ImportedSession, flatten_nodes + + +class InvalidImport(ValueError): + """Raised when an upload or parser parameters cannot be interpreted.""" + + +def _get_object(value: Any, name: str) -> dict[str, Any]: + if not isinstance(value, dict): + raise ValueError(f"{name} must be an object") + return value + + +def _get_identifier(value: Any, name: str) -> str: + if not isinstance(value, str) or not value.strip(): + raise ValueError(f"{name} must be a nonempty string") + return value + + +def _get_number(value: Any, name: str) -> float | None: + if value is None: + return None + if ( + isinstance(value, bool) + or not isinstance(value, int | float) + or not math.isfinite(value) + or value < 0 + ): + raise ValueError(f"{name} must be finite and nonnegative") + return float(value) + + +def _get_time(start: datetime | None, entry: dict[str, Any]) -> datetime | None: + offset = _get_number(entry.get("time_in_call_secs"), "time_in_call_secs") + return ( + start + timedelta(seconds=offset) + if start is not None and offset is not None + else None + ) + + +def _decode_json(value: Any) -> Any: + if isinstance(value, str): + try: + return json.loads(value) + except ValueError: + pass + return value + + +def _get_usage(entry: dict[str, Any]) -> dict[str, Any]: + raw = entry.get("llm_usage") + if raw is None: + return {} + models = _get_object( + _get_object(raw, "llm_usage").get("model_usage", {}), "model_usage" + ) + # Several model invocations cannot be represented as one served model. + if len(models) != 1: + return {} + model, raw_usage = next(iter(models.items())) + usage = _get_object(raw_usage, "model usage") + tokens: dict[str, int] = {} + input_counts: list[int] = [] + prices: list[Decimal] = [] + for category, data in usage.items(): + if category not in ( + "input", + "input_cache_read", + "input_cache_write", + "output_total", + ): + continue + data = _get_object(data, "usage category") + count = data.get("tokens") + if count is not None: + if isinstance(count, bool) or not isinstance(count, int) or count < 0: + raise ValueError("token counts must be nonnegative integers") + if category == "output_total": + tokens["output_tokens"] = count + else: + input_counts.append(count) + if category == "input_cache_read": + tokens["cached_input_tokens"] = count + price = data.get("price") + if price is not None: + _get_number(price, "model usage price") + prices.append(Decimal(str(price))) + if input_counts: + tokens["input_tokens"] = sum(input_counts) + return { + "model": _get_identifier(model, "model name"), + "tokens": TokenUsage(**tokens) if tokens else None, + "cost": sum(prices, Decimal(0)) if prices else None, + } + + +def _get_tool_records(entry: dict[str, Any], key: str) -> list[dict[str, Any]]: + records = entry.get(key) + if records is None: + return [] + if not isinstance(records, list): + raise ValueError(f"{key} must be an array") + return [_get_object(record, key) for record in records] + + +def _create_tool_node( + conversation_id: str, + request_id: str, + call: tuple[int, dict[str, Any]] | None, + result: tuple[int, dict[str, Any]] | None, + transcript: list[dict[str, Any]], + start: datetime | None, +) -> ImportedNode: + source = call[1] if call is not None else result[1] if result is not None else {} + name = _get_identifier(source.get("tool_name"), "tool_name") + if call is not None and result is not None and result[1].get("tool_name") != name: + raise ValueError("tool call and result names disagree") + result_value = result[1].get("result_value") if result is not None else None + outputs = _decode_json(result_value) + is_error = result[1].get("is_error", False) if result is not None else False + if not isinstance(is_error, bool): + raise ValueError("is_error must be a boolean") + started_at = _get_time(start, transcript[call[0]]) if call is not None else None + ended_at = _get_time(start, transcript[result[0]]) if result is not None else None + if started_at is not None and ended_at is not None and ended_at < started_at: + raise ValueError("tool result precedes its call") + error = None + if is_error and result is not None: + raw_error = result[1].get("raw_error_message") + error = ( + raw_error + if isinstance(raw_error, str) and raw_error + else ( + result_value + if isinstance(result_value, str) + else json.dumps(result_value) + ) + ) + return ImportedNode( + external_id=f"{conversation_id}:tool:{request_id}", + parent_external_id=f"{conversation_id}:transcript:{call[0]}" + if call is not None + else None, + node_type=NodeType.TOOL_CALL, + name=name, + tool_name=name, + status=NodeStatus.FAILED + if is_error + else NodeStatus.COMPLETED + if result is not None + else NodeStatus.IN_PROGRESS, + error=error, + started_at=started_at, + ended_at=ended_at, + inputs=_decode_json(call[1].get("params_as_json")) + if call is not None + else None, + outputs=outputs, + output_text_selector="" if isinstance(outputs, str) else None, + attributes={}, + metadata={ + "elevenlabs": { + "request_id": request_id, + "tool_call": call[1] if call is not None else None, + "tool_result": result[1] if result is not None else None, + "call_transcript_index": call[0] if call is not None else None, + "result_transcript_index": result[0] if result is not None else None, + "call_missing": call is None, + "result_missing": result is None, + } + }, + ) + + +def _create_session( + record: dict[str, Any], provenance: dict[str, Any] +) -> ImportedSession: + conversation_id = _get_identifier(record.get("conversation_id"), "conversation_id") + status = record.get("status") + if status not in ("done", "failed"): + raise ValueError("conversation must have terminal status done or failed") + raw_transcript = record.get("transcript") + if not isinstance(raw_transcript, list): + raise ValueError("transcript must be an array of full conversation entries") + transcript = [_get_object(entry, "transcript entry") for entry in raw_transcript] + raw_metadata = record.get("metadata") + provider_metadata = _get_object( + raw_metadata if raw_metadata is not None else {}, "metadata" + ) + start_secs = _get_number( + provider_metadata.get("start_time_unix_secs"), "start_time_unix_secs" + ) + duration = _get_number( + provider_metadata.get("call_duration_secs"), "call_duration_secs" + ) + start = datetime.fromtimestamp(start_secs, UTC) if start_secs is not None else None + end = ( + start + timedelta(seconds=duration) + if start is not None and duration is not None + else None + ) + calls: dict[str, tuple[int, dict[str, Any]]] = {} + results: dict[str, tuple[int, dict[str, Any]]] = {} + for index, entry in enumerate(transcript): + for key, bucket in (("tool_calls", calls), ("tool_results", results)): + for tool in _get_tool_records(entry, key): + request_id = _get_identifier(tool.get("request_id"), "request_id") + if request_id in bucket: + raise ValueError(f"duplicate request_id in {key}") + bucket[request_id] = (index, tool) + + nodes: list[ImportedNode] = [] + user_messages: list[dict[str, str]] = [] + agent_messages: list[dict[str, str]] = [] + for index, entry in enumerate(transcript): + role = entry.get("role") + if role not in ("agent", "user"): + raise ValueError("transcript role must be agent or user") + message = entry.get("message") + if message is not None and not isinstance(message, str): + raise ValueError("transcript message must be a string or null") + if "interrupted" in entry and not isinstance(entry["interrupted"], bool): + raise ValueError("interrupted must be a boolean") + text = {"message": message} if message is not None else None + if text is not None: + (user_messages if role == "user" else agent_messages).append(text) + nodes.append( + ImportedNode( + external_id=f"{conversation_id}:transcript:{index}", + node_type=NodeType.SPAN, + name=f"{role} utterance" if message is not None else f"{role} event", + status=NodeStatus.COMPLETED, + started_at=_get_time(start, entry), + inputs=text if role == "user" else None, + outputs=text if role == "agent" else None, + input_text_selector="/message" + if role == "user" and text is not None + else None, + output_text_selector="/message" + if role == "agent" and text is not None + else None, + attributes={"role": role}, + metadata={"elevenlabs": {**entry, "transcript_index": index}}, + **_get_usage(entry), + ) + ) + for tool in _get_tool_records(entry, "tool_calls"): + request_id = tool["request_id"] + nodes.append( + _create_tool_node( + conversation_id, + request_id, + calls[request_id], + results.get(request_id), + transcript, + start, + ) + ) + for tool in _get_tool_records(entry, "tool_results"): + request_id = tool["request_id"] + if request_id not in calls: + nodes.append( + _create_tool_node( + conversation_id, + request_id, + None, + results[request_id], + transcript, + start, + ) + ) + + excluded_metadata = { + "phone_call", + "batch_call", + "whatsapp", + "sms", + "initiator_id", + "authorization_method", + } + metadata = { + key: record[key] + for key in ( + "agent_id", + "agent_name", + "branch_id", + "version_id", + "analysis", + "visited_agents", + "conversation_product", + "environment", + "tag_ids", + "has_audio", + "has_user_audio", + "has_response_audio", + "has_auxiliary_audio", + ) + if key in record + } + metadata.update( + { + "conversation_id": conversation_id, + "source_status": status, + "provider_metadata": { + key: value + for key, value in provider_metadata.items() + if key not in excluded_metadata + }, + "provenance": { + **provenance, + "session_input": "first_user_message", + "session_output": "last_agent_message", + }, + } + ) + if record.get("has_audio") is True: + metadata["recording_url"] = ( + "https://api.elevenlabs.io/v1/convai/conversations/" + f"{quote(conversation_id, safe='')}/audio" + ) + metadata["recording_requires_authentication"] = True + error = provider_metadata.get("error") + session = ImportedSession( + external_id=conversation_id, + name=record.get("agent_name"), + status=SessionStatus.FAILED if status == "failed" else SessionStatus.COMPLETED, + error=error + if isinstance(error, str) + else json.dumps(error) + if error is not None + else None, + started_at=start, + ended_at=end, + inputs=user_messages[0] if user_messages else None, + outputs=agent_messages[-1] if agent_messages else None, + input_text_selector="/message" if user_messages else None, + output_text_selector="/message" if agent_messages else None, + framework="elevenlabs", + metadata={"elevenlabs": metadata}, + nodes=nodes, + ) + flatten_nodes(session.nodes) + session.model_dump_json() + return session + + +def parse( + payload: bytes, params: dict[str, Any] +) -> Iterator[ImportedSession | ImportFailure]: + """Parse conversation details, batches, or transcription webhook exports. + + Args: + payload: UTF-8 JSON with complete conversation details. + params: Empty parameter object. + + Yields: + One session per conversation, or a failure for an invalid record. + + Raises: + InvalidImport: Parameters or the outer JSON document are invalid. + """ + if not isinstance(params, dict) or params: + raise InvalidImport("ElevenLabs parser accepts no parameters") + try: + document = json.loads(payload.decode("utf-8")) + except (ValueError, RecursionError) as exc: + raise InvalidImport("ElevenLabs upload must be valid UTF-8 JSON") from exc + provenance: dict[str, Any] = {"format": "conversation"} + if isinstance(document, dict) and "type" in document: + if document["type"] != "post_call_transcription": + raise InvalidImport( + "Only post_call_transcription webhook exports are supported" + ) + provenance = { + "format": "post_call_transcription", + "event_timestamp": document.get("event_timestamp"), + } + records = [document.get("data")] + elif isinstance(document, dict) and "conversations" in document: + records = document["conversations"] + provenance = {"format": "conversations"} + elif isinstance(document, list): + records = document + provenance = {"format": "array"} + elif isinstance(document, dict): + records = [document] + else: + raise InvalidImport("ElevenLabs upload must contain conversation objects") + if not isinstance(records, list): + raise InvalidImport("conversations must be an array") + grouped: dict[str, list[Any]] = {} + for value in records: + identifier = value.get("conversation_id") if isinstance(value, dict) else None + if isinstance(identifier, str): + grouped.setdefault(identifier, []).append(value) + seen: set[str] = set() + for index, value in enumerate(records, start=1): + external_id = value.get("conversation_id") if isinstance(value, dict) else None + try: + record = _get_object(value, "conversation") + identifier = _get_identifier(external_id, "conversation_id") + if identifier in seen: + continue + seen.add(identifier) + if any(other != record for other in grouped[identifier]): + raise ValueError("conflicting duplicate conversation_id") + session = _create_session(record, provenance) + yield session + except (ValueError, OverflowError, RecursionError) as exc: + yield ImportFailure( + line=index, + external_id=external_id if isinstance(external_id, str) else None, + error=f"Invalid ElevenLabs conversation: {exc}", + ) + + +class ElevenLabsImporter: + """Parse exported conversations and fetch them from ElevenLabs.""" + + def parse( + self, payload: bytes, params: dict[str, Any] + ) -> Iterator[ImportedSession | ImportFailure]: + """Parse complete conversation exports.""" + return parse(payload, params) + + async def fetch(self, query: dict[str, Any]) -> AsyncIterator[bytes]: + """Fetch complete conversation details matching a validated query.""" + from kitaru_elevenlabs_importer.api import fetch + + async for payload in fetch(query): + yield payload + + +importer = ElevenLabsImporter() diff --git a/plugins/pyproject.toml b/plugins/pyproject.toml index 6e7cdb25b..3d45e17cc 100644 --- a/plugins/pyproject.toml +++ b/plugins/pyproject.toml @@ -8,6 +8,7 @@ members = ["packages/*"] kitaru = { path = "..", editable = true } kitaru-claude-agent-sdk = { workspace = true } kitaru-braintrust-importer = { workspace = true } +kitaru-elevenlabs-importer = { workspace = true } kitaru-langfuse-importer = { workspace = true } kitaru-langgraph = { workspace = true } kitaru-langsmith-importer = { workspace = true } @@ -25,6 +26,7 @@ dev = [ "kitaru[server]", "kitaru-claude-agent-sdk", "kitaru-braintrust-importer[adapter]", + "kitaru-elevenlabs-importer", "kitaru-langfuse-importer[adapter]", "kitaru-langgraph[deepagents]", "kitaru-langsmith-importer[adapter]", @@ -85,6 +87,7 @@ known-first-party = [ "kitaru", "kitaru_claude_agent_sdk", "kitaru_evaluator", + "kitaru_elevenlabs_importer", "kitaru_braintrust_importer", "kitaru_jsonl_importer", "kitaru_langfuse_importer", diff --git a/plugins/tests/importers/fixtures/elevenlabs/README.md b/plugins/tests/importers/fixtures/elevenlabs/README.md new file mode 100644 index 000000000..95fbb1d66 --- /dev/null +++ b/plugins/tests/importers/fixtures/elevenlabs/README.md @@ -0,0 +1,17 @@ +# ElevenLabs conversation fixtures + +These are sanitized `GET /v1/convai/conversations/{conversation_id}` responses from five real voice calls to a temporary Kitaru test-shop agent on October 2, 2026. The calls used the ElevenLabs JavaScript SDK 1.26.0, client tools, and the `qwen35-397b-a17b` model. The shop, orders, return policy, and deliberate tool failure are synthetic lab scenarios, not customer conversations. + +| Fixture | Recorded behavior | +| --- | --- | +| `happy_path.json` | A successful `get_order` call followed by an agent answer. | +| `multiple_tools.json` | `get_order` followed by `get_return_policy` within one conversation. | +| `tool_error.json` | A deliberate client-tool execution error, followed by a successful conversational recovery. | +| `interruption.json` | An interrupted agent response preserves both spoken text and `original_message`. | +| `not_found.json` | Two separate `get_order` requests return `found: false` without a tool execution error. | + +Agent, branch, version, conversation, voice, and tool request IDs were replaced consistently with synthetic fixture IDs. The lab agent name was replaced. Transcript text, tool arguments/results, timings, latency metrics, token counts, reported prices, charging breakdowns, analysis, and audio-availability flags retain the API response shape and values. No API credentials, original account identifiers, or audio files are included. + +The original JSON and MP3 exports remain in the ignored local `design/elevenlabs-lab/exports/` directory. These JSON fixtures are sufficient to test parsing without an ElevenLabs account or network access. They do not verify ElevenLabs retention settings, webhook delivery, audio playback, or conversation replay. + +API reference: [Get conversation details](https://elevenlabs.io/docs/api-reference/conversations/get). diff --git a/plugins/tests/importers/fixtures/elevenlabs/happy_path.json b/plugins/tests/importers/fixtures/elevenlabs/happy_path.json new file mode 100644 index 000000000..0fed6bf22 --- /dev/null +++ b/plugins/tests/importers/fixtures/elevenlabs/happy_path.json @@ -0,0 +1,917 @@ +{ + "agent_id": "agent_fixture_shop", + "agent_name": "Kitaru fixture shop", + "conversation_product": "agent", + "status": "done", + "user_id": null, + "branch_id": "branch_fixture_main", + "version_id": "version_fixture_one", + "metadata": { + "start_time_unix_secs": 1790959195, + "accepted_time_unix_secs": 1790959195, + "call_duration_secs": 25, + "queue_wait_secs": null, + "cost": 195, + "deletion_settings": { + "deletion_time_unix_secs": 1793551224, + "deleted_logs_at_time_unix_secs": null, + "deleted_audio_at_time_unix_secs": null, + "deleted_transcript_at_time_unix_secs": null, + "delete_transcript_and_pii": false, + "delete_audio": false + }, + "feedback": { + "type": null, + "overall_score": null, + "likes": 0, + "dislikes": 0, + "rating": null, + "comment": null + }, + "authorization_method": "signed_url", + "charging": { + "dev_discount": false, + "is_burst": false, + "tier": "free", + "llm_usage": { + "irreversible_generation": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 3687, + "price": 0.00232281 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 83, + "price": 0.001245 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 3687, + "price": 0.00232281 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 83, + "price": 0.001245 + } + } + } + ] + }, + "initiated_generation": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 5968, + "price": 0.0037598400000000004 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 111, + "price": 0.001665 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 5968, + "price": 0.0037598400000000004 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 111, + "price": 0.001665 + } + } + } + ] + } + }, + "llm_price": 0.00542484, + "llm_charge": 55, + "call_charge": 140, + "platform_charge": 140, + "platform_usage": { + "category_usage": { + "voice": { + "credits": 140, + "price": 0.013892176518146879, + "quantity": 0.4240053490347539 + } + } + }, + "platform_price": 0.013892176518146879, + "free_minutes_consumed": 0.0, + "free_llm_dollars_consumed": 0.0, + "tts_usage": { + "primary_tts_model": "eleven_v4_turbo", + "total_audio_output_seconds": 15.999999999999996, + "total_characters": 288, + "per_voice_usage": [ + { + "voice_id": "voice_fixture_one", + "audio_output_seconds": 15.999999999999996 + } + ] + }, + "asr_usage": { + "asr_model": "scribe_realtime", + "total_transcription_calls": 39, + "total_audio_input_seconds": 7.152000000000003 + }, + "analysis": null + }, + "phone_call": null, + "batch_call": null, + "termination_reason": "Client disconnected: 1000", + "error": null, + "warnings": [], + "main_language": "en", + "rag_usage": null, + "text_only": false, + "features_usage": { + "language_detection": { + "enabled": false, + "used": false + }, + "transfer_to_agent": { + "enabled": false, + "used": false + }, + "transfer_to_number": { + "enabled": false, + "used": false + }, + "multivoice": { + "enabled": false, + "used": false + }, + "dtmf_tones": { + "enabled": false, + "used": false + }, + "external_mcp_servers": { + "enabled": false, + "used": false + }, + "pii_zrm_workspace": false, + "pii_zrm_agent": false, + "tool_dynamic_variable_updates": { + "enabled": false, + "used": false + }, + "is_livekit": false, + "voicemail_detection": { + "enabled": false, + "used": false + }, + "dtmf_input": { + "enabled": false, + "used": false + }, + "workflow": { + "enabled": false, + "tool_node": { + "enabled": false, + "used": false + }, + "standalone_agent_node": { + "enabled": false, + "used": false + }, + "phone_number_node": { + "enabled": false, + "used": false + }, + "end_node": { + "enabled": false, + "used": false + }, + "override_agent_node": { + "enabled": false, + "used": false + }, + "say_node": { + "enabled": false, + "used": false + }, + "update_state_node": { + "enabled": false, + "used": false + }, + "subgraph_node": { + "enabled": false, + "used": false + }, + "transfer_to_genesys_node": { + "enabled": false, + "used": false + } + }, + "agent_testing": { + "enabled": false, + "tests_ran_after_last_modification": false, + "tests_ran_in_last_7_days": false + }, + "versioning": { + "enabled": true, + "used": false + }, + "file_input": { + "enabled": true, + "used": false + }, + "freeform_procedure": { + "enabled": false, + "used": false + }, + "structured_procedure": { + "enabled": false, + "used": false + } + }, + "eleven_assistant": { + "is_eleven_assistant": false + }, + "initiator_id": null, + "conversation_initiation_source": "js_sdk", + "conversation_initiation_source_version": "1.26.0", + "timezone": "Europe/Amsterdam", + "async_metadata": null, + "whatsapp": null, + "sms": null, + "agent_created_from": "api", + "agent_last_updated_from": "api", + "voice_rewards": [], + "cost_fiat": 0.01931701651814688 + }, + "analysis": { + "evaluation_criteria_results": {}, + "data_collection_results": {}, + "evaluation_criteria_results_list": [], + "data_collection_results_list": [], + "call_successful": "success", + "call_success_score": null, + "transcript_summary": "The user asked for the delivery date of order 1001. The agent used a tool to confirm the order has shipped and is estimated to arrive on October 7th. The user was satisfied, and the conversation ended positively.", + "call_summary_title": "Order Delivery Status", + "scoped": [], + "sentiment_analysis": { + "overall_label": "positive", + "overall_sentiment_score": 0.3, + "overall_frustration_score": 0.0, + "min_user_sentiment_score": 0.0, + "max_user_frustration_score": 0.0, + "num_scored_user_turns": 2 + } + }, + "visited_agents": [ + { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main" + } + ], + "conversation_initiation_client_data": { + "conversation_config_override": { + "asr": null, + "turn": null, + "tts": { + "model_id": null, + "voice_id": null, + "supported_voices": null, + "stability": null, + "speed": null, + "similarity_boost": null, + "pronunciation_dictionary_locators": null + }, + "conversation": { + "text_only": false, + "max_duration_seconds": null + }, + "agent": { + "first_message": null, + "language": null, + "max_conversation_duration_message": null, + "prompt": null + } + }, + "custom_llm_extra_body": {}, + "user_id": null, + "source_info": { + "source": "js_sdk", + "version": "1.26.0" + }, + "branch_id": null, + "environment": null, + "starting_workflow_node_id": null, + "experiments": [], + "procedure_ids": null, + "dynamic_variables": { + "system__agent_id": "agent_fixture_shop", + "system__agent_turns": 3, + "system__current_agent_turns": 3, + "system__current_subagent_turns": 3, + "system__current_agent_id": "agent_fixture_shop", + "system__conversation_id": "conv_fixture_happy_path", + "system__call_duration_secs": 25, + "system__time_utc": "2026-10-02T16:40:20.603952+00:00", + "system__timezone": "Europe/Amsterdam", + "system__time": "Friday, 18:40 02 October 2026", + "system__time_hour": 18, + "system__time_minute": 40, + "system__time_second": 20, + "system__is_text_only": false, + "system__channel": "js_sdk", + "system__caller_id": null, + "system__called_number": null, + "system__call_sid": null, + "system__call_id": null, + "system__conversation_history": "{\"x-elevenlabs-history\": true, \"entries\": [{\"role\": \"agent\", \"message\": \"Hi, this is the Kitaru test shop. How can I help you today?\"}, {\"role\": \"user\", \"message\": \"Uh, hey, can you check the delivery date for order 1001?\"}, {\"role\": \"agent\", \"tool_requests\": [{\"tool_name\": \"get_order\", \"params_as_json\": {\"order_id\": \"1001\"}}]}, {\"role\": \"agent\", \"tool_results\": [{\"tool_name\": \"get_order\", \"result_value\": \"{\\\"found\\\":true,\\\"order_id\\\":\\\"1001\\\",\\\"status\\\":\\\"shipped\\\",\\\"estimated_delivery\\\":\\\"October 7\\\",\\\"product_type\\\":\\\"standard\\\"}\"}]}, {\"role\": \"agent\", \"message\": \"[friendly] Your order one zero zero one has shipped and is estimated to arrive on October seven. Is there anything else you need help with?\"}, {\"role\": \"user\", \"message\": \"Uh, no, I think that's great. Thanks.\"}, {\"role\": \"agent\", \"message\": \"[cheerful] You're very welcome! Have a wonderful day and enjoy your order when it arrives.\"}]}", + "system__user_id": null, + "system__initiator_id": null, + "system__environment": "production" + }, + "tool_mock_config": { + "mocking_strategy": "none", + "fallback_strategy": "raise_error", + "mocked_tool_names": [] + }, + "tool_mock_overrides": {} + }, + "environment": "production", + "experiment_assignments": null, + "hiding_reason": null, + "smb_metadata": null, + "conversation_id": "conv_fixture_happy_path", + "has_audio": true, + "has_user_audio": true, + "has_response_audio": true, + "has_auxiliary_audio": false, + "transcript": [ + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": "Hi, this is the Kitaru test shop. How can I help you today?", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 0, + "conversation_turn_metrics": { + "metrics": { + "convai_tts_service_ttfb": { + "elapsed_time": 0.07953741913661361 + }, + "convai_ttf_audio_since_silence": { + "elapsed_time": 0.08598784822970629 + }, + "convai_turn_silence_before_initiation": { + "elapsed_time": 0.0 + } + }, + "convai_asr_provider": null, + "convai_tts_model": "eleven_v4_turbo", + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 1, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "user", + "agent_metadata": null, + "message": "Uh, hey, can you check the delivery date for order 1001?", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 4, + "conversation_turn_metrics": { + "metrics": { + "convai_asr_trailing_service_latency": { + "elapsed_time": 0.08513424918055534 + } + }, + "convai_asr_provider": "Scribe v2 Realtime", + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": "audio", + "source_event_id": 49, + "used_static_kb_document_ids": [], + "analysis": { + "user_sentiment_score": 0.0, + "user_frustration_score": 0.0 + }, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": null, + "multivoice_message": null, + "tool_calls": [ + { + "type": "client", + "request_id": "request_fixture_happy_path_1", + "tool_name": "get_order", + "params_as_json": "{\"order_id\": \"1001\"}", + "tool_has_been_called": true, + "tool_details": { + "type": "client", + "parameters": "{\"order_id\": \"1001\"}" + } + } + ], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 8, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttfb": { + "elapsed_time": 0.31723957462236285 + }, + "convai_llm_tool_request_generation_latency": { + "elapsed_time": 0.3177112746052444 + } + }, + "convai_asr_provider": null, + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1162, + "price": 0.0007320600000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 30, + "price": 0.00045 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1162, + "price": 0.0007320600000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 30, + "price": 0.00045 + } + } + } + ] + }, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 49, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": null, + "multivoice_message": null, + "tool_calls": [], + "tool_results": [ + { + "request_id": "request_fixture_happy_path_1", + "tool_name": "get_order", + "result_value": "{\"found\":true,\"order_id\":\"1001\",\"status\":\"shipped\",\"estimated_delivery\":\"October 7\",\"product_type\":\"standard\"}", + "is_error": false, + "is_blocked": false, + "tool_has_been_called": true, + "tool_latency_secs": 0.16312616784125566, + "error_type": "", + "raw_error_message": "", + "dynamic_variable_updates": [], + "type": "client" + } + ], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 8, + "conversation_turn_metrics": null, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 49, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": "[friendly] Your order one zero zero one has shipped and is estimated to arrive on October seven. Is there anything else you need help with?", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 8, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttf_sentence": { + "elapsed_time": 0.25363810593262315 + }, + "convai_llm_service_ttfb": { + "elapsed_time": 0.1848239367827773 + }, + "convai_llm_service_tt_last_sentence": { + "elapsed_time": 0.3244772348552942 + }, + "convai_tts_service_ttfb": { + "elapsed_time": 0.0745398891158402 + }, + "convai_ttf_audio_since_silence": { + "elapsed_time": 1.1790303361676633 + }, + "convai_turn_silence_before_initiation": { + "elapsed_time": 0.192 + } + }, + "convai_asr_provider": null, + "convai_tts_model": "eleven_v4_turbo", + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1239, + "price": 0.0007805700000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 30, + "price": 0.00045 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1239, + "price": 0.0007805700000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 30, + "price": 0.00045 + } + } + } + ] + }, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 49, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "user", + "agent_metadata": null, + "message": "Uh, no, I think that's great. Thanks.", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 17, + "conversation_turn_metrics": { + "metrics": { + "convai_asr_trailing_service_latency": { + "elapsed_time": 0.06499631004408002 + } + }, + "convai_asr_provider": "Scribe v2 Realtime", + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": "audio", + "source_event_id": 118, + "used_static_kb_document_ids": [], + "analysis": { + "user_sentiment_score": 0.4, + "user_frustration_score": 0.0 + }, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": "[cheerful] You're very welcome! Have a wonderful day and enjoy your order when it arrives.", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 19, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttf_sentence": { + "elapsed_time": 0.28745100600644946 + }, + "convai_llm_service_ttfb": { + "elapsed_time": 0.22584062675014138 + }, + "convai_llm_service_tt_last_sentence": { + "elapsed_time": 0.3250950458459556 + }, + "convai_tts_service_ttfb": { + "elapsed_time": 0.0885465289466083 + }, + "convai_ttf_audio_since_silence": { + "elapsed_time": 0.8796210129894316 + }, + "convai_turn_silence_before_initiation": { + "elapsed_time": 0.384 + }, + "convai_turn_asr_latency": { + "elapsed_time": 0.06499631004408002 + } + }, + "convai_asr_provider": null, + "convai_tts_model": "eleven_v4_turbo", + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1286, + "price": 0.00081018 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 23, + "price": 0.000345 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1286, + "price": 0.00081018 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 23, + "price": 0.000345 + } + } + } + ] + }, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 118, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + } + ], + "tag_ids": [], + "realtime_config_snapshots": null, + "orchestrator": "conversational", + "otlp_traces": null +} diff --git a/plugins/tests/importers/fixtures/elevenlabs/interruption.json b/plugins/tests/importers/fixtures/elevenlabs/interruption.json new file mode 100644 index 000000000..d12bedc31 --- /dev/null +++ b/plugins/tests/importers/fixtures/elevenlabs/interruption.json @@ -0,0 +1,1069 @@ +{ + "agent_id": "agent_fixture_shop", + "agent_name": "Kitaru fixture shop", + "conversation_product": "agent", + "status": "done", + "user_id": null, + "branch_id": "branch_fixture_main", + "version_id": "version_fixture_one", + "metadata": { + "start_time_unix_secs": 1790959537, + "accepted_time_unix_secs": 1790959538, + "call_duration_secs": 52, + "queue_wait_secs": null, + "cost": 343, + "deletion_settings": { + "deletion_time_unix_secs": 1793551595, + "deleted_logs_at_time_unix_secs": null, + "deleted_audio_at_time_unix_secs": null, + "deleted_transcript_at_time_unix_secs": null, + "delete_transcript_and_pii": false, + "delete_audio": false + }, + "feedback": { + "type": null, + "overall_score": null, + "likes": 0, + "dislikes": 0, + "rating": null, + "comment": null + }, + "authorization_method": "signed_url", + "charging": { + "dev_discount": false, + "is_burst": false, + "tier": "free", + "llm_usage": { + "irreversible_generation": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 5209, + "price": 0.00328167 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 184, + "price": 0.00276 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 5209, + "price": 0.00328167 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 184, + "price": 0.00276 + } + } + } + ] + }, + "initiated_generation": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 5209, + "price": 0.00328167 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 184, + "price": 0.00276 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 5209, + "price": 0.00328167 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 184, + "price": 0.00276 + } + } + } + ] + } + }, + "llm_price": 0.00604167, + "llm_charge": 61, + "call_charge": 282, + "platform_charge": 282, + "platform_usage": { + "category_usage": { + "voice": { + "credits": 282, + "price": 0.028058170090452767, + "quantity": 0.8532778815288717 + } + } + }, + "platform_price": 0.028058170090452767, + "free_minutes_consumed": 0.0, + "free_llm_dollars_consumed": 0.0, + "tts_usage": { + "primary_tts_model": "eleven_v4_turbo", + "total_audio_output_seconds": 38.800000000000004, + "total_characters": 765, + "per_voice_usage": [ + { + "voice_id": "voice_fixture_one", + "audio_output_seconds": 38.800000000000004 + } + ] + }, + "asr_usage": { + "asr_model": "scribe_realtime", + "total_transcription_calls": 85, + "total_audio_input_seconds": 14.872000000000009 + }, + "analysis": null + }, + "phone_call": null, + "batch_call": null, + "termination_reason": "Client disconnected: 1000", + "error": null, + "warnings": [], + "main_language": "en", + "rag_usage": null, + "text_only": false, + "features_usage": { + "language_detection": { + "enabled": false, + "used": false + }, + "transfer_to_agent": { + "enabled": false, + "used": false + }, + "transfer_to_number": { + "enabled": false, + "used": false + }, + "multivoice": { + "enabled": false, + "used": false + }, + "dtmf_tones": { + "enabled": false, + "used": false + }, + "external_mcp_servers": { + "enabled": false, + "used": false + }, + "pii_zrm_workspace": false, + "pii_zrm_agent": false, + "tool_dynamic_variable_updates": { + "enabled": false, + "used": false + }, + "is_livekit": false, + "voicemail_detection": { + "enabled": false, + "used": false + }, + "dtmf_input": { + "enabled": false, + "used": false + }, + "workflow": { + "enabled": false, + "tool_node": { + "enabled": false, + "used": false + }, + "standalone_agent_node": { + "enabled": false, + "used": false + }, + "phone_number_node": { + "enabled": false, + "used": false + }, + "end_node": { + "enabled": false, + "used": false + }, + "override_agent_node": { + "enabled": false, + "used": false + }, + "say_node": { + "enabled": false, + "used": false + }, + "update_state_node": { + "enabled": false, + "used": false + }, + "subgraph_node": { + "enabled": false, + "used": false + }, + "transfer_to_genesys_node": { + "enabled": false, + "used": false + } + }, + "agent_testing": { + "enabled": false, + "tests_ran_after_last_modification": false, + "tests_ran_in_last_7_days": false + }, + "versioning": { + "enabled": true, + "used": false + }, + "file_input": { + "enabled": true, + "used": false + }, + "freeform_procedure": { + "enabled": false, + "used": false + }, + "structured_procedure": { + "enabled": false, + "used": false + } + }, + "eleven_assistant": { + "is_eleven_assistant": false + }, + "initiator_id": null, + "conversation_initiation_source": "js_sdk", + "conversation_initiation_source_version": "1.26.0", + "timezone": "Europe/Amsterdam", + "async_metadata": null, + "whatsapp": null, + "sms": null, + "agent_created_from": "api", + "agent_last_updated_from": "api", + "voice_rewards": [], + "cost_fiat": 0.03409984009045277 + }, + "analysis": { + "evaluation_criteria_results": {}, + "data_collection_results": {}, + "evaluation_criteria_results_list": [], + "data_collection_results_list": [], + "call_successful": "success", + "call_success_score": null, + "transcript_summary": "The user inquired about returning a clearance item. The agent provided the return policy, stating a 30-day window for unused items in original packaging, noting that clearance items are final sale. When the user couldn't provide an order number, the agent advised them to call back with it for further assistance.", + "call_summary_title": "Clearance Item Return Policy", + "scoped": [], + "sentiment_analysis": { + "overall_label": "neutral", + "overall_sentiment_score": 0.0, + "overall_frustration_score": 0.0, + "min_user_sentiment_score": 0.0, + "max_user_frustration_score": 0.0, + "num_scored_user_turns": 2 + } + }, + "visited_agents": [ + { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main" + } + ], + "conversation_initiation_client_data": { + "conversation_config_override": { + "asr": null, + "turn": null, + "tts": { + "model_id": null, + "voice_id": null, + "supported_voices": null, + "stability": null, + "speed": null, + "similarity_boost": null, + "pronunciation_dictionary_locators": null + }, + "conversation": { + "text_only": false, + "max_duration_seconds": null + }, + "agent": { + "first_message": null, + "language": null, + "max_conversation_duration_message": null, + "prompt": null + } + }, + "custom_llm_extra_body": {}, + "user_id": null, + "source_info": { + "source": "js_sdk", + "version": "1.26.0" + }, + "branch_id": null, + "environment": null, + "starting_workflow_node_id": null, + "experiments": [], + "procedure_ids": null, + "dynamic_variables": { + "system__agent_id": "agent_fixture_shop", + "system__agent_turns": 4, + "system__current_agent_turns": 4, + "system__current_subagent_turns": 4, + "system__current_agent_id": "agent_fixture_shop", + "system__conversation_id": "conv_fixture_interruption", + "system__call_duration_secs": 51, + "system__time_utc": "2026-10-02T16:46:29.047749+00:00", + "system__timezone": "Europe/Amsterdam", + "system__time": "Friday, 18:46 02 October 2026", + "system__time_hour": 18, + "system__time_minute": 46, + "system__time_second": 29, + "system__is_text_only": false, + "system__channel": "js_sdk", + "system__caller_id": null, + "system__called_number": null, + "system__call_sid": null, + "system__call_id": null, + "system__conversation_history": "{\"x-elevenlabs-history\": true, \"entries\": [{\"role\": \"agent\", \"message\": \"Hi, this is the Kitaru test shop. How can I help you today?\"}, {\"role\": \"user\", \"message\": \"Um, yeah. Hey, so I want to return a regular item, um, please. So, like, yeah, can you tell me a little bit how that would work?\"}, {\"role\": \"agent\", \"message\": \"[friendly] I can certainly help you with that!...\"}, {\"role\": \"user\", \"message\": \"Yeah, actually, um, so it was actually a clearance item. Uh, yeah.\"}, {\"role\": \"agent\", \"tool_requests\": [{\"tool_name\": \"get_return_policy\", \"params_as_json\": {\"product_type\": \"clearance\"}}]}, {\"role\": \"agent\", \"tool_results\": [{\"tool_name\": \"get_return_policy\", \"result_value\": \"{\\\"product_type\\\":\\\"clearance\\\",\\\"return_window_days\\\":30,\\\"conditions\\\":[\\\"unused\\\",\\\"original packaging\\\"],\\\"final_sale\\\":true}\"}]}, {\"role\": \"agent\", \"message\": \"[friendly] For clearance items, you have thirty days to return them as long as they are unused and in their original packaging. However, please note that clearance items are considered final sale, so you might not be able to get a refund depending on the specific circumstances. Do you have the order number for this item so I can check the details for you?\"}, {\"role\": \"user\", \"message\": \"Uh, yeah, actually no.\"}, {\"role\": \"agent\", \"message\": \"[friendly] No problem at all! If you can find your order number later, just give us a call back and we\\u2019ll be happy to assist you further. Is there anything else I can help you with in the meantime?\"}]}", + "system__user_id": null, + "system__initiator_id": null, + "system__environment": "production" + }, + "tool_mock_config": { + "mocking_strategy": "none", + "fallback_strategy": "raise_error", + "mocked_tool_names": [] + }, + "tool_mock_overrides": {} + }, + "environment": "production", + "experiment_assignments": null, + "hiding_reason": null, + "smb_metadata": null, + "conversation_id": "conv_fixture_interruption", + "has_audio": true, + "has_user_audio": true, + "has_response_audio": true, + "has_auxiliary_audio": false, + "transcript": [ + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": "Hi, this is the Kitaru test shop. How can I help you today?", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 0, + "conversation_turn_metrics": { + "metrics": { + "convai_tts_service_ttfb": { + "elapsed_time": 0.06878088880330324 + }, + "convai_ttf_audio_since_silence": { + "elapsed_time": 0.07673765998333693 + }, + "convai_turn_silence_before_initiation": { + "elapsed_time": 0.0 + } + }, + "convai_asr_provider": null, + "convai_tts_model": "eleven_v4_turbo", + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 1, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "user", + "agent_metadata": null, + "message": "Um, yeah. Hey, so I want to return a regular item, um, please. So, like, yeah, can you tell me a little bit how that would work?", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 4, + "conversation_turn_metrics": { + "metrics": { + "convai_asr_trailing_service_latency": { + "elapsed_time": 0.025308349635452032 + } + }, + "convai_asr_provider": "Scribe v2 Realtime", + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": "audio", + "source_event_id": 67, + "used_static_kb_document_ids": [], + "analysis": { + "user_sentiment_score": 0.0, + "user_frustration_score": 0.0 + }, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": "[friendly] I can certainly help you with that!...", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 11, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttf_sentence": { + "elapsed_time": 0.2976644868031144 + }, + "convai_llm_service_ttfb": { + "elapsed_time": 0.24042117781937122 + }, + "convai_llm_service_tt_last_sentence": { + "elapsed_time": 0.39115668600425124 + }, + "convai_tts_service_ttfb": { + "elapsed_time": 0.08051408920437098 + }, + "convai_ttf_audio_since_silence": { + "elapsed_time": 0.7382220249399543 + }, + "convai_turn_silence_before_initiation": { + "elapsed_time": 0.224 + }, + "convai_turn_asr_latency": { + "elapsed_time": 0.025308349635452032 + } + }, + "convai_asr_provider": null, + "convai_tts_model": "eleven_v4_turbo", + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1215, + "price": 0.0007654500000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 35, + "price": 0.000525 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1215, + "price": 0.0007654500000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 35, + "price": 0.000525 + } + } + } + ] + }, + "interrupted": true, + "ignored_as_backchannel": false, + "original_message": "[friendly] I can certainly help you with that! To give you the exact details, could you please tell me the order number for the item you want to return?", + "reasoning": [], + "source_medium": null, + "source_event_id": 67, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "user", + "agent_metadata": null, + "message": "Yeah, actually, um, so it was actually a clearance item. Uh, yeah.", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 13, + "conversation_turn_metrics": { + "metrics": { + "convai_asr_trailing_service_latency": { + "elapsed_time": 0.049526090268045664 + } + }, + "convai_asr_provider": "Scribe v2 Realtime", + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": "audio", + "source_event_id": 110, + "used_static_kb_document_ids": [], + "analysis": { + "user_sentiment_score": 0.0, + "user_frustration_score": 0.0 + }, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": null, + "multivoice_message": null, + "tool_calls": [ + { + "type": "client", + "request_id": "request_fixture_interruption_1", + "tool_name": "get_return_policy", + "params_as_json": "{\"product_type\": \"clearance\"}", + "tool_has_been_called": true, + "tool_details": { + "type": "client", + "parameters": "{\"product_type\": \"clearance\"}" + } + } + ], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 17, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttfb": { + "elapsed_time": 0.39596395613625646 + }, + "convai_llm_tool_request_generation_latency": { + "elapsed_time": 0.3965116459876299 + } + }, + "convai_asr_provider": null, + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1255, + "price": 0.00079065 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 29, + "price": 0.000435 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1255, + "price": 0.00079065 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 29, + "price": 0.000435 + } + } + } + ] + }, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 110, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": null, + "multivoice_message": null, + "tool_calls": [], + "tool_results": [ + { + "request_id": "request_fixture_interruption_1", + "tool_name": "get_return_policy", + "result_value": "{\"product_type\":\"clearance\",\"return_window_days\":30,\"conditions\":[\"unused\",\"original packaging\"],\"final_sale\":true}", + "is_error": false, + "is_blocked": false, + "tool_has_been_called": true, + "tool_latency_secs": 0.15679317899048328, + "error_type": "", + "raw_error_message": "", + "dynamic_variable_updates": [], + "type": "client" + } + ], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 18, + "conversation_turn_metrics": null, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 110, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": "[friendly] For clearance items, you have thirty days to return them as long as they are unused and in their original packaging. However, please note that clearance items are considered final sale, so you might not be able to get a refund depending on the specific circumstances. Do you have the order number for this item so I can check the details for you?", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 19, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttf_sentence": { + "elapsed_time": 0.8087338320910931 + }, + "convai_llm_service_ttfb": { + "elapsed_time": 0.5760788340121508 + }, + "convai_llm_service_tt_last_sentence": { + "elapsed_time": 1.1950654489919543 + }, + "convai_tts_service_ttfb": { + "elapsed_time": 0.08837637910619378 + }, + "convai_ttf_audio_since_silence": { + "elapsed_time": 1.8379955851249397 + }, + "convai_turn_silence_before_initiation": { + "elapsed_time": 0.224 + } + }, + "convai_asr_provider": null, + "convai_tts_model": "eleven_v4_turbo", + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1327, + "price": 0.00083601 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 73, + "price": 0.001095 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1327, + "price": 0.00083601 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 73, + "price": 0.001095 + } + } + } + ] + }, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 110, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "user", + "agent_metadata": null, + "message": "Uh, yeah, actually no.", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 38, + "conversation_turn_metrics": { + "metrics": { + "convai_asr_trailing_service_latency": { + "elapsed_time": 0.05709898006170988 + } + }, + "convai_asr_provider": "Scribe v2 Realtime", + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": "audio", + "source_event_id": 248, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": "[friendly] No problem at all! If you can find your order number later, just give us a call back and we’ll be happy to assist you further. Is there anything else I can help you with in the meantime?", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 40, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttf_sentence": { + "elapsed_time": 0.2928104973398149 + }, + "convai_llm_service_ttfb": { + "elapsed_time": 0.22915253834798932 + }, + "convai_llm_service_tt_last_sentence": { + "elapsed_time": 0.44571369607001543 + }, + "convai_tts_service_ttfb": { + "elapsed_time": 0.07912154914811254 + }, + "convai_ttf_audio_since_silence": { + "elapsed_time": 0.7778393752649426 + }, + "convai_turn_silence_before_initiation": { + "elapsed_time": 0.224 + }, + "convai_turn_asr_latency": { + "elapsed_time": 0.05709898006170988 + } + }, + "convai_asr_provider": null, + "convai_tts_model": "eleven_v4_turbo", + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1412, + "price": 0.0008895600000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 47, + "price": 0.000705 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1412, + "price": 0.0008895600000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 47, + "price": 0.000705 + } + } + } + ] + }, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 248, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + } + ], + "tag_ids": [], + "realtime_config_snapshots": null, + "orchestrator": "conversational", + "otlp_traces": null +} diff --git a/plugins/tests/importers/fixtures/elevenlabs/multiple_tools.json b/plugins/tests/importers/fixtures/elevenlabs/multiple_tools.json new file mode 100644 index 000000000..e64130b01 --- /dev/null +++ b/plugins/tests/importers/fixtures/elevenlabs/multiple_tools.json @@ -0,0 +1,1076 @@ +{ + "agent_id": "agent_fixture_shop", + "agent_name": "Kitaru fixture shop", + "conversation_product": "agent", + "status": "done", + "user_id": null, + "branch_id": "branch_fixture_main", + "version_id": "version_fixture_one", + "metadata": { + "start_time_unix_secs": 1790959431, + "accepted_time_unix_secs": 1790959431, + "call_duration_secs": 51, + "queue_wait_secs": null, + "cost": 352, + "deletion_settings": { + "deletion_time_unix_secs": 1793551487, + "deleted_logs_at_time_unix_secs": null, + "deleted_audio_at_time_unix_secs": null, + "deleted_transcript_at_time_unix_secs": null, + "delete_transcript_and_pii": false, + "delete_audio": false + }, + "feedback": { + "type": null, + "overall_score": null, + "likes": 0, + "dislikes": 0, + "rating": null, + "comment": null + }, + "authorization_method": "signed_url", + "charging": { + "dev_discount": false, + "is_burst": false, + "tier": "free", + "llm_usage": { + "irreversible_generation": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 5274, + "price": 0.00332262 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 177, + "price": 0.0026550000000000002 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 5274, + "price": 0.00332262 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 177, + "price": 0.0026550000000000002 + } + } + } + ] + }, + "initiated_generation": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 6548, + "price": 0.00412524 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 208, + "price": 0.0031200000000000004 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 6548, + "price": 0.00412524 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 208, + "price": 0.0031200000000000004 + } + } + } + ] + } + }, + "llm_price": 0.00724524, + "llm_charge": 72, + "call_charge": 280, + "platform_charge": 280, + "platform_usage": { + "category_usage": { + "voice": { + "credits": 280, + "price": 0.027761318595055484, + "quantity": 0.8473126846986512 + } + } + }, + "platform_price": 0.027761318595055484, + "free_minutes_consumed": 0.0, + "free_llm_dollars_consumed": 0.0, + "tts_usage": { + "primary_tts_model": "eleven_v4_turbo", + "total_audio_output_seconds": 34.559999999999995, + "total_characters": 686, + "per_voice_usage": [ + { + "voice_id": "voice_fixture_one", + "audio_output_seconds": 34.559999999999995 + } + ] + }, + "asr_usage": { + "asr_model": "scribe_realtime", + "total_transcription_calls": 70, + "total_audio_input_seconds": 12.144000000000005 + }, + "analysis": null + }, + "phone_call": null, + "batch_call": null, + "termination_reason": "Client disconnected: 1000", + "error": null, + "warnings": [], + "main_language": "en", + "rag_usage": null, + "text_only": false, + "features_usage": { + "language_detection": { + "enabled": false, + "used": false + }, + "transfer_to_agent": { + "enabled": false, + "used": false + }, + "transfer_to_number": { + "enabled": false, + "used": false + }, + "multivoice": { + "enabled": false, + "used": false + }, + "dtmf_tones": { + "enabled": false, + "used": false + }, + "external_mcp_servers": { + "enabled": false, + "used": false + }, + "pii_zrm_workspace": false, + "pii_zrm_agent": false, + "tool_dynamic_variable_updates": { + "enabled": false, + "used": false + }, + "is_livekit": false, + "voicemail_detection": { + "enabled": false, + "used": false + }, + "dtmf_input": { + "enabled": false, + "used": false + }, + "workflow": { + "enabled": false, + "tool_node": { + "enabled": false, + "used": false + }, + "standalone_agent_node": { + "enabled": false, + "used": false + }, + "phone_number_node": { + "enabled": false, + "used": false + }, + "end_node": { + "enabled": false, + "used": false + }, + "override_agent_node": { + "enabled": false, + "used": false + }, + "say_node": { + "enabled": false, + "used": false + }, + "update_state_node": { + "enabled": false, + "used": false + }, + "subgraph_node": { + "enabled": false, + "used": false + }, + "transfer_to_genesys_node": { + "enabled": false, + "used": false + } + }, + "agent_testing": { + "enabled": false, + "tests_ran_after_last_modification": false, + "tests_ran_in_last_7_days": false + }, + "versioning": { + "enabled": true, + "used": false + }, + "file_input": { + "enabled": true, + "used": false + }, + "freeform_procedure": { + "enabled": false, + "used": false + }, + "structured_procedure": { + "enabled": false, + "used": false + } + }, + "eleven_assistant": { + "is_eleven_assistant": false + }, + "initiator_id": null, + "conversation_initiation_source": "js_sdk", + "conversation_initiation_source_version": "1.26.0", + "timezone": "Europe/Amsterdam", + "async_metadata": null, + "whatsapp": null, + "sms": null, + "agent_created_from": "api", + "agent_last_updated_from": "api", + "voice_rewards": [], + "cost_fiat": 0.035006558595055484 + }, + "analysis": { + "evaluation_criteria_results": {}, + "data_collection_results": {}, + "evaluation_criteria_results_list": [], + "data_collection_results_list": [], + "call_successful": "success", + "call_success_score": null, + "transcript_summary": "The user requested to return order 1002, stating it was unused and in original packaging. The agent confirmed the order's eligibility for return based on the provided details and the standard return policy. The agent then provided concise instructions for the return process, including logging into the website, printing a shipping label, packing the item, and dropping it off, with a note on refund processing time.", + "call_summary_title": "Order Return Process", + "scoped": [], + "sentiment_analysis": { + "overall_label": "neutral", + "overall_sentiment_score": -0.05, + "overall_frustration_score": 0.2, + "min_user_sentiment_score": -0.1, + "max_user_frustration_score": 0.2, + "num_scored_user_turns": 2 + } + }, + "visited_agents": [ + { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main" + } + ], + "conversation_initiation_client_data": { + "conversation_config_override": { + "asr": null, + "turn": null, + "tts": { + "model_id": null, + "voice_id": null, + "supported_voices": null, + "stability": null, + "speed": null, + "similarity_boost": null, + "pronunciation_dictionary_locators": null + }, + "conversation": { + "text_only": false, + "max_duration_seconds": null + }, + "agent": { + "first_message": null, + "language": null, + "max_conversation_duration_message": null, + "prompt": null + } + }, + "custom_llm_extra_body": {}, + "user_id": null, + "source_info": { + "source": "js_sdk", + "version": "1.26.0" + }, + "branch_id": null, + "environment": null, + "starting_workflow_node_id": null, + "experiments": [], + "procedure_ids": null, + "dynamic_variables": { + "system__agent_id": "agent_fixture_shop", + "system__agent_turns": 3, + "system__current_agent_turns": 3, + "system__current_subagent_turns": 3, + "system__current_agent_id": "agent_fixture_shop", + "system__conversation_id": "conv_fixture_multiple_tools", + "system__call_duration_secs": 50, + "system__time_utc": "2026-10-02T16:44:42.015023+00:00", + "system__timezone": "Europe/Amsterdam", + "system__time": "Friday, 18:44 02 October 2026", + "system__time_hour": 18, + "system__time_minute": 44, + "system__time_second": 42, + "system__is_text_only": false, + "system__channel": "js_sdk", + "system__caller_id": null, + "system__called_number": null, + "system__call_sid": null, + "system__call_id": null, + "system__conversation_history": "{\"x-elevenlabs-history\": true, \"entries\": [{\"role\": \"agent\", \"message\": \"Hi, this is the Kitaru test shop. How can I help you today?\"}, {\"role\": \"user\", \"message\": \"Uh, yeah, I want to return order 1002. Um, so basically, it's unused, and it's in its original packaging.\"}, {\"role\": \"agent\", \"tool_requests\": [{\"tool_name\": \"get_order\", \"params_as_json\": {\"order_id\": \"1002\"}}]}, {\"role\": \"agent\", \"tool_results\": [{\"tool_name\": \"get_order\", \"result_value\": \"{\\\"found\\\":true,\\\"order_id\\\":\\\"1002\\\",\\\"status\\\":\\\"delivered\\\",\\\"return_eligible\\\":true,\\\"product_type\\\":\\\"standard\\\"}\"}]}, {\"role\": \"agent\", \"tool_requests\": [{\"tool_name\": \"get_return_policy\", \"params_as_json\": {\"product_type\": \"standard\"}}]}, {\"role\": \"agent\", \"tool_results\": [{\"tool_name\": \"get_return_policy\", \"result_value\": \"{\\\"product_type\\\":\\\"standard\\\",\\\"return_window_days\\\":30,\\\"conditions\\\":[\\\"unused\\\",\\\"original packaging\\\"],\\\"final_sale\\\":false}\"}]}, {\"role\": \"agent\", \"message\": \"[helpful] Great news, your order is eligible for return since it's unused and in the original packaging. You have thirty days from delivery to send it back. Would you like me to guide you through the return steps?\"}, {\"role\": \"user\", \"message\": \"Uh, yeah, sure, but do it concisely, please.\"}, {\"role\": \"agent\", \"message\": \"[helpful] First, visit our website and log into your account to find the return section. Select order one zero zero two and follow the prompts to print your prepaid shipping label. Pack the item securely, attach the label, and drop it off at any authorized carrier location. Once we receive it, your refund will be processed within five business days.\"}]}", + "system__user_id": null, + "system__initiator_id": null, + "system__environment": "production" + }, + "tool_mock_config": { + "mocking_strategy": "none", + "fallback_strategy": "raise_error", + "mocked_tool_names": [] + }, + "tool_mock_overrides": {} + }, + "environment": "production", + "experiment_assignments": null, + "hiding_reason": null, + "smb_metadata": null, + "conversation_id": "conv_fixture_multiple_tools", + "has_audio": true, + "has_user_audio": true, + "has_response_audio": true, + "has_auxiliary_audio": false, + "transcript": [ + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": "Hi, this is the Kitaru test shop. How can I help you today?", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 0, + "conversation_turn_metrics": { + "metrics": { + "convai_tts_service_ttfb": { + "elapsed_time": 0.0783543293364346 + }, + "convai_ttf_audio_since_silence": { + "elapsed_time": 0.09011303773149848 + }, + "convai_turn_silence_before_initiation": { + "elapsed_time": 0.0 + } + }, + "convai_asr_provider": null, + "convai_tts_model": "eleven_v4_turbo", + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 1, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "user", + "agent_metadata": null, + "message": "Uh, yeah, I want to return order 1002. Um, so basically, it's unused, and it's in its original packaging.", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 4, + "conversation_turn_metrics": { + "metrics": { + "convai_asr_trailing_service_latency": { + "elapsed_time": 0.049030168913304806 + } + }, + "convai_asr_provider": "Scribe v2 Realtime", + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": "audio", + "source_event_id": 72, + "used_static_kb_document_ids": [], + "analysis": { + "user_sentiment_score": 0.0, + "user_frustration_score": 0.0 + }, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": null, + "multivoice_message": null, + "tool_calls": [ + { + "type": "client", + "request_id": "request_fixture_multiple_tools_1", + "tool_name": "get_order", + "params_as_json": "{\"order_id\": \"1002\"}", + "tool_has_been_called": true, + "tool_details": { + "type": "client", + "parameters": "{\"order_id\": \"1002\"}" + } + } + ], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 11, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttfb": { + "elapsed_time": 0.4333607139997184 + }, + "convai_llm_tool_request_generation_latency": { + "elapsed_time": 0.43398785404860973 + } + }, + "convai_asr_provider": null, + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1210, + "price": 0.0007622999999999999 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 30, + "price": 0.00045 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1210, + "price": 0.0007622999999999999 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 30, + "price": 0.00045 + } + } + } + ] + }, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 72, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": null, + "multivoice_message": null, + "tool_calls": [], + "tool_results": [ + { + "request_id": "request_fixture_multiple_tools_1", + "tool_name": "get_order", + "result_value": "{\"found\":true,\"order_id\":\"1002\",\"status\":\"delivered\",\"return_eligible\":true,\"product_type\":\"standard\"}", + "is_error": false, + "is_blocked": false, + "tool_has_been_called": true, + "tool_latency_secs": 0.14714236883446574, + "error_type": "", + "raw_error_message": "", + "dynamic_variable_updates": [], + "type": "client" + } + ], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 12, + "conversation_turn_metrics": null, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 72, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": null, + "multivoice_message": null, + "tool_calls": [ + { + "type": "client", + "request_id": "request_fixture_multiple_tools_2", + "tool_name": "get_return_policy", + "params_as_json": "{\"product_type\": \"standard\"}", + "tool_has_been_called": true, + "tool_details": { + "type": "client", + "parameters": "{\"product_type\": \"standard\"}" + } + } + ], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 12, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttfb": { + "elapsed_time": 0.36745701637119055 + }, + "convai_llm_tool_request_generation_latency": { + "elapsed_time": 0.36803221609443426 + } + }, + "convai_asr_provider": null, + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1287, + "price": 0.00081081 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 28, + "price": 0.00042 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1287, + "price": 0.00081081 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 28, + "price": 0.00042 + } + } + } + ] + }, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 72, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": null, + "multivoice_message": null, + "tool_calls": [], + "tool_results": [ + { + "request_id": "request_fixture_multiple_tools_2", + "tool_name": "get_return_policy", + "result_value": "{\"product_type\":\"standard\",\"return_window_days\":30,\"conditions\":[\"unused\",\"original packaging\"],\"final_sale\":false}", + "is_error": false, + "is_blocked": false, + "tool_has_been_called": true, + "tool_latency_secs": 0.1437152079306543, + "error_type": "", + "raw_error_message": "", + "dynamic_variable_updates": [], + "type": "client" + } + ], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 12, + "conversation_turn_metrics": null, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 72, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": "[helpful] Great news, your order is eligible for return since it's unused and in the original packaging. You have thirty days from delivery to send it back. Would you like me to guide you through the return steps?", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 13, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttf_sentence": { + "elapsed_time": 0.27063289610669017 + }, + "convai_llm_service_ttfb": { + "elapsed_time": 0.1979310573078692 + }, + "convai_llm_service_tt_last_sentence": { + "elapsed_time": 0.41168468492105603 + }, + "convai_tts_service_ttfb": { + "elapsed_time": 0.09444231865927577 + }, + "convai_ttf_audio_since_silence": { + "elapsed_time": 1.827789249215275 + }, + "convai_turn_silence_before_initiation": { + "elapsed_time": 0.16 + } + }, + "convai_asr_provider": null, + "convai_tts_model": "eleven_v4_turbo", + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1357, + "price": 0.00085491 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 47, + "price": 0.000705 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1357, + "price": 0.00085491 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 47, + "price": 0.000705 + } + } + } + ] + }, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 72, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "user", + "agent_metadata": null, + "message": "Uh, yeah, sure, but do it concisely, please.", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 25, + "conversation_turn_metrics": { + "metrics": { + "convai_asr_trailing_service_latency": { + "elapsed_time": 0.03900693915784359 + } + }, + "convai_asr_provider": "Scribe v2 Realtime", + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": "audio", + "source_event_id": 173, + "used_static_kb_document_ids": [], + "analysis": { + "user_sentiment_score": -0.1, + "user_frustration_score": 0.2 + }, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": "[helpful] First, visit our website and log into your account to find the return section. Select order one zero zero two and follow the prompts to print your prepaid shipping label. Pack the item securely, attach the label, and drop it off at any authorized carrier location. Once we receive it, your refund will be processed within five business days.", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 28, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttf_sentence": { + "elapsed_time": 0.35271935584023595 + }, + "convai_llm_service_ttfb": { + "elapsed_time": 0.2550375168211758 + }, + "convai_llm_service_tt_last_sentence": { + "elapsed_time": 1.1288630859926343 + }, + "convai_tts_service_ttfb": { + "elapsed_time": 0.09585437923669815 + }, + "convai_ttf_audio_since_silence": { + "elapsed_time": 0.8641075908280909 + }, + "convai_turn_silence_before_initiation": { + "elapsed_time": 0.192 + }, + "convai_turn_asr_latency": { + "elapsed_time": 0.03900693915784359 + } + }, + "convai_asr_provider": null, + "convai_tts_model": "eleven_v4_turbo", + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1420, + "price": 0.0008946000000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 72, + "price": 0.00108 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1420, + "price": 0.0008946000000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 72, + "price": 0.00108 + } + } + } + ] + }, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 173, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + } + ], + "tag_ids": [], + "realtime_config_snapshots": null, + "orchestrator": "conversational", + "otlp_traces": null +} diff --git a/plugins/tests/importers/fixtures/elevenlabs/not_found.json b/plugins/tests/importers/fixtures/elevenlabs/not_found.json new file mode 100644 index 000000000..0daabc7ae --- /dev/null +++ b/plugins/tests/importers/fixtures/elevenlabs/not_found.json @@ -0,0 +1,1073 @@ +{ + "agent_id": "agent_fixture_shop", + "agent_name": "Kitaru fixture shop", + "conversation_product": "agent", + "status": "done", + "user_id": null, + "branch_id": "branch_fixture_main", + "version_id": "version_fixture_one", + "metadata": { + "start_time_unix_secs": 1790959606, + "accepted_time_unix_secs": 1790959606, + "call_duration_secs": 30, + "queue_wait_secs": null, + "cost": 214, + "deletion_settings": { + "deletion_time_unix_secs": 1793551642, + "deleted_logs_at_time_unix_secs": null, + "deleted_audio_at_time_unix_secs": null, + "deleted_transcript_at_time_unix_secs": null, + "delete_transcript_and_pii": false, + "delete_audio": false + }, + "feedback": { + "type": null, + "overall_score": null, + "likes": 0, + "dislikes": 0, + "rating": null, + "comment": null + }, + "authorization_method": "signed_url", + "charging": { + "dev_discount": false, + "is_burst": false, + "tier": "free", + "llm_usage": { + "irreversible_generation": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 4998, + "price": 0.00314874 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 135, + "price": 0.002025 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 4998, + "price": 0.00314874 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 135, + "price": 0.002025 + } + } + } + ] + }, + "initiated_generation": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 4998, + "price": 0.00314874 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 135, + "price": 0.002025 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 4998, + "price": 0.00314874 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 135, + "price": 0.002025 + } + } + } + ] + } + }, + "llm_price": 0.00517374, + "llm_charge": 51, + "call_charge": 163, + "platform_charge": 163, + "platform_usage": { + "category_usage": { + "voice": { + "credits": 163, + "price": 0.016104515037802046, + "quantity": 0.49104591023642563 + } + } + }, + "platform_price": 0.016104515037802046, + "free_minutes_consumed": 0.0, + "free_llm_dollars_consumed": 0.0, + "tts_usage": { + "primary_tts_model": "eleven_v4_turbo", + "total_audio_output_seconds": 22.0, + "total_characters": 375, + "per_voice_usage": [ + { + "voice_id": "voice_fixture_one", + "audio_output_seconds": 22.0 + } + ] + }, + "asr_usage": { + "asr_model": "scribe_realtime", + "total_transcription_calls": 49, + "total_audio_input_seconds": 8.848000000000003 + }, + "analysis": null + }, + "phone_call": null, + "batch_call": null, + "termination_reason": "Client disconnected: 1000", + "error": null, + "warnings": [], + "main_language": "en", + "rag_usage": null, + "text_only": false, + "features_usage": { + "language_detection": { + "enabled": false, + "used": false + }, + "transfer_to_agent": { + "enabled": false, + "used": false + }, + "transfer_to_number": { + "enabled": false, + "used": false + }, + "multivoice": { + "enabled": false, + "used": false + }, + "dtmf_tones": { + "enabled": false, + "used": false + }, + "external_mcp_servers": { + "enabled": false, + "used": false + }, + "pii_zrm_workspace": false, + "pii_zrm_agent": false, + "tool_dynamic_variable_updates": { + "enabled": false, + "used": false + }, + "is_livekit": false, + "voicemail_detection": { + "enabled": false, + "used": false + }, + "dtmf_input": { + "enabled": false, + "used": false + }, + "workflow": { + "enabled": false, + "tool_node": { + "enabled": false, + "used": false + }, + "standalone_agent_node": { + "enabled": false, + "used": false + }, + "phone_number_node": { + "enabled": false, + "used": false + }, + "end_node": { + "enabled": false, + "used": false + }, + "override_agent_node": { + "enabled": false, + "used": false + }, + "say_node": { + "enabled": false, + "used": false + }, + "update_state_node": { + "enabled": false, + "used": false + }, + "subgraph_node": { + "enabled": false, + "used": false + }, + "transfer_to_genesys_node": { + "enabled": false, + "used": false + } + }, + "agent_testing": { + "enabled": false, + "tests_ran_after_last_modification": false, + "tests_ran_in_last_7_days": false + }, + "versioning": { + "enabled": true, + "used": false + }, + "file_input": { + "enabled": true, + "used": false + }, + "freeform_procedure": { + "enabled": false, + "used": false + }, + "structured_procedure": { + "enabled": false, + "used": false + } + }, + "eleven_assistant": { + "is_eleven_assistant": false + }, + "initiator_id": null, + "conversation_initiation_source": "js_sdk", + "conversation_initiation_source_version": "1.26.0", + "timezone": "Europe/Amsterdam", + "async_metadata": null, + "whatsapp": null, + "sms": null, + "agent_created_from": "api", + "agent_last_updated_from": "api", + "voice_rewards": [], + "cost_fiat": 0.021278255037802046 + }, + "analysis": { + "evaluation_criteria_results": {}, + "data_collection_results": {}, + "evaluation_criteria_results_list": [], + "data_collection_results_list": [], + "call_successful": "success", + "call_success_score": null, + "transcript_summary": "The user requested to look up order number \"4040\". The agent used a tool to search for this order but reported that it was not found. The user then corrected the order number to \"404040\". The agent performed another search with the updated number, but this order was also not found, and the conversation ended with the agent unable to locate the requested order.", + "call_summary_title": "Order Lookup Failure", + "scoped": [], + "sentiment_analysis": { + "overall_label": "neutral", + "overall_sentiment_score": -0.1, + "overall_frustration_score": 0.1, + "min_user_sentiment_score": -0.1, + "max_user_frustration_score": 0.1, + "num_scored_user_turns": 2 + } + }, + "visited_agents": [ + { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main" + } + ], + "conversation_initiation_client_data": { + "conversation_config_override": { + "asr": null, + "turn": null, + "tts": { + "model_id": null, + "voice_id": null, + "supported_voices": null, + "stability": null, + "speed": null, + "similarity_boost": null, + "pronunciation_dictionary_locators": null + }, + "conversation": { + "text_only": false, + "max_duration_seconds": null + }, + "agent": { + "first_message": null, + "language": null, + "max_conversation_duration_message": null, + "prompt": null + } + }, + "custom_llm_extra_body": {}, + "user_id": null, + "source_info": { + "source": "js_sdk", + "version": "1.26.0" + }, + "branch_id": null, + "environment": null, + "starting_workflow_node_id": null, + "experiments": [], + "procedure_ids": null, + "dynamic_variables": { + "system__agent_id": "agent_fixture_shop", + "system__agent_turns": 3, + "system__current_agent_turns": 3, + "system__current_subagent_turns": 3, + "system__current_agent_id": "agent_fixture_shop", + "system__conversation_id": "conv_fixture_not_found", + "system__call_duration_secs": 29, + "system__time_utc": "2026-10-02T16:47:16.137210+00:00", + "system__timezone": "Europe/Amsterdam", + "system__time": "Friday, 18:47 02 October 2026", + "system__time_hour": 18, + "system__time_minute": 47, + "system__time_second": 16, + "system__is_text_only": false, + "system__channel": "js_sdk", + "system__caller_id": null, + "system__called_number": null, + "system__call_sid": null, + "system__call_id": null, + "system__conversation_history": "{\"x-elevenlabs-history\": true, \"entries\": [{\"role\": \"agent\", \"message\": \"Hi, this is the Kitaru test shop. How can I help you today?\"}, {\"role\": \"user\", \"message\": \"Um, yeah, could you look up order number 4040 for me, please?\"}, {\"role\": \"agent\", \"tool_requests\": [{\"tool_name\": \"get_order\", \"params_as_json\": {\"order_id\": \"4040\"}}]}, {\"role\": \"agent\", \"tool_results\": [{\"tool_name\": \"get_order\", \"result_value\": \"{\\\"found\\\":false,\\\"order_id\\\":\\\"4040\\\"}\"}]}, {\"role\": \"agent\", \"message\": \"I'm sorry, but I couldn't find an order with the number four zero four zero. Could you please double-check the number and try again?\"}, {\"role\": \"user\", \"message\": \"Uh, yeah, that's four zero four zero four zero.\"}, {\"role\": \"agent\", \"tool_requests\": [{\"tool_name\": \"get_order\", \"params_as_json\": {\"order_id\": \"404040\"}}]}, {\"role\": \"agent\", \"tool_results\": [{\"tool_name\": \"get_order\", \"result_value\": \"{\\\"found\\\":false,\\\"order_id\\\":\\\"404040\\\"}\"}]}, {\"role\": \"agent\", \"message\": \"I'm sorry, but I still can't find an order with the number four zero four zero four zero. Could you...\"}]}", + "system__user_id": null, + "system__initiator_id": null, + "system__environment": "production" + }, + "tool_mock_config": { + "mocking_strategy": "none", + "fallback_strategy": "raise_error", + "mocked_tool_names": [] + }, + "tool_mock_overrides": {} + }, + "environment": "production", + "experiment_assignments": null, + "hiding_reason": null, + "smb_metadata": null, + "conversation_id": "conv_fixture_not_found", + "has_audio": true, + "has_user_audio": true, + "has_response_audio": true, + "has_auxiliary_audio": false, + "transcript": [ + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": "Hi, this is the Kitaru test shop. How can I help you today?", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 0, + "conversation_turn_metrics": { + "metrics": { + "convai_tts_service_ttfb": { + "elapsed_time": 0.09213665500283241 + }, + "convai_ttf_audio_since_silence": { + "elapsed_time": 0.09862786112353206 + }, + "convai_turn_silence_before_initiation": { + "elapsed_time": 0.0 + } + }, + "convai_asr_provider": null, + "convai_tts_model": "eleven_v4_turbo", + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 1, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "user", + "agent_metadata": null, + "message": "Um, yeah, could you look up order number 4040 for me, please?", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 4, + "conversation_turn_metrics": { + "metrics": { + "convai_asr_trailing_service_latency": { + "elapsed_time": 0.07637937413528562 + } + }, + "convai_asr_provider": "Scribe v2 Realtime", + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": "audio", + "source_event_id": 51, + "used_static_kb_document_ids": [], + "analysis": { + "user_sentiment_score": 0.0, + "user_frustration_score": 0.0 + }, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": null, + "multivoice_message": null, + "tool_calls": [ + { + "type": "client", + "request_id": "request_fixture_not_found_1", + "tool_name": "get_order", + "params_as_json": "{\"order_id\": \"4040\"}", + "tool_has_been_called": true, + "tool_details": { + "type": "client", + "parameters": "{\"order_id\": \"4040\"}" + } + } + ], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 8, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttfb": { + "elapsed_time": 0.4392536277882755 + }, + "convai_llm_tool_request_generation_latency": { + "elapsed_time": 0.4397364077158272 + } + }, + "convai_asr_provider": null, + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1164, + "price": 0.0007333200000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 30, + "price": 0.00045 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1164, + "price": 0.0007333200000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 30, + "price": 0.00045 + } + } + } + ] + }, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 51, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": null, + "multivoice_message": null, + "tool_calls": [], + "tool_results": [ + { + "request_id": "request_fixture_not_found_1", + "tool_name": "get_order", + "result_value": "{\"found\":false,\"order_id\":\"4040\"}", + "is_error": false, + "is_blocked": false, + "tool_has_been_called": true, + "tool_latency_secs": 0.14009310584515333, + "error_type": "", + "raw_error_message": "", + "dynamic_variable_updates": [], + "type": "client" + } + ], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 8, + "conversation_turn_metrics": null, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 51, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": "I'm sorry, but I couldn't find an order with the number four zero four zero. Could you please double-check the number and try again?", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 9, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttf_sentence": { + "elapsed_time": 0.2362885493785143 + }, + "convai_llm_service_ttfb": { + "elapsed_time": 0.19001681730151176 + }, + "convai_llm_service_tt_last_sentence": { + "elapsed_time": 0.30761194694787264 + }, + "convai_tts_service_ttfb": { + "elapsed_time": 0.09225476486608386 + }, + "convai_ttf_audio_since_silence": { + "elapsed_time": 1.3673847548775375 + }, + "convai_turn_silence_before_initiation": { + "elapsed_time": 0.288 + } + }, + "convai_asr_provider": null, + "convai_tts_model": "eleven_v4_turbo", + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1224, + "price": 0.00077112 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 31, + "price": 0.000465 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1224, + "price": 0.00077112 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 31, + "price": 0.000465 + } + } + } + ] + }, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 51, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "user", + "agent_metadata": null, + "message": "Uh, yeah, that's four zero four zero four zero.", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 19, + "conversation_turn_metrics": { + "metrics": { + "convai_asr_trailing_service_latency": { + "elapsed_time": 0.08353659976273775 + } + }, + "convai_asr_provider": "Scribe v2 Realtime", + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": "audio", + "source_event_id": 135, + "used_static_kb_document_ids": [], + "analysis": { + "user_sentiment_score": -0.1, + "user_frustration_score": 0.1 + }, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": null, + "multivoice_message": null, + "tool_calls": [ + { + "type": "client", + "request_id": "request_fixture_not_found_2", + "tool_name": "get_order", + "params_as_json": "{\"order_id\": \"404040\"}", + "tool_has_been_called": true, + "tool_details": { + "type": "client", + "parameters": "{\"order_id\": \"404040\"}" + } + } + ], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 21, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttfb": { + "elapsed_time": 0.32154745794832706 + }, + "convai_llm_tool_request_generation_latency": { + "elapsed_time": 0.3220615079626441 + } + }, + "convai_asr_provider": null, + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1273, + "price": 0.00080199 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 32, + "price": 0.00048 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1273, + "price": 0.00080199 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 32, + "price": 0.00048 + } + } + } + ] + }, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 135, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": null, + "multivoice_message": null, + "tool_calls": [], + "tool_results": [ + { + "request_id": "request_fixture_not_found_2", + "tool_name": "get_order", + "result_value": "{\"found\":false,\"order_id\":\"404040\"}", + "is_error": false, + "is_blocked": false, + "tool_has_been_called": true, + "tool_latency_secs": 0.15961632411926985, + "error_type": "", + "raw_error_message": "", + "dynamic_variable_updates": [], + "type": "client" + } + ], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 22, + "conversation_turn_metrics": null, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 135, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": "I'm sorry, but I still can't find an order with the number four zero four zero four zero. Could you...", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 22, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttf_sentence": { + "elapsed_time": 0.2494119107723236 + }, + "convai_llm_service_ttfb": { + "elapsed_time": 0.19293136382475495 + }, + "convai_llm_service_tt_last_sentence": { + "elapsed_time": 0.3889020667411387 + }, + "convai_tts_service_ttfb": { + "elapsed_time": 0.08646715804934502 + }, + "convai_ttf_audio_since_silence": { + "elapsed_time": 1.2900159732215106 + }, + "convai_turn_silence_before_initiation": { + "elapsed_time": 0.224 + } + }, + "convai_asr_provider": null, + "convai_tts_model": "eleven_v4_turbo", + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1337, + "price": 0.0008423100000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 42, + "price": 0.00063 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1337, + "price": 0.0008423100000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 42, + "price": 0.00063 + } + } + } + ] + }, + "interrupted": true, + "ignored_as_backchannel": false, + "original_message": "I'm sorry, but I still can't find an order with the number four zero four zero four zero. Could you please verify the order number again or check if it might be from a different store?", + "reasoning": [], + "source_medium": null, + "source_event_id": 135, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + } + ], + "tag_ids": [], + "realtime_config_snapshots": null, + "orchestrator": "conversational", + "otlp_traces": null +} diff --git a/plugins/tests/importers/fixtures/elevenlabs/tool_error.json b/plugins/tests/importers/fixtures/elevenlabs/tool_error.json new file mode 100644 index 000000000..42f986103 --- /dev/null +++ b/plugins/tests/importers/fixtures/elevenlabs/tool_error.json @@ -0,0 +1,914 @@ +{ + "agent_id": "agent_fixture_shop", + "agent_name": "Kitaru fixture shop", + "conversation_product": "agent", + "status": "done", + "user_id": null, + "branch_id": "branch_fixture_main", + "version_id": "version_fixture_one", + "metadata": { + "start_time_unix_secs": 1790959490, + "accepted_time_unix_secs": 1790959490, + "call_duration_secs": 24, + "queue_wait_secs": null, + "cost": 181, + "deletion_settings": { + "deletion_time_unix_secs": 1793551519, + "deleted_logs_at_time_unix_secs": null, + "deleted_audio_at_time_unix_secs": null, + "deleted_transcript_at_time_unix_secs": null, + "delete_transcript_and_pii": false, + "delete_audio": false + }, + "feedback": { + "type": null, + "overall_score": null, + "likes": 0, + "dislikes": 0, + "rating": null, + "comment": null + }, + "authorization_method": "signed_url", + "charging": { + "dev_discount": false, + "is_burst": false, + "tier": "free", + "llm_usage": { + "irreversible_generation": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 2389, + "price": 0.00150507 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 78, + "price": 0.00117 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 2389, + "price": 0.00150507 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 78, + "price": 0.00117 + } + } + } + ] + }, + "initiated_generation": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 4915, + "price": 0.00309645 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 119, + "price": 0.0017850000000000001 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 4915, + "price": 0.00309645 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 119, + "price": 0.0017850000000000001 + } + } + } + ] + } + }, + "llm_price": 0.004881450000000001, + "llm_charge": 49, + "call_charge": 132, + "platform_charge": 132, + "platform_usage": { + "category_usage": { + "voice": { + "credits": 132, + "price": 0.01307987531039398, + "quantity": 0.39939016092102975 + } + } + }, + "platform_price": 0.01307987531039398, + "free_minutes_consumed": 0.0, + "free_llm_dollars_consumed": 0.0, + "tts_usage": { + "primary_tts_model": "eleven_v4_turbo", + "total_audio_output_seconds": 18.679375, + "total_characters": 376, + "per_voice_usage": [ + { + "voice_id": "voice_fixture_one", + "audio_output_seconds": 18.679375 + } + ] + }, + "asr_usage": { + "asr_model": "scribe_realtime", + "total_transcription_calls": 33, + "total_audio_input_seconds": 6.288000000000002 + }, + "analysis": null + }, + "phone_call": null, + "batch_call": null, + "termination_reason": "Client disconnected: 1000", + "error": null, + "warnings": [], + "main_language": "en", + "rag_usage": null, + "text_only": false, + "features_usage": { + "language_detection": { + "enabled": false, + "used": false + }, + "transfer_to_agent": { + "enabled": false, + "used": false + }, + "transfer_to_number": { + "enabled": false, + "used": false + }, + "multivoice": { + "enabled": false, + "used": false + }, + "dtmf_tones": { + "enabled": false, + "used": false + }, + "external_mcp_servers": { + "enabled": false, + "used": false + }, + "pii_zrm_workspace": false, + "pii_zrm_agent": false, + "tool_dynamic_variable_updates": { + "enabled": false, + "used": false + }, + "is_livekit": false, + "voicemail_detection": { + "enabled": false, + "used": false + }, + "dtmf_input": { + "enabled": false, + "used": false + }, + "workflow": { + "enabled": false, + "tool_node": { + "enabled": false, + "used": false + }, + "standalone_agent_node": { + "enabled": false, + "used": false + }, + "phone_number_node": { + "enabled": false, + "used": false + }, + "end_node": { + "enabled": false, + "used": false + }, + "override_agent_node": { + "enabled": false, + "used": false + }, + "say_node": { + "enabled": false, + "used": false + }, + "update_state_node": { + "enabled": false, + "used": false + }, + "subgraph_node": { + "enabled": false, + "used": false + }, + "transfer_to_genesys_node": { + "enabled": false, + "used": false + } + }, + "agent_testing": { + "enabled": false, + "tests_ran_after_last_modification": false, + "tests_ran_in_last_7_days": false + }, + "versioning": { + "enabled": true, + "used": false + }, + "file_input": { + "enabled": true, + "used": false + }, + "freeform_procedure": { + "enabled": false, + "used": false + }, + "structured_procedure": { + "enabled": false, + "used": false + } + }, + "eleven_assistant": { + "is_eleven_assistant": false + }, + "initiator_id": null, + "conversation_initiation_source": "js_sdk", + "conversation_initiation_source_version": "1.26.0", + "timezone": "Europe/Amsterdam", + "async_metadata": null, + "whatsapp": null, + "sms": null, + "agent_created_from": "api", + "agent_last_updated_from": "api", + "voice_rewards": [], + "cost_fiat": 0.01796132531039398 + }, + "analysis": { + "evaluation_criteria_results": {}, + "data_collection_results": {}, + "evaluation_criteria_results_list": [], + "data_collection_results_list": [], + "call_successful": "success", + "call_success_score": null, + "transcript_summary": "The user inquired about order number 5000. The agent attempted to retrieve the order status but encountered a \"Deliberate lab failure: order service unavailable\" error. The agent informed the user that the order service was unavailable and advised them to try again later or use an alternative contact method. The user acknowledged this and ended the conversation.", + "call_summary_title": "Order Lookup Failure", + "scoped": [], + "sentiment_analysis": { + "overall_label": "neutral", + "overall_sentiment_score": 0.0, + "overall_frustration_score": 0.1, + "min_user_sentiment_score": 0.0, + "max_user_frustration_score": 0.0, + "num_scored_user_turns": 1 + } + }, + "visited_agents": [ + { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main" + } + ], + "conversation_initiation_client_data": { + "conversation_config_override": { + "asr": null, + "turn": null, + "tts": { + "model_id": null, + "voice_id": null, + "supported_voices": null, + "stability": null, + "speed": null, + "similarity_boost": null, + "pronunciation_dictionary_locators": null + }, + "conversation": { + "text_only": false, + "max_duration_seconds": null + }, + "agent": { + "first_message": null, + "language": null, + "max_conversation_duration_message": null, + "prompt": null + } + }, + "custom_llm_extra_body": {}, + "user_id": null, + "source_info": { + "source": "js_sdk", + "version": "1.26.0" + }, + "branch_id": null, + "environment": null, + "starting_workflow_node_id": null, + "experiments": [], + "procedure_ids": null, + "dynamic_variables": { + "system__agent_id": "agent_fixture_shop", + "system__agent_turns": 3, + "system__current_agent_turns": 3, + "system__current_subagent_turns": 3, + "system__current_agent_id": "agent_fixture_shop", + "system__conversation_id": "conv_fixture_tool_error", + "system__call_duration_secs": 23, + "system__time_utc": "2026-10-02T16:45:14.614061+00:00", + "system__timezone": "Europe/Amsterdam", + "system__time": "Friday, 18:45 02 October 2026", + "system__time_hour": 18, + "system__time_minute": 45, + "system__time_second": 14, + "system__is_text_only": false, + "system__channel": "js_sdk", + "system__caller_id": null, + "system__called_number": null, + "system__call_sid": null, + "system__call_id": null, + "system__conversation_history": "{\"x-elevenlabs-history\": true, \"entries\": [{\"role\": \"agent\", \"message\": \"Hi, this is the Kitaru test shop. How can I help you today?\"}, {\"role\": \"user\", \"message\": \"Um, hey. So what is happening with order number 5,000, please?\"}, {\"role\": \"agent\", \"tool_requests\": [{\"tool_name\": \"get_order\", \"params_as_json\": {\"order_id\": \"5000\"}}]}, {\"role\": \"agent\", \"tool_results\": [{\"tool_name\": \"get_order\", \"result_value\": \"Client tool execution failed: Deliberate lab failure: order service unavailable\", \"is_error\": true}]}, {\"role\": \"agent\", \"message\": \"I'm sorry, but our order service is currently unavailable, so I can't look up your order right now. Please try again later or contact us through another channel. Is there anything else I can help you with in the meantime?\"}, {\"role\": \"user\", \"message\": \"Uh, no, that's fine. Thanks.\"}, {\"role\": \"agent\", \"message\": \"You're...\"}]}", + "system__user_id": null, + "system__initiator_id": null, + "system__environment": "production" + }, + "tool_mock_config": { + "mocking_strategy": "none", + "fallback_strategy": "raise_error", + "mocked_tool_names": [] + }, + "tool_mock_overrides": {} + }, + "environment": "production", + "experiment_assignments": null, + "hiding_reason": null, + "smb_metadata": null, + "conversation_id": "conv_fixture_tool_error", + "has_audio": true, + "has_user_audio": true, + "has_response_audio": true, + "has_auxiliary_audio": false, + "transcript": [ + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": "Hi, this is the Kitaru test shop. How can I help you today?", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 0, + "conversation_turn_metrics": { + "metrics": { + "convai_tts_service_ttfb": { + "elapsed_time": 0.08731612702831626 + }, + "convai_ttf_audio_since_silence": { + "elapsed_time": 0.09453795664012432 + }, + "convai_turn_silence_before_initiation": { + "elapsed_time": 0.0 + } + }, + "convai_asr_provider": null, + "convai_tts_model": "eleven_v4_turbo", + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 1, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "user", + "agent_metadata": null, + "message": "Um, hey. So what is happening with order number 5,000, please?", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 4, + "conversation_turn_metrics": { + "metrics": { + "convai_asr_trailing_service_latency": { + "elapsed_time": 0.0463983379304409 + } + }, + "convai_asr_provider": "Scribe v2 Realtime", + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": "audio", + "source_event_id": 49, + "used_static_kb_document_ids": [], + "analysis": { + "user_sentiment_score": 0.0, + "user_frustration_score": 0.0 + }, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": null, + "multivoice_message": null, + "tool_calls": [ + { + "type": "client", + "request_id": "request_fixture_tool_error_1", + "tool_name": "get_order", + "params_as_json": "{\"order_id\": \"5000\"}", + "tool_has_been_called": true, + "tool_details": { + "type": "client", + "parameters": "{\"order_id\": \"5000\"}" + } + } + ], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 8, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttfb": { + "elapsed_time": 0.3120590392500162 + }, + "convai_llm_tool_request_generation_latency": { + "elapsed_time": 0.3127112090587616 + } + }, + "convai_asr_provider": null, + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1164, + "price": 0.0007333200000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 30, + "price": 0.00045 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1164, + "price": 0.0007333200000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 30, + "price": 0.00045 + } + } + } + ] + }, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 49, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": null, + "multivoice_message": null, + "tool_calls": [], + "tool_results": [ + { + "request_id": "request_fixture_tool_error_1", + "tool_name": "get_order", + "result_value": "Client tool execution failed: Deliberate lab failure: order service unavailable", + "is_error": true, + "is_blocked": false, + "tool_has_been_called": true, + "tool_latency_secs": 0.15653563383966684, + "error_type": "unknown", + "raw_error_message": "", + "dynamic_variable_updates": [], + "type": "client" + } + ], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 8, + "conversation_turn_metrics": null, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 49, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": "I'm sorry, but our order service is currently unavailable, so I can't look up your order right now. Please try again later or contact us through another channel. Is there anything else I can help you with in the meantime?", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 8, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttf_sentence": { + "elapsed_time": 0.2509329109452665 + }, + "convai_llm_service_ttfb": { + "elapsed_time": 0.19087178306654096 + }, + "convai_llm_service_tt_last_sentence": { + "elapsed_time": 0.39299305574968457 + }, + "convai_tts_service_ttfb": { + "elapsed_time": 0.094560616184026 + }, + "convai_ttf_audio_since_silence": { + "elapsed_time": 1.2356858350746334 + }, + "convai_turn_silence_before_initiation": { + "elapsed_time": 0.256 + } + }, + "convai_asr_provider": null, + "convai_tts_model": "eleven_v4_turbo", + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1225, + "price": 0.00077175 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 48, + "price": 0.00072 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1225, + "price": 0.00077175 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 48, + "price": 0.00072 + } + } + } + ] + }, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": null, + "source_event_id": 49, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "user", + "agent_metadata": null, + "message": "Uh, no, that's fine. Thanks.", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": null, + "time_in_call_secs": 22, + "conversation_turn_metrics": { + "metrics": { + "convai_asr_trailing_service_latency": { + "elapsed_time": 0.07359790708869696 + } + }, + "convai_asr_provider": "Scribe v2 Realtime", + "convai_tts_model": null, + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": null, + "interrupted": false, + "ignored_as_backchannel": false, + "original_message": null, + "reasoning": [], + "source_medium": "audio", + "source_event_id": 142, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + }, + { + "role": "agent", + "agent_metadata": { + "agent_id": "agent_fixture_shop", + "branch_id": "branch_fixture_main", + "workflow_node_id": null, + "version_id": "version_fixture_one" + }, + "message": "You're...", + "multivoice_message": null, + "tool_calls": [], + "tool_results": [], + "feedback": null, + "llm_override": null, + "producing_llm": "qwen35-397b-a17b", + "time_in_call_secs": 23, + "conversation_turn_metrics": { + "metrics": { + "convai_llm_service_ttf_sentence": { + "elapsed_time": 0.26585123129189014 + }, + "convai_llm_service_ttfb": { + "elapsed_time": 0.21633561328053474 + }, + "convai_llm_service_tt_last_sentence": { + "elapsed_time": 0.3013628199696541 + }, + "convai_tts_service_ttfb": { + "elapsed_time": 0.08856218680739403 + }, + "convai_ttf_audio_since_silence": { + "elapsed_time": 15.199356202762575 + }, + "convai_turn_silence_before_initiation": { + "elapsed_time": 14.72 + }, + "convai_turn_asr_latency": { + "elapsed_time": 0.07359790708869696 + } + }, + "convai_asr_provider": null, + "convai_tts_model": "eleven_v4_turbo", + "convai_tts_cascade": null + }, + "rag_retrieval_info": null, + "llm_usage": { + "model_usage": { + "qwen35-397b-a17b": { + "input": { + "tokens": 1288, + "price": 0.0008114400000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 22, + "price": 0.00033 + } + } + }, + "detailed_model_usage": [ + { + "config": { + "model": "qwen35-397b-a17b", + "service_tier": "standard" + }, + "usage": { + "input": { + "tokens": 1288, + "price": 0.0008114400000000001 + }, + "input_cache_read": { + "tokens": 0, + "price": 0.0 + }, + "input_cache_write": { + "tokens": 0, + "price": 0.0 + }, + "output_total": { + "tokens": 22, + "price": 0.00033 + } + } + } + ] + }, + "interrupted": true, + "ignored_as_backchannel": false, + "original_message": "You're welcome! Feel free to reach out anytime if you need further assistance. Have a great day!", + "reasoning": [], + "source_medium": null, + "source_event_id": 142, + "used_static_kb_document_ids": [], + "analysis": null, + "user_identifier": null, + "id": null, + "compaction": null, + "triggered_guardrails": [], + "file_input": null, + "file_inputs": [], + "contextual_update_info": null, + "config_snapshot_id": null, + "reasoned": false + } + ], + "tag_ids": [], + "realtime_config_snapshots": null, + "orchestrator": "conversational", + "otlp_traces": null +} diff --git a/plugins/tests/importers/fuzz_strategies.py b/plugins/tests/importers/fuzz_strategies.py index 64878938a..4a61de5b3 100644 --- a/plugins/tests/importers/fuzz_strategies.py +++ b/plugins/tests/importers/fuzz_strategies.py @@ -15,6 +15,7 @@ import copy import json +from importlib import import_module from pathlib import Path from types import ModuleType from typing import Any @@ -33,7 +34,10 @@ from kitaru.api_models.v1.session import SessionStatus from kitaru.api_models.v1.session_node import NodeStatus, NodeType +elevenlabs = import_module("kitaru_elevenlabs_importer.importer") + IMPORTERS: dict[str, ModuleType] = { + "elevenlabs": elevenlabs, "langfuse": langfuse, "braintrust": braintrust, "langsmith": langsmith, @@ -760,7 +764,69 @@ def record(draw: st.DrawFn) -> dict[str, Any]: return st.lists(record(), min_size=1, max_size=30) +@st.composite +def _elevenlabs_records(draw: st.DrawFn) -> list[dict[str, Any]]: + """Exercise real transcript and tool shapes with occasional hostile fields.""" + records = [] + for index in range(draw(st.integers(min_value=2, max_value=4))): + request_id = f"request-{index}" + message = _mostly(st.text(max_size=20), _WEIRD_TEXT) + tokens = _mostly( + st.integers(min_value=0, max_value=1_000), adversarial_json_value(1) + ) + record = { + "conversation_id": f"conversation-{index}", + "agent_id": "fixture-agent", + "status": "done", + "metadata": { + "start_time_unix_secs": 1_790_899_200, + "call_duration_secs": 5, + }, + "transcript": [ + {"role": "user", "message": draw(message), "time_in_call_secs": 0}, + { + "role": "agent", + "message": None, + "time_in_call_secs": 1, + "tool_calls": [ + { + "request_id": request_id, + "tool_name": "lookup", + "params_as_json": json.dumps(draw(_MASTRA_JSON)), + } + ], + "llm_usage": { + "model_usage": { + "fixture-model": { + "input": {"tokens": draw(tokens), "price": 0}, + "output_total": {"tokens": draw(tokens), "price": 0}, + } + } + }, + }, + { + "role": "agent", + "message": None, + "time_in_call_secs": 2, + "tool_results": [ + { + "request_id": request_id, + "tool_name": "lookup", + "result_value": draw(_JSON_IN_STRING), + "is_error": draw(st.booleans()), + "tool_latency_secs": 0.1, + } + ], + }, + {"role": "agent", "message": draw(message), "time_in_call_secs": 3}, + ], + } + records.append(record) + return records + + _RECORD_STRATEGIES = { + "elevenlabs": _elevenlabs_records, "langfuse": _langfuse_records, "braintrust": _braintrust_records, "langsmith": _langsmith_records, @@ -781,7 +847,7 @@ def encode_records(name: str, records: list[dict[str, Any]]) -> bytes: """Serialize records in the container shape each importer accepts.""" if name in {"langfuse", "logfire", "jsonl"}: return b"\n".join(json.dumps(r).encode() for r in records) - # braintrust, langsmith, mastra, mlflow, and phoenix accept a JSON array. + # The remaining importers accept a JSON array. return json.dumps(records).encode() @@ -797,6 +863,8 @@ def importer_params(name: str) -> SearchStrategy[dict[str, Any]]: record-normalizing code these properties are about. `invalid_params()` covers the rejection paths separately. """ + if name == "elevenlabs": + return st.just({}) if name == "mastra": return st.fixed_dictionaries( {}, @@ -837,6 +905,8 @@ def importer_params(name: str) -> SearchStrategy[dict[str, Any]]: def invalid_params(name: str) -> SearchStrategy[dict[str, Any]]: """Generate parameter dictionaries each importer should reject.""" + if name == "elevenlabs": + return st.fixed_dictionaries({"unknown": adversarial_json_value(1)}) if name == "mastra": return st.one_of( st.fixed_dictionaries({"unknown": adversarial_json_value(1)}), diff --git a/plugins/tests/importers/test_elevenlabs.py b/plugins/tests/importers/test_elevenlabs.py new file mode 100644 index 000000000..96462778d --- /dev/null +++ b/plugins/tests/importers/test_elevenlabs.py @@ -0,0 +1,876 @@ +"""ElevenLabs conversation parsing and read-only API import contracts.""" + +import copy +import json +from collections.abc import Callable +from datetime import UTC, datetime, timedelta +from decimal import Decimal +from functools import partial +from pathlib import Path +from typing import Any + +import httpx +import pytest + +import kitaru_elevenlabs_importer.api as api_module +from kitaru.api_models.v1.imports import ImportFailure +from kitaru.api_models.v1.session import SessionStatus +from kitaru.api_models.v1.session_node import ( + NodeStatus, + NodeType, + SessionNodeCreateRequest, +) +from kitaru.json_pointer import resolve_json_pointer +from kitaru.task.importer import ImportedSession, flatten_nodes +from kitaru_elevenlabs_importer.api import fetch +from kitaru_elevenlabs_importer.importer import InvalidImport, importer, parse + +FIXTURES = Path(__file__).parent / "fixtures" / "elevenlabs" +FIXTURE_NAMES = [ + "happy_path", + "multiple_tools", + "tool_error", + "interruption", + "not_found", +] + + +def _load(name: str) -> dict[str, Any]: + return json.loads((FIXTURES / f"{name}.json").read_bytes()) + + +def _parse(value: Any, **params: Any) -> list[ImportedSession | ImportFailure]: + return list(parse(json.dumps(value).encode(), params)) + + +def _get_session(value: Any) -> ImportedSession: + [item] = _parse(value) + assert isinstance(item, ImportedSession), item + return item + + +def _tools(session: ImportedSession) -> list[SessionNodeCreateRequest]: + return [ + node + for node in flatten_nodes(session.nodes) + if node.node_type == NodeType.TOOL_CALL + ] + + +def _conversation(conversation_id: str = "conv_test") -> dict[str, Any]: + return { + "agent_id": "agent_test", + "conversation_id": conversation_id, + "status": "done", + "metadata": {"start_time_unix_secs": 1_790_899_200, "call_duration_secs": 5}, + "transcript": [ + {"role": "user", "message": "Where is my order?", "time_in_call_secs": 0}, + {"role": "agent", "message": "It has shipped.", "time_in_call_secs": 2}, + ], + } + + +@pytest.mark.parametrize("name", FIXTURE_NAMES) +def test_real_voice_exports_preserve_session_messages_and_source_order( + name: str, +) -> None: + source = _load(name) + session = _get_session(source) + assert session.external_id == source["conversation_id"] + assert session.framework == "elevenlabs" + assert session.status == SessionStatus.COMPLETED + first_user = next( + turn["message"] for turn in source["transcript"] if turn["role"] == "user" + ) + last_agent = next( + turn["message"] + for turn in reversed(source["transcript"]) + if turn["role"] == "agent" and turn["message"] + ) + assert session.input_text_selector == "/message" + assert session.output_text_selector == "/message" + assert resolve_json_pointer(session.inputs, session.input_text_selector) == ( + True, + first_user, + ) + assert resolve_json_pointer(session.outputs, session.output_text_selector) == ( + True, + last_agent, + ) + nodes = flatten_nodes(session.nodes) + events = [node for node in nodes if node.node_type == NodeType.SPAN] + assert [node.external_id for node in events] == [ + f"{source['conversation_id']}:transcript:{index}" + for index in range(len(source["transcript"])) + ] + assert not any(node.node_type == NodeType.LLM_CALL for node in nodes) + assert len({node.external_id for node in nodes}) == len(nodes) + assert session.started_at == datetime.fromtimestamp( + source["metadata"]["start_time_unix_secs"], UTC + ) + assert session.started_at is not None + assert session.ended_at == session.started_at + timedelta( + seconds=source["metadata"]["call_duration_secs"] + ) + for node, turn in zip(events, source["transcript"], strict=True): + assert node.started_at == session.started_at + timedelta( + seconds=turn["time_in_call_secs"] + ) + if turn["message"]: + value, selector = ( + (node.inputs, node.input_text_selector) + if turn["role"] == "user" + else (node.outputs, node.output_text_selector) + ) + assert selector is not None + assert resolve_json_pointer(value, selector) == (True, turn["message"]) + session.model_dump_json() + + +def test_tool_arguments_results_and_parent_link_are_preserved() -> None: + source = _load("happy_path") + session = _get_session(source) + [tool] = _tools(session) + assert tool.tool_name == "get_order" + assert tool.inputs == {"order_id": "1001"} + assert tool.outputs == { + "found": True, + "order_id": "1001", + "status": "shipped", + "estimated_delivery": "October 7", + "product_type": "standard", + } + assert ( + tool.external_id == "conv_fixture_happy_path:tool:request_fixture_happy_path_1" + ) + assert tool.parent_external_id == "conv_fixture_happy_path:transcript:2" + assert tool.status == NodeStatus.COMPLETED + assert ( + tool.metadata["elevenlabs"]["tool_call"] + == source["transcript"][2]["tool_calls"][0] + ) + assert ( + tool.metadata["elevenlabs"]["tool_result"] + == source["transcript"][3]["tool_results"][0] + ) + + +def test_multiple_distinct_tools_preserve_order_and_arguments() -> None: + [order, policy] = _tools(_get_session(_load("multiple_tools"))) + assert [order.tool_name, policy.tool_name] == ["get_order", "get_return_policy"] + assert order.inputs == {"order_id": "1002"} + assert policy.inputs == {"product_type": "standard"} + assert order.status == policy.status == NodeStatus.COMPLETED + + +def test_repeated_tool_names_join_by_request_id_instead_of_name_or_position() -> None: + source = _load("not_found") + first = source["transcript"][3]["tool_results"][0] + second = source["transcript"][7]["tool_results"][0] + source["transcript"][2]["tool_calls"].extend(source["transcript"][6]["tool_calls"]) + source["transcript"][6]["tool_calls"] = [] + source["transcript"][3]["tool_results"] = [second, first] + source["transcript"][7]["tool_results"] = [] + [one, two] = _tools(_get_session(source)) + assert one.tool_name == two.tool_name == "get_order" + assert one.inputs == {"order_id": "4040"} + assert one.outputs == {"found": False, "order_id": "4040"} + assert two.inputs == {"order_id": "404040"} + assert two.outputs == {"found": False, "order_id": "404040"} + assert one.external_id != two.external_id + + +def test_execution_error_and_business_not_found_have_different_statuses() -> None: + failed_session = _get_session(_load("tool_error")) + [failed_tool] = _tools(failed_session) + assert failed_tool.status == NodeStatus.FAILED + assert failed_tool.error is not None + assert "order service unavailable" in failed_tool.error + assert failed_session.status == SessionStatus.COMPLETED + missing_tools = _tools(_get_session(_load("not_found"))) + assert all( + node.status == NodeStatus.COMPLETED and node.error is None + for node in missing_tools + ) + assert all(node.outputs["found"] is False for node in missing_tools) + + +def test_interruption_keeps_spoken_message_and_original_message_distinct() -> None: + source = _load("interruption") + session = _get_session(source) + node = next( + node for node in session.nodes if node.external_id.endswith(":transcript:2") + ) + assert node.outputs == { + "message": "[friendly] I can certainly help you with that!..." + } + metadata = node.metadata["elevenlabs"] + assert metadata["interrupted"] is True + assert metadata["original_message"] == source["transcript"][2]["original_message"] + assert metadata["original_message"] != node.outputs["message"] + + +@pytest.mark.parametrize("name", FIXTURE_NAMES) +def test_usage_uses_each_transcript_event_once_without_recounting_aggregates( + name: str, +) -> None: + source = _load(name) + session = _get_session(source) + nodes = flatten_nodes(session.nodes) + turns = [turn for turn in source["transcript"] if turn.get("llm_usage")] + expected_input = sum( + turn["llm_usage"]["model_usage"]["qwen35-397b-a17b"]["input"]["tokens"] + for turn in turns + ) + expected_output = sum( + turn["llm_usage"]["model_usage"]["qwen35-397b-a17b"]["output_total"]["tokens"] + for turn in turns + ) + expected_cost = sum( + ( + Decimal(str(category["price"])) + for turn in turns + for category in turn["llm_usage"]["model_usage"][ + "qwen35-397b-a17b" + ].values() + ), + Decimal(0), + ) + assert ( + sum(node.tokens.input_tokens or 0 for node in nodes if node.tokens) + == expected_input + ) + assert ( + sum(node.tokens.output_tokens or 0 for node in nodes if node.tokens) + == expected_output + ) + assert sum((node.cost or Decimal(0) for node in nodes), Decimal(0)) == expected_cost + assert expected_cost != Decimal(str(source["metadata"]["cost_fiat"])) + assert all(node.tokens is None and node.cost is None for node in _tools(session)) + assert all( + node.model_provider is None and node.requested_model is None for node in nodes + ) + assert {node.model for node in nodes if node.model} == {"qwen35-397b-a17b"} + assert ( + session.metadata["elevenlabs"]["provider_metadata"]["charging"] + == source["metadata"]["charging"] + ) + + +def test_absent_per_turn_usage_does_not_copy_conversation_totals_onto_nodes() -> None: + source = _load("happy_path") + for turn in source["transcript"]: + turn["llm_usage"] = None + session = _get_session(source) + assert all( + node.tokens is None and node.cost is None + for node in flatten_nodes(session.nodes) + ) + assert ( + session.metadata["elevenlabs"]["provider_metadata"]["cost_fiat"] + == source["metadata"]["cost_fiat"] + ) + + +def test_missing_prices_and_unknown_categories_do_not_invent_node_costs() -> None: + source = _load("happy_path") + for turn in source["transcript"]: + if not turn.get("llm_usage"): + continue + usage = turn["llm_usage"]["model_usage"]["qwen35-397b-a17b"] + for category in usage.values(): + category.pop("price") + usage["future_category"] = {"tokens": 100_000, "price": 999} + session = _get_session(source) + assert any(node.tokens is not None for node in session.nodes) + assert all(node.cost is None for node in flatten_nodes(session.nodes)) + node = next( + node for node in session.nodes if node.external_id.endswith(":transcript:2") + ) + assert node.metadata["elevenlabs"]["llm_usage"]["model_usage"]["qwen35-397b-a17b"][ + "future_category" + ] == {"tokens": 100_000, "price": 999} + + +def test_multiple_models_remain_raw_without_a_guessed_served_model() -> None: + source = _conversation() + source["transcript"][1]["llm_usage"] = { + "model_usage": { + "first-model": {"input": {"tokens": 12, "price": 0.1}}, + "second-model": {"output_total": {"tokens": 34, "price": 0.2}}, + } + } + session = _get_session(source) + node = session.nodes[1] + assert node.model is None and node.tokens is None and node.cost is None + assert ( + node.metadata["elevenlabs"]["llm_usage"] == source["transcript"][1]["llm_usage"] + ) + + +def test_cached_token_counts_preserve_zero_and_do_not_invent_reasoning_tokens() -> None: + source = _load("happy_path") + usage = source["transcript"][2]["llm_usage"]["model_usage"]["qwen35-397b-a17b"] + usage["input_cache_read"] = {"tokens": 25, "price": 0} + usage["input_cache_write"] = {"tokens": 31, "price": 0} + session = _get_session(source) + node = next( + node for node in session.nodes if node.external_id.endswith(":transcript:2") + ) + assert node.tokens is not None + assert node.tokens.cached_input_tokens == 25 + assert node.tokens.reasoning_tokens is None + assert node.tokens.input_tokens == 1218 + + +def test_audio_is_an_authenticated_provider_reference_without_downloading_data() -> ( + None +): + source = _load("happy_path") + session = _get_session(source) + metadata = session.metadata["elevenlabs"] + assert metadata["has_audio"] is True + assert metadata["has_user_audio"] is True + assert metadata["has_response_audio"] is True + assert ( + metadata["recording_url"] + == "https://api.elevenlabs.io/v1/convai/conversations/conv_fixture_happy_path/audio" + ) + source["has_audio"] = False + assert "recording_url" not in _get_session(source).metadata["elevenlabs"] + + +@pytest.mark.parametrize("container", ["single", "array", "conversations", "webhook"]) +def test_supported_containers_keep_stable_source_and_node_identity( + container: str, +) -> None: + source = _load("happy_path") + value: Any = source + if container == "array": + value = [source] + elif container == "conversations": + value = {"conversations": [source]} + elif container == "webhook": + value = { + "type": "post_call_transcription", + "event_timestamp": 1_759_440_100, + "data": source, + } + session = _get_session(value) + expected = _get_session(source) + assert session.external_id == expected.external_id + assert flatten_nodes(session.nodes) == flatten_nodes(expected.nodes) + assert session.inputs == expected.inputs + assert session.outputs == expected.outputs + assert _get_session(value) == session + + +def test_array_keeps_all_five_conversations_separate() -> None: + items = _parse([_load(name) for name in FIXTURE_NAMES]) + assert len(items) == 5 + assert all(isinstance(item, ImportedSession) for item in items) + assert {item.external_id for item in items} == { + f"conv_fixture_{name}" for name in FIXTURE_NAMES + } + + +@pytest.mark.parametrize( + "params", + [{"join_on": "agent_id"}, {"source_namespace": "tenant"}, {"download_audio": True}], +) +def test_unknown_parser_parameters_are_rejected(params: dict[str, Any]) -> None: + with pytest.raises(InvalidImport): + list(parse(json.dumps(_conversation()).encode(), params)) + + +@pytest.mark.parametrize("payload", [b"{", b"\xff\xfe", b"[]{}"]) +def test_unreadable_documents_raise_the_public_parser_exception(payload: bytes) -> None: + with pytest.raises(InvalidImport): + list(parse(payload, {})) + + +@pytest.mark.parametrize( + ("field", "value"), + [ + ("conversation_id", None), + ("conversation_id", ""), + ("conversation_id", False), + ("transcript", None), + ("transcript", {}), + ("transcript", [False]), + ("transcript", [{"role": "unknown", "message": "hello"}]), + ("transcript", [{"role": "user", "message": []}]), + ("metadata", False), + ], +) +def test_malformed_conversation_is_isolated_from_healthy_neighbors( + field: str, value: Any +) -> None: + broken = _conversation("conv_broken") + broken[field] = value + items = _parse([_conversation("conv_before"), broken, _conversation("conv_after")]) + assert [ + item.external_id for item in items if isinstance(item, ImportedSession) + ] == ["conv_before", "conv_after"] + failures = [item for item in items if isinstance(item, ImportFailure)] + assert len(failures) == 1 + for item in items: + item.model_dump_json() + + +@pytest.mark.parametrize( + "mutation", + [ + "missing_call_id", + "duplicate_call_id", + "duplicate_result_id", + ], +) +def test_tool_structure_errors_fail_only_the_containing_conversation( + mutation: str, +) -> None: + source = _load("happy_path") + call = source["transcript"][2]["tool_calls"][0] + result = source["transcript"][3]["tool_results"][0] + if mutation == "missing_call_id": + del call["request_id"] + elif mutation == "duplicate_call_id": + source["transcript"][2]["tool_calls"].append(copy.deepcopy(call)) + elif mutation == "duplicate_result_id": + source["transcript"][3]["tool_results"].append(copy.deepcopy(result)) + items = _parse([source, _conversation("conv_healthy")]) + assert len(items) == 2 + assert isinstance(items[0], ImportFailure) + assert isinstance(items[1], ImportedSession) + assert items[1].external_id == "conv_healthy" + + +def test_missing_tool_result_remains_explicitly_incomplete() -> None: + source = _load("happy_path") + source["transcript"][3]["tool_results"] = [] + [tool] = _tools(_get_session(source)) + assert tool.inputs == {"order_id": "1001"} + assert tool.outputs is None + assert tool.status == NodeStatus.IN_PROGRESS + assert tool.metadata["elevenlabs"]["result_missing"] is True + + +def test_orphan_tool_result_does_not_invent_arguments_or_parent_call() -> None: + source = _load("happy_path") + source["transcript"][2]["tool_calls"] = [] + [tool] = _tools(_get_session(source)) + assert tool.inputs is None + assert tool.outputs["found"] is True + assert tool.status == NodeStatus.COMPLETED + assert tool.parent_external_id != "conv_fixture_happy_path:transcript:2" + + +def test_non_json_tool_values_remain_raw_text() -> None: + source = _load("happy_path") + source["transcript"][2]["tool_calls"][0]["params_as_json"] = "{unparsed source" + source["transcript"][3]["tool_results"][0]["result_value"] = ( + "Order service is unavailable." + ) + [tool] = _tools(_get_session(source)) + assert tool.inputs == "{unparsed source" + assert tool.outputs == "Order service is unavailable." + assert tool.output_text_selector == "" + + +def test_identical_conversations_are_deduplicated() -> None: + source = _conversation() + [session] = _parse([source, copy.deepcopy(source)]) + assert isinstance(session, ImportedSession) + assert session.external_id == "conv_test" + + +@pytest.mark.parametrize("event_type", ["post_call_audio", "call_initiation_failure"]) +def test_non_transcription_webhooks_are_rejected(event_type: str) -> None: + with pytest.raises(InvalidImport): + _parse({"type": event_type, "data": _conversation()}) + + +def test_conflicting_conversation_ids_are_order_independent() -> None: + source = _conversation() + changed = copy.deepcopy(source) + changed["transcript"][-1]["message"] = "It was cancelled." + for order in ([source, changed], [changed, source]): + items = _parse([*order, _conversation("conv_healthy")]) + assert [ + item.external_id for item in items if isinstance(item, ImportedSession) + ] == ["conv_healthy"] + failures = [item for item in items if isinstance(item, ImportFailure)] + assert len(failures) == 1 + assert failures[0].external_id == "conv_test" + + +def _mock_api( + monkeypatch: pytest.MonkeyPatch, + handler: Callable[[httpx.Request], httpx.Response], +) -> list[httpx.Request]: + requests: list[httpx.Request] = [] + + def respond(request: httpx.Request) -> httpx.Response: + requests.append(request) + return handler(request) + + client = httpx.AsyncClient + monkeypatch.setattr( + api_module.httpx, + "AsyncClient", + partial(client, transport=httpx.MockTransport(respond)), + ) + monkeypatch.setenv("ELEVENLABS_API_KEY", "elevenlabs-fixture-secret") + return requests + + +async def _fetch(query: dict[str, Any]) -> list[dict[str, Any]]: + return [json.loads(payload) async for payload in fetch(query)] + + +async def test_api_uses_environment_credential_and_exact_explicit_ids( + monkeypatch: pytest.MonkeyPatch, +) -> None: + requests = _mock_api( + monkeypatch, + lambda request: httpx.Response( + 200, json=_conversation(request.url.path.rsplit("/", 1)[-1]) + ), + ) + records = await _fetch( + {"trace_ids": ["conv_second", "conv_first"], "concurrency": 1} + ) + assert [record["conversation_id"] for record in records] == [ + "conv_second", + "conv_first", + ] + assert [request.url.path for request in requests] == [ + "/v1/convai/conversations/conv_second", + "/v1/convai/conversations/conv_first", + ] + assert all( + request.headers["xi-api-key"] == "elevenlabs-fixture-secret" + for request in requests + ) + assert all( + request.url.host == "api.elevenlabs.io" and not request.url.query + for request in requests + ) + items = [item for record in records for item in _parse(record)] + assert all(isinstance(item, ImportedSession) for item in items) + + +async def test_api_pagination_preserves_time_and_agent_filters( + monkeypatch: pytest.MonkeyPatch, +) -> None: + def respond(request: httpx.Request) -> httpx.Response: + if request.url.path == "/v1/convai/conversations": + if request.url.params.get("cursor") == "page-two": + return httpx.Response( + 200, + json={ + "conversations": [ + {"conversation_id": "conv_second", "status": "failed"} + ], + "has_more": False, + "next_cursor": None, + }, + ) + return httpx.Response( + 200, + json={ + "conversations": [ + {"conversation_id": "conv_first", "status": "done"}, + {"conversation_id": "conv_live", "status": "in-progress"}, + {"conversation_id": "conv_processing", "status": "processing"}, + ], + "has_more": True, + "next_cursor": "page-two", + }, + ) + record = _conversation(request.url.path.rsplit("/", 1)[-1]) + if record["conversation_id"] == "conv_second": + record["status"] = "failed" + return httpx.Response(200, json=record) + + requests = _mock_api(monkeypatch, respond) + records = await _fetch( + { + "since": "2026-10-02T00:00:00Z", + "until": "2026-10-03T00:00:00Z", + "agent_id": "agent_test", + "page_size": 2, + "concurrency": 1, + } + ) + assert [record["conversation_id"] for record in records] == [ + "conv_first", + "conv_second", + ] + listings = [ + request + for request in requests + if request.url.path == "/v1/convai/conversations" + ] + assert len(listings) == 2 + for request in listings: + assert request.url.params["agent_id"] == "agent_test" + assert request.url.params["page_size"] == "2" + assert request.url.params["call_start_after_unix"] == "1790899200" + assert request.url.params["call_start_before_unix"] == "1790985600" + assert listings[1].url.params["cursor"] == "page-two" + assert not any( + request.url.path.endswith(("/conv_live", "/conv_processing", "/audio")) + for request in requests + ) + + +async def test_api_empty_listing_yields_no_payloads( + monkeypatch: pytest.MonkeyPatch, +) -> None: + _mock_api( + monkeypatch, + lambda request: httpx.Response( + 200, json={"conversations": [], "has_more": False, "next_cursor": None} + ), + ) + assert await _fetch({"since": "2026-10-02T00:00:00Z"}) == [] + + +async def test_api_filters_rounded_provider_window_with_exact_detail_times( + monkeypatch: pytest.MonkeyPatch, +) -> None: + start_times = { + "conv_before": 1_790_899_200, + "conv_lower": 1_790_899_200.5, + "conv_inside": 1_790_899_201, + "conv_upper": 1_790_899_202.5, + "conv_after": 1_790_899_203, + } + + def respond(request: httpx.Request) -> httpx.Response: + if request.url.path == "/v1/convai/conversations": + return httpx.Response( + 200, + json={ + "conversations": [ + {"conversation_id": identifier, "status": "done"} + for identifier in start_times + ], + "has_more": False, + }, + ) + identifier = request.url.path.rsplit("/", 1)[-1] + record = _conversation(identifier) + record["metadata"]["start_time_unix_secs"] = start_times[identifier] + return httpx.Response(200, json=record) + + requests = _mock_api(monkeypatch, respond) + records = await _fetch( + {"since": "2026-10-02T00:00:00.500000Z", "until": "2026-10-02T00:00:02.500000Z"} + ) + assert [record["conversation_id"] for record in records] == [ + "conv_lower", + "conv_inside", + ] + listing = requests[0] + assert listing.url.params["call_start_after_unix"] == "1790899200" + assert listing.url.params["call_start_before_unix"] == "1790899203" + + +@pytest.mark.parametrize("cursor", [None, ""]) +async def test_api_rejects_missing_cursor_when_more_pages_are_claimed( + monkeypatch: pytest.MonkeyPatch, + cursor: str | None, +) -> None: + _mock_api( + monkeypatch, + lambda request: httpx.Response( + 200, json={"conversations": [], "has_more": True, "next_cursor": cursor} + ), + ) + with pytest.raises(ValueError, match="cursor"): + await _fetch({"since": "2026-10-02T00:00:00Z"}) + + +async def test_api_rejects_repeated_cursor_without_looping( + monkeypatch: pytest.MonkeyPatch, +) -> None: + requests = _mock_api( + monkeypatch, + lambda request: httpx.Response( + 200, + json={"conversations": [], "has_more": True, "next_cursor": "same-cursor"}, + ), + ) + with pytest.raises(ValueError, match="cursor"): + await _fetch({"since": "2026-10-02T00:00:00Z"}) + assert len(requests) == 2 + + +async def test_api_bounds_empty_pages_with_fresh_cursors( + monkeypatch: pytest.MonkeyPatch, +) -> None: + pages = 0 + + def respond(request: httpx.Request) -> httpx.Response: + nonlocal pages + pages += 1 + assert pages <= 2, "Empty pagination exceeded the request bound" + return httpx.Response( + 200, + json={ + "conversations": [], + "has_more": True, + "next_cursor": f"page-{pages}", + }, + ) + + requests = _mock_api(monkeypatch, respond) + with pytest.raises(ValueError, match="pagination exceeded page-request limit"): + await _fetch({"since": "2026-10-02T00:00:00Z", "limit": 2}) + assert len(requests) == 2 + assert requests[1].url.params["cursor"] == "page-1" + + +async def test_api_limits_scanned_conversations_including_unfinished_calls( + monkeypatch: pytest.MonkeyPatch, +) -> None: + requests = _mock_api( + monkeypatch, + lambda request: httpx.Response( + 200, + json={ + "conversations": [ + {"conversation_id": "conv_live", "status": "in-progress"} + ], + "has_more": True, + "next_cursor": "more", + }, + ), + ) + assert await _fetch({"since": "2026-10-02T00:00:00Z", "limit": 1}) == [] + assert len(requests) == 1 + assert requests[0].url.params["page_size"] == "1" + + +async def test_api_retries_rate_limit_without_exposing_provider_body( + monkeypatch: pytest.MonkeyPatch, +) -> None: + attempts = 0 + + def respond(request: httpx.Request) -> httpx.Response: + nonlocal attempts + attempts += 1 + if attempts == 1: + return httpx.Response( + 429, headers={"retry-after": "0"}, text="private diagnostic" + ) + return httpx.Response(200, json=_conversation()) + + _mock_api(monkeypatch, respond) + [record] = await _fetch({"trace_ids": ["conv_test"]}) + assert record["conversation_id"] == "conv_test" + assert attempts == 2 + + +async def test_api_rejects_a_different_returned_conversation_id( + monkeypatch: pytest.MonkeyPatch, +) -> None: + _mock_api( + monkeypatch, + lambda request: httpx.Response(200, json=_conversation("conv_wrong")), + ) + with pytest.raises(ValueError, match="conversation_id"): + await _fetch({"trace_ids": ["conv_requested"]}) + + +@pytest.mark.parametrize( + "response", + [httpx.Response(200, text="private broken JSON"), httpx.Response(200, json=[])], +) +async def test_api_rejects_unreadable_responses_without_printing_payloads( + monkeypatch: pytest.MonkeyPatch, + response: httpx.Response, +) -> None: + _mock_api(monkeypatch, lambda request: response) + with pytest.raises(ValueError) as error: + await _fetch({"trace_ids": ["conv_test"]}) + assert "private broken JSON" not in str(error.value) + + +async def test_importer_instance_exposes_the_same_fetch_and_parse_contract( + monkeypatch: pytest.MonkeyPatch, +) -> None: + _mock_api(monkeypatch, lambda request: httpx.Response(200, json=_conversation())) + [payload] = [ + payload async for payload in importer.fetch({"trace_ids": ["conv_test"]}) + ] + [session] = list(importer.parse(payload, {})) + assert isinstance(session, ImportedSession) + assert session.external_id == "conv_test" + + +@pytest.mark.parametrize( + "query", + [ + {}, + {"conversation_ids": ["conv_test"]}, + {"trace_ids": []}, + {"trace_ids": [False]}, + {"trace_ids": ["conv_test"], "api_key": "do-not-send"}, + {"since": "2026-10-02"}, + {"since": "2026-10-03T00:00:00Z", "until": "2026-10-02T00:00:00Z"}, + {"trace_ids": ["conv_test"], "concurrency": 0}, + {"trace_ids": ["conv_test"], "page_size": 101}, + ], +) +async def test_api_rejects_invalid_queries_before_network_access( + monkeypatch: pytest.MonkeyPatch, query: dict[str, Any] +) -> None: + requests = _mock_api( + monkeypatch, lambda request: pytest.fail("Invalid query made a network request") + ) + with pytest.raises(ValueError): + await _fetch(query) + assert requests == [] + + +async def test_api_missing_environment_credential_is_clear_and_offline( + monkeypatch: pytest.MonkeyPatch, +) -> None: + requests = _mock_api( + monkeypatch, + lambda request: pytest.fail("Missing credential made a network request"), + ) + monkeypatch.delenv("ELEVENLABS_API_KEY", raising=False) + with pytest.raises(ValueError, match="ELEVENLABS_API_KEY"): + await _fetch({"trace_ids": ["conv_test"]}) + assert requests == [] + + +@pytest.mark.parametrize("status", [401, 403, 404, 500]) +async def test_api_http_errors_do_not_turn_into_empty_success( + monkeypatch: pytest.MonkeyPatch, status: int +) -> None: + _mock_api( + monkeypatch, + lambda request: httpx.Response(status, json={"detail": "fixture error"}), + ) + with pytest.raises(ValueError, match=f"HTTP {status}") as error: + await _fetch({"trace_ids": ["conv_test"]}) + assert "fixture error" not in str(error.value) + assert "elevenlabs-fixture-secret" not in str(error.value) + + +@pytest.mark.parametrize("status", ["initiated", "in-progress", "processing"]) +async def test_api_explicit_ids_do_not_import_partial_conversations( + monkeypatch: pytest.MonkeyPatch, status: str +) -> None: + record = _conversation() + record["status"] = status + _mock_api(monkeypatch, lambda request: httpx.Response(200, json=record)) + [record] = await _fetch({"trace_ids": ["conv_test"]}) + [failure] = _parse(record) + assert isinstance(failure, ImportFailure) + assert failure.external_id == "conv_test" diff --git a/plugins/tests/importers/test_fuzz_parse.py b/plugins/tests/importers/test_fuzz_parse.py index 0fe88087f..f6a621855 100644 --- a/plugins/tests/importers/test_fuzz_parse.py +++ b/plugins/tests/importers/test_fuzz_parse.py @@ -83,7 +83,7 @@ def test_parse_contract_on_records(name: str, data: st.DataObject) -> None: def test_parse_contract_on_invalid_params(name: str, data: st.DataObject) -> None: """Cover the parameter-validation branches `importer_params()` avoids.""" params = data.draw(invalid_params(name)) - if name == "mastra": + if name in {"elevenlabs", "mastra"}: with pytest.raises(IMPORTERS[name].InvalidImport): list(IMPORTERS[name].parse(b"{}", params)) return diff --git a/plugins/uv.lock b/plugins/uv.lock index d8106eace..13d4a887f 100644 --- a/plugins/uv.lock +++ b/plugins/uv.lock @@ -19,6 +19,7 @@ exclude-newer-span = "P3D" members = [ "kitaru-braintrust-importer", "kitaru-claude-agent-sdk", + "kitaru-elevenlabs-importer", "kitaru-evaluator", "kitaru-jsonl-importer", "kitaru-langfuse-importer", @@ -41,6 +42,7 @@ dev = [ { name = "kitaru", extras = ["server"], editable = "../" }, { name = "kitaru-braintrust-importer", extras = ["adapter"], editable = "packages/braintrust-importer" }, { name = "kitaru-claude-agent-sdk", editable = "packages/claude-agent-sdk" }, + { name = "kitaru-elevenlabs-importer", editable = "packages/elevenlabs-importer" }, { name = "kitaru-langfuse-importer", extras = ["adapter"], editable = "packages/langfuse-importer" }, { name = "kitaru-langgraph", extras = ["deepagents"], editable = "packages/langgraph" }, { name = "kitaru-langsmith-importer", extras = ["adapter"], editable = "packages/langsmith-importer" }, @@ -1465,6 +1467,18 @@ requires-dist = [ { name = "kitaru", editable = "../" }, ] +[[package]] +name = "kitaru-elevenlabs-importer" +version = "0.1.0" +source = { editable = "packages/elevenlabs-importer" } +dependencies = [ + { name = "kitaru" }, +] + +[package.metadata] +requires-dist = [{ name = "kitaru", editable = "../" }] +provides-extras = ["api"] + [[package]] name = "kitaru-evaluator" version = "0.2.0" diff --git a/release/release-units.toml b/release/release-units.toml index dd830d2f5..a0cd4b847 100644 --- a/release/release-units.toml +++ b/release/release-units.toml @@ -83,6 +83,21 @@ tag-prefix = "python/kitaru-claude-agent-sdk/v" maintenance-branch-prefix = "release/claude-agent-sdk/" checks = [] +[[units]] +slug = "elevenlabs-importer" +path = "plugins/packages/elevenlabs-importer" +distribution = "kitaru-elevenlabs-importer" +registry = "pypi" +version-source = "plugins/packages/elevenlabs-importer/pyproject.toml" +changelog = "plugins/packages/elevenlabs-importer/CHANGELOG.md" +lock-source = "plugins/uv.lock" +default-catalog = false +release-label = "requires:plugin:elevenlabs" +impact-paths = ["plugins/packages/elevenlabs-importer/**"] +tag-prefix = "python/kitaru-elevenlabs-importer/v" +maintenance-branch-prefix = "release/elevenlabs/" +checks = [] + [[units]] slug = "evaluator" path = "plugins/packages/evaluator" diff --git a/src/kitaru/worker/process.py b/src/kitaru/worker/process.py index dde628a8e..41de1d064 100644 --- a/src/kitaru/worker/process.py +++ b/src/kitaru/worker/process.py @@ -77,6 +77,7 @@ { "kitaru-braintrust-importer", "kitaru-claude-agent-sdk", + "kitaru-elevenlabs-importer", "kitaru-evaluator", "kitaru-jsonl-importer", "kitaru-langfuse-importer", diff --git a/tests/scripts/test_release_units.py b/tests/scripts/test_release_units.py index bc34b7e65..a3b708e7c 100644 --- a/tests/scripts/test_release_units.py +++ b/tests/scripts/test_release_units.py @@ -30,6 +30,7 @@ "kitaru": "kitaru", "braintrust-importer": "kitaru-braintrust-importer", "claude-agent-sdk": "kitaru-claude-agent-sdk", + "elevenlabs-importer": "kitaru-elevenlabs-importer", "evaluator": "kitaru-evaluator", "jsonl-importer": "kitaru-jsonl-importer", "langfuse-importer": "kitaru-langfuse-importer", @@ -629,7 +630,7 @@ def test_plugin_matrix_is_generated_from_the_plugin_units_in_three_shards() -> N shards = matrix["include"] assert [shard["shard"] for shard in shards] == ["1/3", "2/3", "3/3"] - assert [len(shard["package_paths"].splitlines()) for shard in shards] == [5, 5, 5] + assert [len(shard["package_paths"].splitlines()) for shard in shards] == [6, 5, 5] assert [ package_path for shard in shards @@ -895,7 +896,7 @@ def _run_cli(*arguments: str) -> subprocess.CompletedProcess[str]: [ (["list"], "SLUG\tDISTRIBUTION\tVERSION\tDEFAULT\tTAG"), (["resolve", "--unit", "kitaru"], "python/kitaru/v"), - (["validate"], "Validated 16 release units."), + (["validate"], "Validated 17 release units."), ( [ "propose-core-version",