Skip to content

Evolve observability documentation #797

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Mar 26, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -109,9 +109,8 @@ jobs:

# Runs the sdk features repo tests with this repo's current SDK code
features-tests:
uses: temporalio/features/.github/workflows/python.yaml@uv
uses: temporalio/features/.github/workflows/python.yaml@main
with:
python-repo-path: ${{github.event.pull_request.head.repo.full_name}}
version: ${{github.event.pull_request.head.ref}}
version-is-repo-ref: true
features-repo-ref: uv
30 changes: 26 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,9 @@ informal introduction to the features and their implementation.
- [Worker Shutdown](#worker-shutdown)
- [Testing](#testing-1)
- [Workflow Replay](#workflow-replay)
- [OpenTelemetry Support](#opentelemetry-support)
- [Observability](#observability)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May need to run npx doctoc README.md, unsure

- [Metrics](#metrics)
- [OpenTelemetry Tracing](#opentelemetry-tracing)
- [Protobuf 3.x vs 4.x](#protobuf-3x-vs-4x)
- [Known Compatibility Issues](#known-compatibility-issues)
- [gevent Patching](#gevent-patching)
Expand Down Expand Up @@ -1344,10 +1346,30 @@ async def check_past_histories(my_client: Client):
)
```

### OpenTelemetry Support
### Observability

OpenTelemetry support requires the optional `opentelemetry` dependencies which are part of the `opentelemetry` extra.
When using `pip`, running
See https://github.com/temporalio/samples-python/tree/main/open_telemetry for a sample demonstrating collection of
metrics and tracing data emitted by the SDK.

#### Metrics

The SDK emits various metrics by default: see https://docs.temporal.io/references/sdk-metrics. To configure additional
attributes to be emitted with all metrics, pass
[global_tags](https://python.temporal.io/temporalio.runtime.TelemetryConfig.html#global_tags) when creating the
[TelemetryConfig](https://python.temporal.io/temporalio.runtime.TelemetryConfig.html).

For emitting custom metrics, the SDK makes a metric meter available:
- In Workflow code, use https://python.temporal.io/temporalio.workflow.html#metric_meter
- In Activity code, use https://python.temporal.io/temporalio.activity.html#metric_meter
- In normal application code, use https://python.temporal.io/temporalio.runtime.Runtime.html#metric_meter
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a bit confused, where does normal application code exist if not in a workflow or activity?

Copy link
Member

@cretz cretz Mar 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where they use the client in non-activity situations, e.g. a client interceptor or anywhere they use the client (but really they probably should use their favorite metrics client/system instead of using Temporal's, but meh)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way I describe it to people is that a codebase using Temporal has 3 distinct categories of code that involve Temporal: workflow code, activity code, and "normal application code" that is using a Temporal client to start and interact with workflows, issue other queries, create schedules etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks both, helpful :)


The attributes emitted by these default to `namespace`, `task_queue`, and `workflow_type`/`activity_type`; use
`with_additional_attributes` to create a meter emitting additional attributes.

#### OpenTelemetry Tracing

Tracing support requires the optional `opentelemetry` dependencies which are part of the `opentelemetry` extra. When
using `pip`, running

pip install 'temporalio[opentelemetry]'

Expand Down
25 changes: 21 additions & 4 deletions tests/worker/test_activity.py
Original file line number Diff line number Diff line change
Expand Up @@ -611,7 +611,12 @@ async def some_activity(param1: SomeClass2, param2: str) -> str:
assert activity_param1 == SomeClass2(foo="str1", bar=SomeClass1(foo=123))


async def test_activity_heartbeat_details(client: Client, worker: ExternalWorker):
async def test_activity_heartbeat_details(
client: Client, worker: ExternalWorker, env: WorkflowEnvironment
):
if env.supports_time_skipping:
pytest.skip("https://github.com/temporalio/sdk-java/issues/2459")

@activity.defn
async def some_activity() -> str:
info = activity.info()
Expand Down Expand Up @@ -698,8 +703,11 @@ def picklable_heartbeat_details_activity() -> str:


async def test_sync_activity_thread_heartbeat_details(
client: Client, worker: ExternalWorker
client: Client, worker: ExternalWorker, env: WorkflowEnvironment
):
if env.supports_time_skipping:
pytest.skip("https://github.com/temporalio/sdk-java/issues/2459")

with concurrent.futures.ThreadPoolExecutor(
max_workers=default_max_concurrent_activities
) as executor:
Expand All @@ -714,8 +722,11 @@ async def test_sync_activity_thread_heartbeat_details(


async def test_sync_activity_process_heartbeat_details(
client: Client, worker: ExternalWorker
client: Client, worker: ExternalWorker, env: WorkflowEnvironment
):
if env.supports_time_skipping:
pytest.skip("https://github.com/temporalio/sdk-java/issues/2459")

with concurrent.futures.ProcessPoolExecutor() as executor:
result = await _execute_workflow_with_activity(
client,
Expand Down Expand Up @@ -1066,8 +1077,14 @@ async def test_activity_async_success(

@pytest.mark.parametrize("use_task_token", [True, False])
async def test_activity_async_heartbeat_and_fail(
client: Client, worker: ExternalWorker, use_task_token: bool
client: Client,
worker: ExternalWorker,
env: WorkflowEnvironment,
use_task_token: bool,
):
if env.supports_time_skipping:
pytest.skip("https://github.com/temporalio/sdk-java/issues/2459")

wrapper = AsyncActivityWrapper()
# Start task w/ max attempts 2, wait for info, send heartbeat, fail
task = asyncio.create_task(
Expand Down
2 changes: 1 addition & 1 deletion tests/worker/test_workflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,7 @@ async def test_workflow_info(client: Client, env: WorkflowEnvironment):
assert info["retry_policy"] == json.loads(
json.dumps(dataclasses.asdict(retry_policy), default=str)
)
assert uuid.UUID(info["run_id"]).version == 4
assert uuid.UUID(info["run_id"]).version == 7
assert info["run_timeout"] is None
datetime.fromisoformat(info["start_time"])
assert info["task_queue"] == worker.task_queue
Expand Down
Loading