Skip to content

feat: opentelemetry sdk LoggerProvider #207

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 39 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
dbda356
feat: LogRecord limits
yuzurihaaa Jan 16, 2025
630f2cf
feat: LogRecord
yuzurihaaa Jan 16, 2025
7119f8e
chore: fix comment
yuzurihaaa Jan 16, 2025
6fb8422
chore: log record limit test
yuzurihaaa Jan 16, 2025
69e4e18
feat: LogRecordProcessor
yuzurihaaa Jan 16, 2025
79640fb
feat: NoopLogRecordProcessor
yuzurihaaa Jan 16, 2025
8ff6c71
feat: LoggerConfig
yuzurihaaa Jan 16, 2025
d3d98e6
feat: Logger
yuzurihaaa Jan 16, 2025
f4b05aa
feat: LoggerProvider
yuzurihaaa Jan 16, 2025
ff5d1e3
feat: exports
yuzurihaaa Jan 16, 2025
1e050b4
wip: change processors initialization from `const`
yuzurihaaa Jan 16, 2025
5eed62e
wip: fix test
yuzurihaaa Jan 16, 2025
99e374f
wip: fix import arrange
yuzurihaaa Jan 20, 2025
38b2137
wip: attach copyright
yuzurihaaa Jan 20, 2025
88eb916
wip: change from `api.SpanContext?` and `api.Context?` to non-null pr…
yuzurihaaa Jan 31, 2025
014f087
wip: remove unused `includeTraceContext`
yuzurihaaa Jan 31, 2025
96938b8
wip: opentelemetry as defaultLoggerName
yuzurihaaa Jan 31, 2025
2a9a80a
wip: make log processors immutable
yuzurihaaa Jan 31, 2025
f0206c6
wip: make all variables private
yuzurihaaa Jan 31, 2025
01636f9
wip: update limit to only create if value changes.
yuzurihaaa Jan 31, 2025
aae1c3e
wip: change from sdk.Attributes to List<api.Attribute>
yuzurihaaa Jan 31, 2025
464a636
wip: change to `DateTime`
yuzurihaaa Feb 10, 2025
2ea0fcf
wip: remove DateTime initialization
yuzurihaaa Feb 10, 2025
078bd4a
wip: fix test
yuzurihaaa Feb 10, 2025
90ecfdc
wip: remove `processors`
yuzurihaaa Feb 12, 2025
4ab30a2
wip: remove `async` `await`
yuzurihaaa Feb 12, 2025
b3f744e
wip: changed to `FutureOr`
yuzurihaaa Feb 12, 2025
9978329
wip: change forceFlush and shutdown to `void`
yuzurihaaa Feb 13, 2025
849e0b0
wip: fix test
yuzurihaaa Feb 13, 2025
bebc35e
wip: change LogRecordLimits from abstract class to class
yuzurihaaa Mar 3, 2025
032fd65
wip: avoid nullable values
yuzurihaaa Mar 3, 2025
442b466
wip: mark protected
yuzurihaaa Mar 3, 2025
3fc382e
wip: remove nullable and pass processors instead of callback.
yuzurihaaa Mar 3, 2025
f0e65ab
fix: failing unit test
yuzurihaaa Mar 3, 2025
b97de95
wip: remove NoopLogRecordProcessor
yuzurihaaa Mar 4, 2025
d9c0429
wip: inline
yuzurihaaa Mar 4, 2025
37fae26
wip: limit list
yuzurihaaa Mar 4, 2025
b9598de
wip: onEmit ReadableLogRecord -> ReadWriteLogRecord
yuzurihaaa Mar 5, 2025
741ed89
wip: update DateTimeProvider
yuzurihaaa Mar 18, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions lib/src/api/logs/logger.dart
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,11 @@
// Licensed under the Apache License, Version 2.0. Please see https://github.com/Workiva/opentelemetry-dart/blob/master/LICENSE for more information

import '../../../api.dart' as api;
import '../../../sdk.dart' as sdk;
import 'log_record.dart';

abstract class Logger {
void emit({
sdk.Attributes? attributes,
List<api.Attribute> attributes = const <api.Attribute>[],
api.Context? context,
dynamic body,
DateTime? observedTimestamp,
Expand Down
2 changes: 1 addition & 1 deletion lib/src/api/logs/logger_provider.dart
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import 'package:opentelemetry/src/api/logs/logger.dart';
abstract class LoggerProvider {
/// Gets or creates a [Logger] instance.
///
/// The meter is identified by the combination of [name], [version],
/// The logger is identified by the combination of [name], [version],
/// [schemaUrl] and [attributes]. The [name] SHOULD uniquely identify the
/// instrumentation scope, such as the instrumentation library
/// (e.g. io.opentelemetry.contrib.mongodb), package, module or class name.
Expand Down
4 changes: 2 additions & 2 deletions lib/src/api/logs/noop/noop_logger.dart
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
// Copyright 2021-2022 Workiva.
// Licensed under the Apache License, Version 2.0. Please see https://github.com/Workiva/opentelemetry-dart/blob/master/LICENSE for more information

import 'package:opentelemetry/src/api/common/attribute.dart';
import 'package:opentelemetry/src/api/context/context.dart';
import 'package:opentelemetry/src/api/logs/log_record.dart';
import 'package:opentelemetry/src/api/logs/logger.dart';
import 'package:opentelemetry/src/sdk/common/attributes.dart';

class NoopLogger implements Logger {
const NoopLogger();

@override
void emit({
Attributes? attributes,
List<Attribute> attributes = const <Attribute>[],
Context? context,
dynamic body,
DateTime? observedTimestamp,
Expand Down
12 changes: 6 additions & 6 deletions lib/src/experimental_api.dart
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,13 @@ export 'api/context/context_manager.dart' show ContextManager;
export 'api/context/noop_context_manager.dart' show NoopContextManager;
export 'api/context/zone_context.dart' show ZoneContext;
export 'api/context/zone_context_manager.dart' show ZoneContextManager;
export 'api/metrics/counter.dart' show Counter;
export 'api/metrics/meter_provider.dart' show MeterProvider;
export 'api/metrics/meter.dart' show Meter;
export 'api/metrics/noop/noop_meter.dart' show NoopMeter;
export 'api/trace/nonrecording_span.dart' show NonRecordingSpan;
export 'api/logs/logger.dart' show Logger;
export 'api/logs/log_record.dart' show Severity;
export 'api/logs/logger.dart' show Logger;
export 'api/logs/logger_provider.dart' show LoggerProvider;
export 'api/logs/noop/noop_logger.dart' show NoopLogger;
export 'api/logs/noop/noop_logger_provider.dart' show NoopLoggerProvider;
export 'api/metrics/counter.dart' show Counter;
export 'api/metrics/meter.dart' show Meter;
export 'api/metrics/meter_provider.dart' show MeterProvider;
export 'api/metrics/noop/noop_meter.dart' show NoopMeter;
export 'api/trace/nonrecording_span.dart' show NonRecordingSpan;
7 changes: 6 additions & 1 deletion lib/src/experimental_sdk.dart
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,12 @@ library experimental_sdk;

import 'package:meta/meta.dart';

export 'sdk/logs/log_record.dart' show ReadableLogRecord, ReadWriteLogRecord, LogRecord;
export 'sdk/logs/log_record_limit.dart' show LogRecordLimits;
export 'sdk/logs/logger.dart' show Logger;
export 'sdk/logs/logger_provider.dart' show LoggerProvider;
export 'sdk/logs/processors/log_record_processor.dart' show LogRecordProcessor;
export 'sdk/metrics/counter.dart' show Counter;
export 'sdk/metrics/meter_provider.dart' show MeterProvider;
export 'sdk/metrics/meter.dart' show Meter;
export 'sdk/metrics/meter_provider.dart' show MeterProvider;
export 'sdk/resource/resource.dart' show Resource;
46 changes: 35 additions & 11 deletions lib/src/sdk/common/limits.dart
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ import 'package:meta/meta.dart';

import '../../../api.dart' as api;
import '../../../sdk.dart' as sdk;
import '../../experimental_sdk.dart' as sdk;

/// Applies given [sdk.SpanLimits] to a list of [api.SpanLink]s.
@protected
List<api.SpanLink> applyLinkLimits(
List<api.SpanLink> links, sdk.SpanLimits limits) {
List<api.SpanLink> applyLinkLimits(List<api.SpanLink> links, sdk.SpanLimits limits) {
final spanLink = <api.SpanLink>[];

for (final link in links) {
Expand All @@ -27,8 +27,7 @@ List<api.SpanLink> applyLinkLimits(
for (final attr in link.attributes) {
// if attributes num is already greater than maxNumAttributesPerLink
// and this key doesn't exist in the list, drop it.
if (attributeMap.length >= limits.maxNumAttributesPerLink &&
!attributeMap.containsKey(attr.key)) {
if (attributeMap.length >= limits.maxNumAttributesPerLink && !attributeMap.containsKey(attr.key)) {
droppedAttributes++;
continue;
}
Expand All @@ -49,8 +48,7 @@ List<api.SpanLink> applyLinkLimits(
}
}

spanLink.add(api.SpanLink(link.context,
attributes: linkAttributes, droppedAttributes: droppedAttributes));
spanLink.add(api.SpanLink(link.context, attributes: linkAttributes, droppedAttributes: droppedAttributes));
}
return spanLink;
}
Expand All @@ -63,20 +61,46 @@ api.Attribute applyAttributeLimits(api.Attribute attr, sdk.SpanLimits limits) {

if (attr.value is String) {
attr = api.Attribute.fromString(
attr.key,
applyAttributeLengthLimit(
attr.value as String, limits.maxNumAttributeLength));
attr.key, applyAttributeLengthLimit(attr.value as String, limits.maxNumAttributeLength));
} else if (attr.value is List<String>) {
final listString = attr.value as List<String>;
for (var j = 0; j < listString.length; j++) {
listString[j] = applyAttributeLengthLimit(
listString[j], limits.maxNumAttributeLength);
listString[j] = applyAttributeLengthLimit(listString[j], limits.maxNumAttributeLength);
}
attr = api.Attribute.fromStringList(attr.key, listString);
}
return attr;
}

@protected
api.Attribute applyAttributeLimitsForLog(
api.Attribute attr,
sdk.LogRecordLimits limits,
) {
// if maxNumAttributeLength is less than zero, then it has unlimited length.
if (limits.attributeValueLengthLimit < 0) return attr;

if (attr.value is String) {
return (attr.value as String).length > limits.attributeValueLengthLimit
? api.Attribute.fromString(attr.key, (attr.value as String).substring(0, limits.attributeValueLengthLimit))
: attr;
} else if (attr.value is List<String>) {
final list = (attr.value as List<String>);
List<String>? truncated;
for (int i = 0; i < list.length; i++) {
final s = list[i];
if (s.length > limits.attributeValueLengthLimit) {
truncated ??= List<String>.from(list, growable: false);
truncated[i] = s.substring(0, limits.attributeValueLengthLimit);
}
}
if (truncated != null) {
return api.Attribute.fromStringList(attr.key, truncated);
}
}
return attr;
}

/// Truncate just strings which length is longer than configuration.
/// Reference: https://github.com/open-telemetry/opentelemetry-java/blob/14ffacd1cdd22f5aa556eeda4a569c7f144eadf2/sdk/common/src/main/java/io/opentelemetry/sdk/internal/AttributeUtil.java#L80
@protected
Expand Down
152 changes: 152 additions & 0 deletions lib/src/sdk/logs/log_record.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
// Copyright 2021-2022 Workiva.
// Licensed under the Apache License, Version 2.0. Please see https://github.com/Workiva/opentelemetry-dart/blob/master/LICENSE for more information

import 'package:meta/meta.dart';
import 'package:opentelemetry/api.dart' as api;
import 'package:opentelemetry/sdk.dart' as sdk;
import 'package:opentelemetry/src/experimental_api.dart' as api;
import 'package:opentelemetry/src/experimental_sdk.dart' as sdk;
import 'package:opentelemetry/src/sdk/common/limits.dart';

/// https://opentelemetry.io/docs/specs/otel/logs/sdk/#readwritelogrecord
abstract class ReadableLogRecord {
DateTime get timeStamp;

DateTime get observedTimestamp;

String get severityText;

api.Severity get severityNumber;

dynamic get body;

sdk.Attributes get attributes;

api.SpanContext get spanContext;

sdk.Resource get resource;

sdk.InstrumentationScope get instrumentationScope;

int get droppedAttributesCount;
}

abstract class ReadWriteLogRecord extends ReadableLogRecord {
set body(dynamic severity);

set severityText(String severity);

set severityNumber(api.Severity severity);
}

class LogRecord implements ReadWriteLogRecord {
@override
final sdk.InstrumentationScope instrumentationScope;

final sdk.Resource _resource;

final sdk.TimeProvider _timeProvider;
final api.Context _context;
final sdk.LogRecordLimits logRecordLimits;
final DateTime? _timeStamp;
final DateTime? _observedTimestamp;

bool _isReadonly = false;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove this flag? The tracing API is organized similarly and does not need it

String _severityText;
api.Severity _severityNumber;
dynamic _body;
int _totalAttributesCount = 0;

final sdk.Attributes _attributes;

@protected
LogRecord({
required this.instrumentationScope,
required this.logRecordLimits,
api.Severity? severityNumber,
String? severityText,
List<api.Attribute> attributes = const <api.Attribute>[],
DateTime? timeStamp,
DateTime? observedTimestamp,
api.Context? context,
dynamic body,
sdk.Resource? resource,
sdk.TimeProvider? timeProvider,
}) : _severityText = severityText ?? api.Severity.unspecified.name,
_resource = resource ?? sdk.Resource([]),
_context = context ?? api.Context.current,
_body = body,
_attributes = sdk.Attributes.empty(),
_severityNumber = severityNumber ?? api.Severity.unspecified,
_timeStamp = timeStamp,
_observedTimestamp = observedTimestamp,
_timeProvider = timeProvider ?? sdk.DateTimeTimeProvider() {
if (attributes.isNotEmpty) setAttributes(attributes);
}


@override
sdk.Resource get resource => _resource;

@override
sdk.Attributes get attributes => _attributes;

@override
dynamic get body => _body;

@override
set body(dynamic body) {
if (_isReadonly) return;
_body = body;
}

@override
api.SpanContext get spanContext => api.spanContextFromContext(_context);

@override
int get droppedAttributesCount => _totalAttributesCount - attributes.length;

@override
DateTime get timeStamp => _timeStamp ?? DateTime.fromMicrosecondsSinceEpoch((_timeProvider.now ~/ 1000).toInt());

@override
DateTime get observedTimestamp =>
_observedTimestamp ?? DateTime.fromMicrosecondsSinceEpoch((_timeProvider.now ~/ 1000).toInt());

@override
api.Severity get severityNumber => _severityNumber;

@override
set severityNumber(api.Severity severity) {
if (_isReadonly) return;
_severityNumber = severity;
}

@override
String get severityText => _severityText;

@override
set severityText(String severity) {
if (_isReadonly) return;
_severityText = severity;
}

void setAttributes(List<api.Attribute> attributes) {
attributes.forEach(setAttribute);
}

void setAttribute(api.Attribute attribute) {
if (_isReadonly) return;
if (attribute.key.isEmpty) return;
if (logRecordLimits.attributeCountLimit == 0) return;
_totalAttributesCount += 1;
_attributes.add(applyAttributeLimitsForLog(attribute, logRecordLimits));
}

/// A LogRecordProcessor may freely modify logRecord for the duration of the OnEmit call.
/// If logRecord is needed after OnEmit returns (i.e. for asynchronous processing) only reads are permitted.
@internal
void makeReadonly() {
_isReadonly = true;
}
}
17 changes: 17 additions & 0 deletions lib/src/sdk/logs/log_record_limit.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
// Copyright 2021-2022 Workiva.
// Licensed under the Apache License, Version 2.0. Please see https://github.com/Workiva/opentelemetry-dart/blob/master/LICENSE for more information

class LogRecordLimits {
final int _attributeCountLimit;
final int _attributeValueLengthLimit;
Comment on lines +5 to +6
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since these are final, can the getters be removed?


const LogRecordLimits({
int attributeCountLimit = 128,
int attributeValueLengthLimit = -1,
}) : _attributeCountLimit = attributeCountLimit,
_attributeValueLengthLimit = attributeValueLengthLimit;

int get attributeCountLimit => _attributeCountLimit;

int get attributeValueLengthLimit => _attributeValueLengthLimit;
}
53 changes: 53 additions & 0 deletions lib/src/sdk/logs/logger.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
// Copyright 2021-2022 Workiva.
// Licensed under the Apache License, Version 2.0. Please see https://github.com/Workiva/opentelemetry-dart/blob/master/LICENSE for more information

import 'package:meta/meta.dart';
import 'package:opentelemetry/api.dart' as api;
import 'package:opentelemetry/sdk.dart' as sdk;
import 'package:opentelemetry/src/api/context/context.dart';
import 'package:opentelemetry/src/experimental_api.dart' as api;
import 'package:opentelemetry/src/experimental_sdk.dart' as sdk;

class Logger extends api.Logger {
final sdk.InstrumentationScope instrumentationScope;
final sdk.Resource _resource;
final sdk.LogRecordLimits logRecordLimits;
final sdk.TimeProvider timeProvider;
final List<sdk.LogRecordProcessor> processors;
Comment on lines +12 to +16
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can all of these fields be private?


@protected
Logger({
required this.instrumentationScope,
required this.logRecordLimits,
required this.timeProvider,
this.processors = const <sdk.LogRecordProcessor>[],
sdk.Resource? resource,
}) : _resource = resource ?? sdk.Resource([]);

@override
void emit({
List<api.Attribute> attributes = const [],
Context? context,
dynamic body,
DateTime? observedTimestamp,
api.Severity? severityNumber,
String? severityText,
DateTime? timeStamp,
}) {
final log = sdk.LogRecord(
logRecordLimits: logRecordLimits,
resource: _resource,
instrumentationScope: instrumentationScope,
context: context,
severityText: severityText,
severityNumber: severityNumber,
attributes: attributes,
body: body,
timeProvider: timeProvider,
);
for (final processor in processors) {
processor.onEmit(log);
}
log.makeReadonly();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be cleaner to have two types where one is immutable and the other is mutable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intention of this makeReadonly to allow mutation (if any) only during onLogEmit?.call.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes it sound like that method should take a read/write log and the caller should keep a read-only log afterwards

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That method here you are referring to is onLogEmit right? It is already takes read/write log final Function(sdk.ReadWriteLogRecord)? onLogEmit since only during onLogEmit the logs can be mutate, but not once it's done.

}
}
Loading