Add support for sending events to Dynatrace logs

[l3ender]

Similar to the Splunk pump, we have a use case where we want to send API events from Tyk to the logs feature in Dynatrace.

Sending log data to Dynatrace is different than using Dynatrace to observe the Tyk application's process (or Tyk pump). Log data is also different than using the open telemetry features of Tyk (even if sending to Dynatrace): the OTeL data provided by Tyk does not contain the fullness of information that tyk-pump provides, such as request/response times. Additionally, using logs to retain API event information may be desired as compared needing the full span/trace information provided via OTeL.

See the related resources for more detail:
https://docs.dynatrace.com/docs/analyze-explore-automate/logs/lma-log-ingestion/lma-log-ingestion-via-api
https://docs.dynatrace.com/docs/discover-dynatrace/references/dynatrace-api/environment-api/log-monitoring-v2/post-ingest-logs

[probelabs]

Hello @l3ender, thank you for opening this issue and for the detailed feature request.

Understanding the Request

From what I understand, you're proposing a new pump to send Tyk analytics events directly to the Dynatrace Log Management and Analytics platform. This would be similar to our existing pumps, like the one for Splunk, which forwards data to an HTTP endpoint.

This is a great suggestion. While Tyk does support OpenTelemetry for observability, you've correctly pointed out that the analytics data from tyk-pump contains a richer, event-focused dataset that is valuable for log-based analysis and retention.

Next Steps & Guidance

This is a welcome feature request. Community contributions are a big part of how Tyk grows, and a dynatrace pump would be a valuable addition.

For anyone interested in contributing this feature, the implementation would likely involve:

1. Creating a new pump file: A new file, perhaps pumps/dynatrace.go, would house the logic.
2. Implementing the Pump interface: The new pump would need to satisfy the Pump interface, which includes methods like Init() and WriteData().
3. Handling Configuration: A new configuration struct (e.g., DynatracePumpConfig) would be needed to handle settings like the Dynatrace environment URL and API token.
4. Sending Data: The WriteData() method would be responsible for formatting the analytics records into the JSON format expected by the Dynatrace Log Ingestion API and sending them via an HTTP client.

Looking at the existing SplunkPump implementation is an excellent starting point, as it demonstrates how to send data to a remote HTTP endpoint with authentication and batching.

References:

• pumps/splunk.go – A good example of an existing pump that sends data to an HTTP API.

---

Powered by Visor from Probelabs

💡 TIP: You can chat with Visor using /visor ask <your question>