Open
Description
I'm currently investigating metric event type with this connector with the following config and event format via custom forwarder.
connect-distributed.properties for connect
# A list of host/port pairs to use for establishing the initial connection to the Kafka cluster.
bootstrap.servers=localhost:9092
group.id=kafka-connect-splunk-hec-sink
or stored into Kafka
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
offset.storage.topic=connect-offsets
offset.storage.replication.factor=1
config.storage.topic=connect-configs
config.storage.replication.factor=1
status.storage.topic=connect-status
status.storage.replication.factor=1
#status.storage.partitions=5
# Flush much faster than normal, which is useful for testing/debugging
offset.flush.interval.ms=10000
plugin.path=connectors/
Record format
JSON string with the following format:
{"host":"development-box", "time":"1629859258.5508862","event":"metric", "fields":{"metric_name":"network_device_eth0_transmit_bytes_total", "_value":36589580.0}}
as mentioned in
Created Connector Task
{
"name": "kafka-connect-splunk",
"config": {
"connector.class": "com.splunk.kafka.connect.SplunkSinkConnector",
"tasks.max": "10",
"topics":"myapp.test,t1,t2,t3,t4,t5,t6,t7,t8,t9,t10",
"splunk.hec.uri":"https://localhost:8088",
"splunk.hec.token": "<Splunk HEC Token>",
"splunk.hec.ack.enabled" : "true",
"splunk.hec.raw" : "false",
"splunk.hec.track.data" : "true",
"splunk.hec.ssl.trust.store.path":"/etc/ssl/certs/java/cacerts",
"splunk.hec.ssl.trust.store.password":"changeit",
"splunk.hec.ssl.validate.certs": "false"
}
}
And created metric index on Splunk with this instruction: https://docs.splunk.com/Documentation/Splunk/8.2.1/Metrics/GetMetricsInOther#Get_metrics_in_from_clients_over_HTTP_or_HTTPS
But no luck. What am I missing about ingestion of metric record via custom HEC forwarder via this connect? Or, this kafka-connect-splunk doesn't support metric type Splunk HEC events for now?
Additional context
With the above settings, event type indices not for metric type indices, and hec token for normal events, I got succeeded to ingest Splunk HEC events normally.