Sample configuration for syslog output to Papertrail using SD token auth

[sigJoe]

I want to share a sample configuration I made to send structured-data token authenticated syslog (i.e. Papertrail & Loggly destinations) using Fluent-bit. I found the documentation wasn't clear on TLS usage (still refers to mode: tls) or how to populate structured data, so maybe somebody else will find this useful. YAML format because it makes Lua functions easier to read

# /etc/fluent-bit/fluent-bit.yaml
service:
  flush: 1
  daemon: off
  log_level: info
  parsers_file: parsers.conf

pipeline:
  inputs:
    - name: tail
      path: /var/log/nginx/access.log
      path_key: path
      read_from_head: true
    - name: tail
      path: /var/log/nginx/error.log
      path_key: path
      read_from_head: true
    - name: tail
      path: /var/log/messages
      path_key: path
      read_from_head: true

  filters:
    - name:  lua
      match: "*"
      call:  set_appname
      # Use the last element of the file path (e.g. /var/log/messages -> messages)
      code:  |
        function set_appname(tag, timestamp, record)
          local lastElement
          for match in string.gmatch(record["path"], "[^/]+") do
              lastElement = match
          end
          record["appname"] = lastElement
          return 2, timestamp, record
        end
    - name:  lua
      match: "*"
      call:  append_token
      # This is necessary because Fluentbit tries to strictly conform to RFC5424
      # e.g. if syslog_sd_key were set to `sd`, then setting `record["sd"] = {foo = "bar"}` would create structed data `[sd foo=bar]`
      # Since we only want SD `[<token>@41018]`, that means the token needs to get set as the key rather than the value
      code:  |
        function append_token(tag, timestamp, record)
            record["${PAPERTRAIL_TOKEN}"] = {}
            return 2, timestamp, record
        end

  outputs:
    - name: syslog
      match: "*"
      host: ${PAPERTRAIL_HOST}
      port: ${PAPERTRAIL_PORT}
      mode: tcp
      syslog_format: rfc5424
      syslog_hostname_preset: ${SYSLOG_HOSTNAME}
      syslog_appname_key: appname
      syslog_message_key: log
      allow_longer_sd_id: true # Our tokens are longer than 32 characters which breaks RFC5424
      syslog_sd_key: ${PAPERTRAIL_TOKEN}
      tls: on
      tls.crt_file: /etc/fluent-bit/syslog.papertrail.crt

Then populate the environment variables in /etc/sysconfig/fluent-bit

# /etc/sysconfig/fluent-bit
SYSLOG_HOSTNAME=my-syslog-hostname
PAPERTRAIL_TOKEN=foobar@41018
PAPERTRAIL_HOST=logs.collector.solarwinds.com
PAPERTRAIL_PORT=6514

[patrick-stephens]

Be good to update the docs with this example - also, which docs were wrong/confusing as we should definitely fix those? Can you send any PRs to fluent-bit-docs?

[sigJoe]

I made a PR: fluent/fluent-bit-docs#1377

I didn't include the set_appname Lua example, as it didn't strictly seem relevant to the syslog output plugin and I didn't know where else to put it.