Multiple Handler configuration sections

[mrichar1]

The following is really a request to get feedback on an idea I have, before I begin any work implementing it. This all depends on #55 being accepted, i.e allowing handlers to accept command-line options.

I've been thinking about how to allow handlers to have multiple configuration sections defined in the config json. This would be useful for handlers like mailer or xmpp, where you might want to have different handlers alerting different users, but not want to have to modify the handler code each time.

I'd like to propose a command-line option, set in sensu-plugin, which is then inherited by handlers - lets call this 'jsonconfig' for now (name isn't important). It's default value would be the name of the handler. It can then be overridden to point to a differently named section when the handler is used, e.g:

/etc/sensu/handlers/notification/mailer.rb --jsonconfig mailer-devteam

'mailer-devteam' in the json config would contain email configuration options for the development team, while 'mailer' (the default) would email the production team.

It would be easy to change handlers on a case-by-case basis, switching them from having hard-coded config paths to using this variable.

For example, mailer.rb currently has:

 mail_to = settings['mailer']['mail_to']

This would change to something like:

mail_to = settings[config[:jsonconfig]]['mail_to']

I'm happy to write the code for this, and also to update any documentation. I'd also be making changes to at least some of the community handlers to support this for my own use, mailer and xmpp, but perhaps others if the changes are minimal.

Thoughts, comments, input etc appreciated!

[dgarbus]

Yes! I was just complaining about the current configuration in the IRC channel on Friday and this seems like a good way to go about fixing it 👍.

I am using Puppet to configure sensu, and right now with the top-level json configuration for handlers, you can really only have one configuration per handler script. Your example of being able to send email to multiple addresses using the same mailer.rb script is exactly what I am looking for in my environment. Also, it looks like this change would not break the way that the puppet module creates handlers, so that's a plus as well.

[danshultz]

you could also do something like the following and allowing your check to specify where/how to route an email or a pagerduty response, etc.

We are doing something similar for pager duty checks, etc.

mail_to = @event['check']['mail_to'] || settings['mailer']['mail_to']

Your check then would like the following

{
  "checks": {
    "rabbitmq-node-sockets-alert": {
      "command": "/etc/sensu/plugins/rabbitmq-node-alerts.rb --metric sockets --warn 0.92",
      "interval": 30,
      "handlers": [
        "endpoint-down"
      ],
      "subscribers": [
        "rabbitmq"
      ],
      "standalone": false,
      "notification": "Rabbit Sockets Alert",
      "mail_to": "person@example.com"
    }
  }
}

[mrichar1]

@danshultz I like this idea - I think it complements my suggestion: mine allows the config variables to be set on a per-handler basis, while yours is on a per-check basis - both are potentially useful!

What do you think of the idea of combining the 2, e.g:

mail_to = @event['check']['mail_to'] || settings[config[:jsonconfig]]['mail_to']

(jsonconfig value in this instance would default to 'mailer' so would be functionally identical to your example above).

[kalabiyau]

@mrichar1 it looks to me that is valid use case and can add some great value to flexibility. @portertech what is your opinion on that?

[abhishekjain88]

@kalabiyau @portertech Any decision made on this?

[portertech]

I recall Mixlib-CLI not playing nice with subclassing a class with arguments. Need to revisit that blocker.

[davidjh]

Thanks - this helped me solve a long-standing problem for me. I modified my mailer handler:

:mail_to => @event['check']['mail_to'] || settings['mailer']['mail_to'],

and added a mail_to value to my checks.

[PChambino]

I would like to propose a new idea to this problem.

Instead of using a command-line option to find the handler configuration, the handler definition could be passed through STDIN together with the event data. This would require changing sensu-server.

By handler configuration I mean settings[:<handler name>] and by handler definition I mean settings[:handlers][:<handler name>].

The handler name could then be used to locate the handler configuration, or the handler options could be within the handler definition.

Thoughts?

[majormoses]

That PR has been merged, should we re-open this up for discussion?