Skip to content

Improve the FailedRunsNotificationCronJob to be more resilient #107

@evenicoulddoit

Description

@evenicoulddoit

In it's existing setup, the FailedRunsNotificationCronJob simply sends an email when the number of failed runs for a certain job reaches a pre-determined level (which defaults to 10). After this point (i.e. at fail 11, 12...) the user will not receive any further emails.

Furthermore, the job will also continue to send emails to the users every time the runcrons task is called, provided the number of failed runs remains at the level which triggers the email. Therefore, if you're using crontab e.g. to run the runcrons management command every 10 minutes, the user will be emailed about the failure every 10 minutes.

#101 aims to fix both problems. By adding a failure_reported flag to the CronJobLog model, Once the number of failures exceeds the user determined limit, they are marked as having been notified.

In addition to the above, a few changes are made to:

  • Provide more settings to modify the email recipients
  • Allow the FailedRunsNotificationCronJob to be extended in order to modify how the notification takes place (e.g. using Slack, text etc..)

Tests are provided and documentation has been added

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions