Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for dead letter in event log #652

Open
anovakovic01 opened this issue Mar 14, 2019 · 2 comments
Open

Add support for dead letter in event log #652

anovakovic01 opened this issue Mar 14, 2019 · 2 comments
Assignees
Milestone

Comments

@anovakovic01
Copy link
Contributor

FEATURE REQUEST

  1. Is there an open issue addressing this request? If it does, please add a "+1" reaction to the
    existing issue, otherwise proceed to step 2.
    No.

  2. Describe the feature you are requesting, as well as the possible use case(s) for it.
    There is a possibility that some events from Redis Streams cannot be processed. In this case, currently, our consumer will be stuck on this event, which is not good. We should send such events to some other stream where we'll store problematic events. This way it will be easier to track such events and also, our consumers won't get stuck on such messages.

  3. Indicate the importance of this feature to you (must-have, should-have, nice-to-have).
    This is a must-have feature.

@nmarcetic
Copy link
Collaborator

@anovakovic01 Can you make example of such an event, when this can occur? A consumer should handle the processing and catch errors or I am missing a point?
Maybe simple ERROR processing failure logging will be enough? I don't get it how We should send such events to some other stream where we'll store problematic events can help to not stuck the consumer?

@anovakovic01
Copy link
Contributor Author

anovakovic01 commented Mar 19, 2019

@nmarcetic Here is nice explanation from Redis Streams documentation.

The counter that you observe in the XPENDING output is the number of deliveries of each message. Such counter is incremented in two ways: when a message is successfully claimed via XCLAIM or when an XREADGROUP call is used in order to access the history of pending messages. When there are failures, it is normal that messages are delivered multiple times, but eventually they usually get processed. However there is sometimes a problem to process a given specific message, because it is corrupted or crafted in a way that triggers a bug in the processing code. In such a case what happens is that consumers will continuously fail to process this particular message. Because we have the counter of the delivery attempts, we can use that counter to detect messages that for some reason are not processable at all. So once the deliveries counter reaches a given large number that you chose, it is probably wiser to put such messages in another stream and send a notification to the system administrator. This is basically the way that Redis streams implement the concept of the dead letter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: ⛏ Backlog
Development

No branches or pull requests

10 participants