Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect retry count returned in case of Activity Function #167

Open
kanupriya15025 opened this issue Sep 27, 2023 · 4 comments
Open

Incorrect retry count returned in case of Activity Function #167

kanupriya15025 opened this issue Sep 27, 2023 · 4 comments
Assignees
Labels
Enhancement New feature or request need investigation Need investigation from a maintainer P2 Priority 2

Comments

@kanupriya15025
Copy link

kanupriya15025 commented Sep 27, 2023

I have an Activity Function which has a retry count set to 3. This is intentionally made to throw an exception and retry. Now, I am trying to get the retry count in the Activity Function but it always returns 0.
Here is a sample code I am trying :

public String aepHTTPPollingOperator(@DurableActivityTrigger(name = "input", dataType = "string") String input,
                        ExecutionContext context) throws JsonMappingException, JsonProcessingException, Exception {

    System.out.println("Context Retry : " + context.getRetryContext().getRetrycount()); //always prints 0
    throw new RuntimeException("test exception")
}

This is how I am calling this activity :

RetryPolicy policy = new RetryPolicy(3, 30);
TaskOptions options = new TaskOptions(policy);
ctx.callActivity("ActivityName", activityInputQO.toString(), options, String.class));
@kaibocai kaibocai added need investigation Need investigation from a maintainer and removed Needs: Triage 🔍 labels Sep 27, 2023
@kaibocai kaibocai self-assigned this Sep 27, 2023
@cgillum
Copy link
Member

cgillum commented Sep 27, 2023

Ah, so unfortunately these are two different retry counts for two different retry features, and they are largely unaware of each other, which makes this understandably confusing.

In your activity function code, the API you're calling (I think) corresponds to a generic Azure Functions retry feature. It's not at all related to retry policies in Durable Functions. It's a local, in-memory, non-durable retry capability with a static configuration that works with a specific set of function triggers. The documentation is here.

Retry policies in orchestrations, on the other hand, only kick in when the orchestration actually receives a failure signal from the activity. When using the local retries, the orchestration is unaware of a failure until the local retries expire. Unfortunately, there's no way currently for an activity function to know whether it is retrying due to an orchestrator retry policy. This is why you're always getting for context.getRetryContext().getRetrycount() - I'm guessing you didn't configure the generic retry feature of Azure Functions for which this API was designed for.

In fact, if you use these two retry features together, you'd get multiple layers of retry.

@kanupriya15025
Copy link
Author

Got it. @cgillum Is there a way to find it out? Our aim is to send a failure metric once the activity fails and if we don't know what retry it is, we'll be sending a failure counter everytime it retries instead of a 1.

@cgillum
Copy link
Member

cgillum commented Sep 28, 2023

We don't have a way to surface this information, unfortunately. We have a backlog item to surface retry count information to activities to help developers implement idempotency policies, but it hasn't yet reached the top of our current priority list.

@kanupriya15025
Copy link
Author

@cgillum Can you share that ticket so that I can track?

@lilyjma lilyjma added Enhancement New feature or request P2 Priority 2 labels Jan 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or request need investigation Need investigation from a maintainer P2 Priority 2
Projects
None yet
Development

No branches or pull requests

4 participants