[FIX] Sistema de webhooks: melhoria com timeout e retentativas configuráveis

[guilhermejansen]

Referência

Resolve #1325

Descrição

Esta PR implementa melhorias significativas no sistema de webhooks da API, adicionando configurabilidade via variáveis de ambiente e corrigindo o bug crítico que causava duplicação de webhooks.

Problemas Resolvidos

1. Correção de bug crítico: Webhooks estavam sendo enviados várias vezes mesmo quando o servidor respondia com sucesso
2. Timeout configurável: Agora é possível definir o tempo máximo de espera por uma resposta
3. Retentativas inteligentes: Implementação de backoff exponencial com jitter para evitar sobrecarga
4. Detecção de erros permanentes: Não tenta novamente em casos de erros como 400, 401, 403, etc.
5. Logs aprimorados: Mais detalhes sobre falhas para facilitar diagnóstico

Novas Configurações

Todas estas configurações são opcionais e possuem valores padrão razoáveis:

# Webhook timeout e configuração de retry
WEBHOOK_REQUEST_TIMEOUT_MS=30000
WEBHOOK_RETRY_MAX_ATTEMPTS=10
WEBHOOK_RETRY_INITIAL_DELAY_SECONDS=5
WEBHOOK_RETRY_USE_EXPONENTIAL_BACKOFF=true
WEBHOOK_RETRY_MAX_DELAY_SECONDS=300
WEBHOOK_RETRY_JITTER_FACTOR=0.2
# Lista de códigos HTTP que não devem gerar retentativas
WEBHOOK_RETRY_NON_RETRYABLE_STATUS_CODES=400,401,403,404,422

Impacto e Compatibilidade

• Esta mudança é totalmente compatível com versões anteriores
• As novas configurações são opcionais, mantendo o comportamento padrão se não forem especificadas
• Melhora significativa na estabilidade e eficiência para sistemas com alto volume de webhooks

Testes Realizados

• Testei a implementação com diferentes cenários:
  • Servidor de destino lento (simulando timeout)
  • Servidor respondendo com diferentes códigos de erro (400, 404, 500)
  • Servidor funcionando normalmente (confirmação de que não há mais duplicação)

Alterações Implementadas

• Adicionada interface para configuração de timeout e retry
• Adicionadas variáveis de ambiente para webhook
• Implementado timeout configurável
• Implementado backoff exponencial e correção de duplicação

Summary by Sourcery

Improves the webhook system by adding configurable options for timeout and retry logic, and fixes a bug that caused duplicate webhooks. It introduces environment variables to configure the webhook system.

Bug Fixes:

• Fixes a bug that caused webhooks to be sent multiple times even when the server responded successfully.

Enhancements:

• Adds configurable timeout for webhook requests.
• Implements exponential backoff with jitter for webhook retries.
• Adds logic to avoid retrying on permanent errors such as 400, 401, and 403 status codes.
• Improves logging for webhook failures to facilitate diagnosis.

[sourcery-ai]

Reviewer's Guide by Sourcery

This pull request introduces significant enhancements to the webhook system, including configurable timeout and retry mechanisms with exponential backoff and jitter. It also fixes a critical bug that caused duplicate webhooks. The changes are implemented by adding new environment variables for configuration, modifying the webhook request logic to include timeout and retry functionality, and updating the configuration service to support the new settings.

No diagrams generated as the changes look simple and do not need a visual representation.

File-Level Changes

Change	Details	Files
Introduced configurable timeout for webhook requests.	Added timeout configuration to the axios.create method when creating the HTTP service for sending webhooks. The timeout value is obtained from the environment variable WEBHOOK_REQUEST_TIMEOUT_MS or defaults to 30000ms if not set.	src/api/integrations/event/webhook/webhook.controller.ts
Implemented configurable retry mechanism with exponential backoff and jitter for webhook requests.	Introduced environment variables to configure retry behavior, including maximum attempts, initial delay, exponential backoff usage, maximum delay, and jitter factor. Implemented a retry loop that attempts to send the webhook request until it succeeds or the maximum number of retries is reached. Implemented exponential backoff with jitter to calculate the delay between retries. Added logic to prevent retries for specific HTTP status codes (e.g., 400, 401, 403, 404, 422) that indicate permanent errors. Enhanced logging to include details about retry attempts, timeouts, and non-retryable errors.	src/api/integrations/event/webhook/webhook.controller.ts
Added new environment variables for configuring webhook behavior.	Added WEBHOOK_REQUEST_TIMEOUT_MS to configure the webhook request timeout. Added WEBHOOK_RETRY_MAX_ATTEMPTS to configure the maximum number of retry attempts. Added WEBHOOK_RETRY_INITIAL_DELAY_SECONDS to configure the initial delay between retries. Added WEBHOOK_RETRY_USE_EXPONENTIAL_BACKOFF to enable or disable exponential backoff. Added WEBHOOK_RETRY_MAX_DELAY_SECONDS to configure the maximum delay between retries. Added WEBHOOK_RETRY_JITTER_FACTOR to configure the jitter factor for the retry delay. Added WEBHOOK_RETRY_NON_RETRYABLE_STATUS_CODES to configure a comma-separated list of HTTP status codes that should not be retried.	src/config/env.config.ts .env.example
Added configuration options for webhooks.	Added REQUEST and RETRY configuration options to the Webhook type definition. The REQUEST configuration allows setting a TIMEOUT_MS value. The RETRY configuration allows setting MAX_ATTEMPTS, INITIAL_DELAY_SECONDS, USE_EXPONENTIAL_BACKOFF, MAX_DELAY_SECONDS, JITTER_FACTOR, and NON_RETRYABLE_STATUS_CODES values.	src/config/env.config.ts

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#1325	The webhook system should not send duplicate requests when the destination server is available and responding successfully.	✅
#1325	The webhook request timeout should be configurable via environment variables.	✅
#1325	The webhook retry parameters (max attempts, initial interval, exponential backoff) should be configurable via environment variables.	✅
#1325	The webhook system should implement exponential backoff for retries.	✅
#1325	The webhook system should not retry on permanent errors (HTTP codes 400, 401, 403, etc.).	✅

Possibly linked issues

• [BUG] Sistema de webhooks envia requisições duplicadas e não possui configuração de timeout e retentativas #1325: The PR fixes the webhook duplication bug and adds configurable timeout and retry logic.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!
• Generate a plan of action for an issue: Comment @sourcery-ai plan on
  an issue to generate a plan of action for it.

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey @guilhermejansen - I've reviewed your changes - here's some feedback:

Overall Comments:

• Consider adding a circuit breaker to prevent cascading failures and improve system resilience.

Here's what I looked at during the review

• 🟢 General issues: all looks good
• 🟢 Security: all looks good
• 🟢 Testing: all looks good
• 🟡 Complexity: 1 issue found
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[acauaferreira]

libera isso pra noooooois

To com muitos erros de WEBHOOK via API, principalmente via DIFY , que quando da algum erro ou demora mais de 30s a conversa simplesmente TRAVA e só deletando a sessão pro bot voltar a funcionar.

[michaelvips]

libera isso pra noooooois

To com muitos erros de WEBHOOK via API, principalmente via DIFY , que quando da algum erro ou demora mais de 30s a conversa simplesmente TRAVA e só deletando a sessão pro bot voltar a funcionar.

faz um fork e cria uma imagem amigo.

[Junior-Miranda]

up

[brunoalbim]

UP

[AlejoTorres2001]

Up!!

[DavidsonGomes]

Ajuste os conflitos por favor, estava para a branch main, o correto é para a develop

[guilhermejansen]

Ajuste os conflitos por favor, estava para a branch main, o correto é para a develop

#1341 corrigido, obrigado @DavidsonGomes !