feat(sinks): Add on-conflict config support to postgres sink

[vaughnw128]

Summary

This PR aims to allow Vector's Postgres sink to gracefully handle inserts of duplicate keys. Currently, this creates an error and creates difficult failure cases where entire batches error and we lose events. We heavily use Vector to ingest S3 data through SQS into Postgres, but are somewhat unable to properly use Vector's end-to-end acknowledgements as this would re-trigger on inserts that we want to disregard. Since Vector's use of (and Postgres's limitations around jsonb_populate_recordset don't allow for automatic ID population, we have to insert these earlier in the process which causes the conflict issues.

I understand this enhancement may be rejected as I'm unsure whether it's an intentional limitation or not. I checked existing issues and PRs and could not find anything!

References

Related: #26304

Vector configuration

To debug these changes, this config can be used:

events.jsonl:

{"id": 1, "message": "already in the table, conflicts with the row from setup.sql"}
{"id": 2, "message": "new row in the same batch as a conflict"}
{"id": 3, "message": "new row"}
{"id": 3, "message": "in-batch duplicate of the previous line"}
{"id": 4, "message": "second batch, no conflicts"}
{"id": 5, "message": "second batch, no conflicts"}
{"id": 6, "message": "second batch, no conflicts"}
{"id": 7, "message": "second batch, no conflicts"}

config.yaml:

sources:
  events:
    type: stdin
    decoding:
      codec: json

  internal:
    type: internal_metrics
    scrape_interval_secs: 1

sinks:
  postgres:
    type: postgres
    inputs: [events]
    endpoint: "${PG_URL}"
    table: dbg_events
    on_conflict: ${ON_CONFLICT:-error}
    batch:
      max_events: 4
      timeout_secs: 1

db.sql:

DROP TABLE IF EXISTS dbg_events;

CREATE TABLE dbg_events (
    id BIGINT PRIMARY KEY,
    message TEXT
);

INSERT INTO dbg_events (id, message) VALUES (1, 'existing');

How did you test this PR?

This PR was tested by building the diff in CI as an image and using it in our testing pipelines to ensure the feature was functional and stable. It was built with the isolated change off of the base at 0.59.0, and was run in Kubernetes via a custom helm chart. The vector config used transforms unformatted JSON into a "pg" Json blob and then assigns it to the database column names. These rows are then inserted with the following sink config:

event_logs_postgres:
  type: postgres
  inputs: [sdu_event_logs]
  endpoint: "SECRET[k8s.DATABASE_URL]?application_name=${APPLICATION_NAME:-vector-generic}"
  table: "event_logs"
  on_conflict: &pg_on_conflict do_nothing
  batch: &pg_batch
    max_events: 500
    timeout_secs: 5
  request: &pg_request
    retry_attempts: 10
    retry_max_duration_secs: 300
    timeout_secs: 60

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on our guidelines.
• No. A maintainer will apply the no-changelog label to this PR.

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-10T02:49:30.796711Z	bc6916f	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[vaughnw128]

I have read the CLA Document and I hereby sign the CLA

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c3316fcec6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Exclude skipped rows from sent-event telemetry

Whenever do_nothing skips a row, this records it as discarded but leaves the original request metadata unchanged; the returned PostgresResponse consequently makes DriverResponse::events_sent increment component_sent_events_total for every input event. A four-event batch with two conflicts therefore reports four sent and two discarded even though only two rows were inserted, corrupting the sink's delivery/drop accounting; construct the response telemetry from the rows actually inserted rather than the full batch.

AGENTS.md reference: AGENTS.md:L210-L210

Useful? React with 👍 / 👎.

[vaughnw128]

Confirmed this behavior with my debugging tests

[vaughnw128]

Useful comment, but I think it doesn't make much of an impact given the reporting provided by other similar sinks. A similar behavior can be seen in the New Relic sink:

vector/src/sinks/new_relic/model.rs

Lines 115 to 118 in 5606b68

	emit!(ComponentEventsDropped::<INTENTIONAL> {
	count: num_non_metric_events,
	reason: "non-metric event"
	});

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6f4f4a127a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[vaughnw128]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: bc6916f0ae

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[vaughnw128]

thank you chibi robo