feat(config)!: parse configuration before interpolation by pront · Pull Request #26475 · vectordotdev/vector

pront · 2026-09-24T15:37:58Z

Summary

Finish the parse-first rollout using the helpers merged in #26441 and #26450. Configuration loading now parses YAML, TOML, or JSON before interpolating environment variables and secrets, then coerces values with the generated schema before deserialization. The HTTP provider uses the same loader.

Substituted quotes and newlines remain string content instead of changing the configuration structure. Environment interpolation remains opt-in. Comments and keys are no longer interpolated; TOML and JSON placeholders must be quoted, and values cannot expand into multiple array elements.

This enacts the structural-interpolation deprecation announced in 0.57.0 for removal in 0.59.0. Includes migration guidance and updated environment-variable and secrets documentation.

References

Vector configuration

With environment interpolation enabled, MY_COUNT=42 is converted to an integer:

sources:
  demo:
    type: demo_logs
    format: json
    count: "${MY_COUNT}"
sinks:
  output:
    type: console
    inputs: [demo]
    encoding:
      codec: json

How did you test this PR?

99 configuration and HTTP-provider tests passed, covering YAML/TOML/JSON coercion, namespaced files, disabled interpolation, and secret retrieval.
make check-generated-docs: 248 component examples passed (4 skipped); no CUE or generated-example changes.
Full default/vendored binary smoke tests: multiline environment and file-secret values are emitted unchanged to a console sink.
Formatting, Markdown, changelog, and deprecation checks.
Scoped workspace Clippy passes with only the existing clippy::unused_async failure in unchanged src/test_util/http.rs:51 excluded.

Does this PR include user facing changes?

Yes. Breaking changelog and migration guidance included.
No. A maintainer will apply the no-changelog label to this PR.

Contributor Guidelines

Please read our Vector contributor resources.
Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
Before pushing, follow our pre-push guidance.
After a review is requested, please avoid force pushes to help us review incrementally.
- Feel free to push as many commits as you want. They will be squashed into one before merging.
- For example, you can run git merge origin master and git push.

chatgpt-codex-connector · 2026-09-24T15:38:15Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-09T19:57:23.052963Z	`f6409b3`	New commits
🔒 Security Review	✅ Completed 2026-10-09T19:53:52.781819Z	`f6409b3`	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ecaeb9b7e1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

datadoghq-integration

Namespaced test files skip schema coercion. Directory fragments also merge before coercion, so valid interpolated scalar values can cause type errors.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit ecaeb9b · @DataDog review to ask questions}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f77daa9344

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-10-05T20:15:57Z

 ) -> Result<T, Vec<String>> {
-    deserialize_config_value(Value::Object(map))
+    let mut value = Value::Object(map);
+    coerce_config(&mut value)?;


Preserve boolean-or-struct configuration shorthands

When a sink uses the supported shorthand healthcheck: false, this unconditional coercion rejects the boolean as ExpectedObject before serde can process it. SinkOuter::healthcheck explicitly uses deserialize_with = "bool_or_struct", whose visitor accepts either a boolean or a map, while its generated schema describes only the struct shape; the same pattern is used by many TLS and acknowledgement fields. Teach coercion about these alternate serde input shapes, or avoid schema-validating already typed non-string values, so previously valid configurations continue to load.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-10-05T20:15:57Z

+      "what": "Environment-variable and secret placeholders in non-string positions",
+      "deprecated_since": "0.57.0",
+      "removed_in": "0.59.0",
+      "description": "Placeholders (`${VAR}` and `SECRET[backend.key]`) in structural positions of\na Vector configuration file are deprecated and will be removed in a future\nrelease. Current behavior is unchanged.\n\nAffected patterns:\n\n- Unquoted placeholders in non-string fields (`count = ${MY_COUNT}`).\n- Placeholders in map keys (`${KEY} = value`).\n- Placeholders in TOML table headers (`[${SECTION}]`).\n- Placeholders as inline array elements that expand to multiple values\n  (`inputs = [${VECTOR_INPUTS}]` where `VECTOR_INPUTS=a,b`).\n\nMigration guidance will accompany the removal PR. `envsubst` can be used today\nto pre-expand env-var placeholders into a fully static config as a workaround.\n\nSee [RFC: parse-first config interpolation](https://github.com/vectordotdev/vector/blob/master/rfcs/2026-06-09-parse-first-config-interpolation.md)\nfor the full rationale."


Describe the enacted interpolation behavior

After moving this entry into deprecations_enacted with removed_in: 0.59.0, the copied description still says the feature “will be removed in a future release,” that current behavior is unchanged, and that migration guidance will arrive later. This directly contradicts both the enacted status and this commit's parse-first behavior, so the generated deprecations page gives upgrading users obsolete guidance; update the description to explain the behavior now in effect and point to the included migration steps.

Useful? React with 👍 / 👎.

Reuse parsed input snapshots for environment interpolation, secret discovery, and configuration assembly. Preserve healthcheck, ASCII delimiter, directory merge, and metric tag compatibility, with regression tests and a released-binary comparison corpus.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4f62338e9b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-10-09T17:12:04Z

 ) -> Result<T, Vec<String>> {
-    deserialize_config_value(Value::Object(map))
+    let mut value = Value::Object(map);
+    coerce_config(&mut value)?;


Support YAML integer spellings during schema coercion

When an interpolated YAML integer uses a valid YAML spelling such as 0x2, parse-first loading leaves the substituted value as a string and this new coercion pass rejects it because parse_integer accepts only decimal Rust integer syntax. Previously interpolation occurred before YAML parsing, so count: ${COUNT} with COUNT=0x2 produced the integer 2; the newly added environment-yaml-hex-integer compatibility case also explicitly expects this to remain accepted. Extend integer coercion to the scalar formats previously recognized by the configuration parser.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-10-09T17:12:04Z

 ) -> Result<T, Vec<String>> {
-    deserialize_config_value(Value::Object(map))
+    let mut value = Value::Object(map);
+    coerce_config(&mut value)?;


Preserve comma-separated no_proxy strings during coercion

When proxy.no_proxy is configured as a comma-separated scalar such as "localhost,127.0.0.1", this new coercion pass sees NoProxy's Vec<String> schema and wraps the scalar into ['localhost,127.0.0.1'] before deserialization. NoProxy intentionally gives these shapes different semantics: its string visitor splits on commas, while its sequence visitor treats each element as one pattern. The resulting configuration therefore contains one literal hostname with a comma and sends requests for both intended hosts through the proxy. Describe NoProxy as a string-or-array schema and preserve the matching input shape instead of wrapping it before serde.

Useful? React with 👍 / 👎.

feat(config)!: parse configuration before interpolation

ecaeb9b

pront requested review from a team as code owners September 24, 2026 15:37

github-actions Bot added meta: breaking change Anything that breaks backward compatibility. docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. labels Sep 24, 2026

chatgpt-codex-connector Bot reviewed Sep 24, 2026

View reviewed changes

Comment thread src/config/loading/config_builder.rs Outdated

Comment thread src/config/loading/loader.rs Outdated

datadoghq-integration Bot reviewed Sep 24, 2026

View reviewed changes

Comment thread src/config/loading/config_builder.rs Outdated

Comment thread src/config/loading/loader.rs Outdated

pront mentioned this pull request Sep 24, 2026

feat(config)!: parse config first, then interpolate into string leaves #25905

Closed

pront added 2 commits October 5, 2026 15:01

fix(config): coerce interpolated namespaced test values

b2edec7

fix(config): defer directory scalar conflicts until coercion

f77daa9

chatgpt-codex-connector Bot reviewed Oct 5, 2026

View reviewed changes

pront added 2 commits October 9, 2026 12:32

Merge origin/master into pront-activate-parse-first

4f62338

github-actions Bot added domain: external docs Anything related to Vector's external, public documentation domain: core Anything related to core crates i.e. vector-core, core-common, etc domain: vdev Anything related to the vdev tooling labels Oct 9, 2026

chatgpt-codex-connector Bot reviewed Oct 9, 2026

View reviewed changes

chore(config): keep compatibility corpus local

f6409b3

pront mentioned this pull request Oct 9, 2026

refactor(codecs): use a validated ASCII character type #26632

Merged

1 of 2 tasks

Navigation Menu

Repository navigation

feat(config)!: parse configuration before interpolation - #26475

feat(config)!: parse configuration before interpolation#26475
pront wants to merge 6 commits into
masterfrom
pront-activate-parse-first

pront commented Sep 24, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 24, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

datadoghq-integration Bot left a comment

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Oct 5, 2026

Uh oh!

chatgpt-codex-connector Bot Oct 5, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Oct 9, 2026

Uh oh!

chatgpt-codex-connector Bot Oct 9, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

pront commented Sep 24, 2026

Summary

References

Vector configuration

How did you test this PR?

Does this PR include user facing changes?

Contributor Guidelines

Uh oh!

chatgpt-codex-connector Bot commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

datadoghq-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

chatgpt-codex-connector Bot commented Sep 24, 2026 •

edited

Loading