[internal/kafka] Validate franz-go configuration options early in Config.Validate #46024

carsonip · 2026-02-11T15:15:59Z

Description

Validate franz-go configuration options early in Config.Validate

Affects exporter/kafka, receiver/kafka, receiver/kafkametrics

Link to tracking issue

Testing

Documentation

Co-authored-by: Cursor <[email protected]>

Extract securityOpts helper from commonOpts and include TLS/SASL/Kerberos options in kgo.ValidateOpts calls, using context.Background(). Add test certificate files to kafkareceiver testdata. Co-authored-by: Cursor <[email protected]>

Co-authored-by: Cursor <[email protected]>

carsonip · 2026-02-11T16:21:30Z

internal/kafka/franz_client.go

+// converting config values to kgo options and calling kgo.ValidateOpts.
+func ValidateClientConfigOpts(cfg configkafka.ClientConfig) error {
+	opts := clientConfigOpts(cfg)
+	secOpts, err := securityOpts(context.Background(), cfg)


[to reviewer] had to use context.Background because tlsConfig.LoadTLSConfig takes a ctx but it isn't available in config validate code path.

carsonip · 2026-02-11T16:33:08Z

receiver/kafkareceiver/config_test.go

-							CAFile:   "ca.pem",
-							CertFile: "cert.pem",
-							KeyFile:  "key.pem",
+							CAFile:   "testdata/ca.pem",


[to reviewer] it was referencing non-existent tls files. This is now fixed by copying files into testdata and referencing them here.

That's ok for me 👍

Thanks for taking it. I was wondering if it does not make sense to dynamically generate certs at the test start, but for config test, maybe it does not take the effort 👍

I would prefer these to be dynamically generated within tests if possible, since if someone was to blindly scan the project and see static keys embedded they may panic.

Copilot

Pull request overview

This PR adds early validation of franz-go (franz-go/kgo) configuration by performing option validation during Config.Validate() for Kafka components, improving feedback on misconfiguration before runtime.

Changes:

Introduces internal/kafka.Validate{Client,Producer,Consumer}ConfigOpts helpers that translate configs to kgo.Opt and validate them.
Hooks the new validation into Config.Validate() for exporter/kafka, receiver/kafka, and receiver/kafkametrics.
Updates kafkareceiver testdata/config and adds TLS fixture PEM files to support TLS validation in tests.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`internal/kafka/franz_client.go`	Refactors franz-go option construction and adds exported validation helpers using `kgo.ValidateOpts`.
`exporter/kafkaexporter/config.go`	Validates producer/client franz-go options during exporter config validation.
`receiver/kafkareceiver/config.go`	Validates consumer/client franz-go options per configured signal during receiver config validation.
`receiver/kafkametricsreceiver/config.go`	Adds config validation that checks franz-go client options early.
`receiver/kafkareceiver/config_test.go`	Adjusts expectations and test cases to account for TLS file paths and default client/consumer configs.
`receiver/kafkareceiver/testdata/config.yaml`	Updates TLS paths to point at new testdata PEM files.
`receiver/kafkareceiver/testdata/ca.pem`	Adds CA certificate fixture for TLS tests.
`receiver/kafkareceiver/testdata/cert.pem`	Adds client certificate fixture for TLS tests.
`receiver/kafkareceiver/testdata/key.pem`	Adds private key fixture for TLS tests.
`.chloggen/kafka-validate-opts-*.yaml`	Adds changelog entries for the affected Kafka components.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

internal/kafka/franz_client.go

receiver/kafkareceiver/config.go

Copilot · 2026-02-11T16:58:02Z

receiver/kafkareceiver/config.go

+	// Validate franz-go options for each configured signal. Deprecated
+	// Topic/ExcludeTopic fields are already migrated by Unmarshal.
+	for _, sig := range []*TopicEncodingConfig{&c.Logs, &c.Metrics, &c.Traces, &c.Profiles} {
+		if len(sig.Topics) == 0 {
+			continue
+		}
+		if err := kafka.ValidateConsumerConfigOpts(c.ClientConfig, c.ConsumerConfig, sig.Topics, sig.ExcludeTopics); err != nil {


Config.Validate currently calls ValidateConsumerConfigOpts once per configured signal; that helper loads/parses TLS material via LoadTLSConfig each time. With the default config this can happen up to 4 times and repeats identical work. Consider validating the client/consumer/security options once (or caching the computed security opts) and only varying the topic-related opts per signal to avoid repeated file I/O and cert parsing during startup validation.

Suggested change

// Validate franz-go options for each configured signal. Deprecated

// Topic/ExcludeTopic fields are already migrated by Unmarshal.

for _, sig := range []*TopicEncodingConfig{&c.Logs, &c.Metrics, &c.Traces, &c.Profiles} {

if len(sig.Topics) == 0 {

continue

}

if err := kafka.ValidateConsumerConfigOpts(c.ClientConfig, c.ConsumerConfig, sig.Topics, sig.ExcludeTopics); err != nil {

// Validate franz-go options. Deprecated Topic/ExcludeTopic fields are already migrated by Unmarshal.

var allTopics []string

var allExcludeTopics []string

for _, sig := range []*TopicEncodingConfig{&c.Logs, &c.Metrics, &c.Traces, &c.Profiles} {

if len(sig.Topics) == 0 {

continue

}

allTopics = append(allTopics, sig.Topics...)

allExcludeTopics = append(allExcludeTopics, sig.ExcludeTopics...)

}

if len(allTopics) > 0 {

if err := kafka.ValidateConsumerConfigOpts(c.ClientConfig, c.ConsumerConfig, allTopics, allExcludeTopics); err != nil {

don't think we can avoid this nicely. When the component starts up and invokes NewClient, it will parse TLS files per signal anyway.

Copilot · 2026-02-11T16:58:02Z

internal/kafka/franz_client.go

+// ValidateClientConfigOpts validates the configkafka.ClientConfig by
+// converting config values to kgo options and calling kgo.ValidateOpts.
+func ValidateClientConfigOpts(cfg configkafka.ClientConfig) error {
+	opts := clientConfigOpts(cfg)
+	secOpts, err := securityOpts(context.Background(), cfg)
+	if err != nil {
+		return err
+	}
+	opts = append(opts, secOpts...)
+	return kgo.ValidateOpts(opts...)
+}
+
+// ValidateProducerConfigOpts validates the configkafka.ClientConfig and
+// configkafka.ProducerConfig by converting config values to kgo options
+// and calling kgo.ValidateOpts.
+func ValidateProducerConfigOpts(clientCfg configkafka.ClientConfig, producerCfg configkafka.ProducerConfig, timeout time.Duration) error {
+	opts := clientConfigOpts(clientCfg)
+	secOpts, err := securityOpts(context.Background(), clientCfg)
+	if err != nil {
+		return err
+	}
+	opts = append(opts, secOpts...)
+	opts = append(opts, producerConfigOpts(producerCfg)...)
+	opts = append(opts, kgo.ProduceRequestTimeout(timeout))
+	return kgo.ValidateOpts(opts...)


The new Validate*ConfigOpts helpers return raw errors from securityOpts / kgo.ValidateOpts. Those errors can be fairly low-level and may not mention which config section they came from. Consider wrapping returned errors with a short prefix (e.g., "invalid client config options" / "invalid producer config options" / "invalid consumer config options") to make Config.Validate() failures clearer for users.

consumer / producer config vs client config isn't a user-facing concept, don't think we need to tell between them in error message.

Co-authored-by: Copilot <[email protected]>

Add testdata entries and test cases to each component's TestLoadConfig to verify that kgo.ValidateOpts catches invalid configurations early. Co-authored-by: Cursor <[email protected]>

paulojmdias · 2026-02-11T20:34:11Z

go.mod/go.sum deps changes detected, please run "make gotidy" and commit the changes in this PR.

PTAL into the CI error above 🙏

carsonip and others added 5 commits February 11, 2026 15:13

Validate franz-go config in Config.Validate

a5e40cb

Refactor

69a5859

Refactor

00168f3

Add comment

a578b54

Add changelog entries for kafka config validation

3ec429a

Co-authored-by: Cursor <[email protected]>

carsonip changed the title ~~[internal/kafka] Validate franz-go config in Config.Validate~~ [internal/kafka] Validate franz-go configuration options early in Config.Validate Feb 11, 2026

carsonip and others added 2 commits February 11, 2026 16:14

Validate TLS and auth options in Config.Validate

9b8f090

Extract securityOpts helper from commonOpts and include TLS/SASL/Kerberos options in kgo.ValidateOpts calls, using context.Background(). Add test certificate files to kafkareceiver testdata. Co-authored-by: Cursor <[email protected]>

Fix rangeValCopy lint: use pointer slice in Validate loop

409aab5

Co-authored-by: Cursor <[email protected]>

carsonip commented Feb 11, 2026

View reviewed changes

carsonip marked this pull request as ready for review February 11, 2026 16:22

carsonip requested review from a team, MovieStoreGuy, axw and dmitryax as code owners February 11, 2026 16:22

github-actions bot assigned VihasMakwana Feb 11, 2026

github-actions bot added receiver/kafka receiver/kafkametrics internal/kafka exporter/kafka labels Feb 11, 2026

github-actions bot requested review from paulojmdias and pavolloffay February 11, 2026 16:27

carsonip mentioned this pull request Feb 11, 2026

[pkg/kafka/configkafka] Add connection_idle_timeout for franz-go #45321

Draft

carsonip commented Feb 11, 2026

View reviewed changes

paulojmdias requested review from Copilot and removed request for paulojmdias and pavolloffay February 11, 2026 16:45

Copilot started reviewing on behalf of paulojmdias February 11, 2026 16:47 View session

Copilot AI reviewed Feb 11, 2026

View reviewed changes

carsonip and others added 2 commits February 11, 2026 17:00

Rearrange functions

bf82c00

Update receiver/kafkareceiver/config.go

731de1f

Co-authored-by: Copilot <[email protected]>

github-actions bot requested review from paulojmdias and pavolloffay February 11, 2026 17:07

Add test cases for invalid franz-go config in TestLoadConfig

85ba429

Add testdata entries and test cases to each component's TestLoadConfig to verify that kgo.ValidateOpts catches invalid configurations early. Co-authored-by: Cursor <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[internal/kafka] Validate franz-go configuration options early in Config.Validate #46024

[internal/kafka] Validate franz-go configuration options early in Config.Validate #46024

carsonip commented Feb 11, 2026 •

edited

Loading

Uh oh!

carsonip Feb 11, 2026

Uh oh!

carsonip Feb 11, 2026

Uh oh!

paulojmdias Feb 11, 2026

Uh oh!

MovieStoreGuy Feb 12, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

carsonip Feb 11, 2026

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

carsonip Feb 11, 2026 •

edited

Loading

Uh oh!

paulojmdias commented Feb 11, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[internal/kafka] Validate franz-go configuration options early in Config.Validate #46024

Are you sure you want to change the base?

[internal/kafka] Validate franz-go configuration options early in Config.Validate #46024

Conversation

carsonip commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Link to tracking issue

Testing

Documentation

Uh oh!

carsonip Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

carsonip Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

paulojmdias Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

MovieStoreGuy Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

carsonip Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

carsonip Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paulojmdias commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

carsonip commented Feb 11, 2026 •

edited

Loading

carsonip Feb 11, 2026 •

edited

Loading

paulojmdias commented Feb 11, 2026 •

edited

Loading