Skip to content

Conversation

@palfrey
Copy link
Member

@palfrey palfrey commented Oct 6, 2025

Description

Like #1806, but with backwards compatibility. Note, that when upgrading the config the max_decoding_message_size has moved into the http part of the config as it's set per-server.

Type of change

Please delete options that aren't relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • This change requires a documentation update

How Has This Been Tested?

bazel test //...

Checklist

  • Updated documentation if needed
  • Tests added/amended
  • bazel test //... passes locally
  • PR is contained in a single commit, using git amend see some docs

This change is Reviewable

@palfrey palfrey force-pushed the bytestream-array-config branch from dc07539 to c2f1fad Compare October 6, 2025 16:47
@palfrey palfrey force-pushed the bytestream-array-config branch from c2f1fad to bed4c17 Compare October 6, 2025 17:02
@palfrey palfrey marked this pull request as ready for review October 7, 2025 09:40
@palfrey palfrey force-pushed the bytestream-array-config branch from bed4c17 to 4a49318 Compare October 7, 2025 09:46
@palfrey palfrey force-pushed the bytestream-array-config branch from 4a49318 to 7085f1a Compare October 7, 2025 10:43
Copy link
Collaborator

@MarcusSorealheis MarcusSorealheis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First of all, this PR adds a lot more consistency with the other stores. Thank you for that.

Copy link
Collaborator

@MarcusSorealheis MarcusSorealheis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall I like this PR. All my comments are nits. One detail I will call out is that we might want to add duplicate name detection for instances. I won’t block this PR over that as we can add it easily eventually.

#[serde(default, deserialize_with = "convert_data_size_with_shellexpand")]
pub max_bytes_per_stream: usize,

/// Maximum number of bytes to decode on each grpc stream chunk.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a breaking change in security boundaries. If someone had different message size limits for different services, they’ll lose that capability.

It’s a fringe use case so should be fine but we should check with the Chromium users.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻 In theory we could do something where we support setting all the http_config stuff at a per-service level, but that would require redoing the config again. It does feel like this being supported for bytestream but not any of the other services is a bit odd TBH.

@@ -0,0 +1,42 @@
{
stores: [],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be called legacy_bytestream_config.json5 or something?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's actually the legacy config for all the services, but I'll rename it as the legacy config

@MarcusSorealheis
Copy link
Collaborator

Tests should pass twice before merging

@palfrey
Copy link
Member Author

palfrey commented Oct 7, 2025

Tests should pass twice before merging

Both ce7f767fe1a094407ed366a5837e526130af44e2 and a93af2f (whose difference is a removal of the "always a particular value" generic from ByteStreamKind) have passed cleanly. Good enough @MarcusSorealheis ?

@MarcusSorealheis
Copy link
Collaborator

More than good enough :P

@palfrey palfrey merged commit e5b0eef into TraceMachina:main Oct 7, 2025
33 checks passed
@palfrey palfrey deleted the bytestream-array-config branch October 7, 2025 21:49
@palfrey palfrey linked an issue Oct 7, 2025 that may be closed by this pull request
MarcusSorealheis pushed a commit to MarcusSorealheis/nativelink that referenced this pull request Nov 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rework the instance_name config api

2 participants