v1.18.0

This is expected to be the final release of this module with a v1.x version. (All subsequent releases are expected to be v2.x.)

The main change here is to insulate users from any future backwards-incompatible changes in bufbuild/protocompile. Previous releases (between 1.15 and 1.17, inclusive) depended on this module, but it hasn't yet released a stable v1.0 version, so it has occasionally been a turbulent dependency. More breaking changes are expected since it is expected to get a totally new compiler and API before it reaches v1.0. So anyone still using v1 of this repo should update to this new release before then to avoid being broken when that happens.

"github.com/jhump/protoreflect/desc/builder"

Changes/fixes:

• If resolving custom options and extensions resulted in a generated extension description created by the older v1 of protoc-gen-go, a nil-dereference panic could occur. This has been fixed. Instead, the legacy extension that would otherwise have caused a panic will now either be ignored or, if BuilderOptions.RequireInterpretedOptions is set to true, result in an error that not all options could be interpreted.
  • The older v1 has import path "github.com/golang/protobuf/protoc-gen-go". It was replaced in 2020 by the v2, which is at path "google.golang.org/protobuf/cmd/protoc-gen-go".

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• The internal compiler no longer comes from a separate Go module. Instead, this module now contains a fork of bufbuild/protocompile, which will maintain stability even when the upstream repo diverges with unstable/breaking changes. The only user-visible changes would be related to Editions support, thanks to unreleased changes in upstream that were included in this repo's fork. (See more below.)

Additions:

• This adds a field named AllowExperimentalEditions to the protoparse.Parser struct. When set to true, the parser will accept and return descriptors for files that use Edition 2024 (the only experimental edition at this time). It is experimental because it does not implement all of the rules and semantics of Edition 2024. But it does provide the ability to parse and compile files that include new language features in Edition 2024, such as visibility modifiers (export and local) as well as the option import modifier.

"github.com/jhump/protoreflect/desc/protoprint"

Changes/fixes:

• If resolving custom options and extensions resulted in a generated extension description created by the older v1 of protoc-gen-go, a nil-dereference panic could occur. This has been fixed. Instead, the legacy extension that would otherwise have caused a panic is ignored and will not be included in the printed output (since it would result in an invalid source file that could not be compiled).

v2.0.0-beta.2

This is a pre-release for an upcoming v2.0.0 of this module.

Once we have more test coverage we'll create a proper release candidate, but for now we're considering this a "beta" of the next iteration of this repo.

(Note: there was also a v2.0.0-beta.1 tag, but it should be ignored, hence there being no release information associated with it.)

What's Changed?

A lot has changed!

This version finally drops the use of the old "v1 API" of the Protobuf Runtime for Go (github.com/golang/protobuf) and instead is built on the current "v2 API": google.golang.org/protobuf.

The v2 API introduced support for descriptors, reflection, and dynamic messages into the core runtime. That means several packages from v1 of this repo no longer exist.

Removed Packages

These packages were marked deprecated in v1.17.0. They have all been dropped for v2. See the list below for suitable replacements:

• github.com/jhump/protoreflect/codec
  If you're looking for a package to inspect or generate the Protobuf binary format, see google.golang.org/protobuf/encoding/protowire.

• github.com/jhump/protoreflect/desc
  If you're looking to use descriptors, see google.golang.org/protobuf/reflect/protoreflect. If you're looking to create descriptors from descriptor protos (or vice versa), see google.golang.org/protobuf/reflect/protodesc.

• github.com/jhump/protoreflect/desc/protoparse
  If you're looking to parse Protobuf source IDL and compile it into descriptors, see github.com/bufbuild/protocompile.

• github.com/jhump/protoreflect/dynamic
  If you're looking for dynamic message capability, see google.golang.org/protobuf/types/dynamicpb. If you're looking for registries, for managing message and extension types to be used when unmarshaling message data, see google.golang.org/protobuf/reflect/protoregistry.

Renamed/Moved Packages

The other functionality in v1 of this module, the stuff that is retained in v2, has been moved around. Because the desc and dynamic directories were removed (since they were home to now-removed packages), most other packages have been shuffled around a little:

• github.com/jhump/protoreflect/desc/builder => github.com/jhump/protoreflect/v2/protobuilder

• github.com/jhump/protoreflect/desc/protoprint => github.com/jhump/protoreflect/v2/protoprint

• github.com/jhump/protoreflect/desc/sourceinfo => github.com/jhump/protoreflect/v2/sourceinfo
  (This includes the cmd/protoc-gen-gosrcinfo command therein.)

• github.com/jhump/protoreflect/dynamic/grpcdynamic => github.com/jhump/protoreflect/v2/grpcdynamic

• github.com/jhump/protoreflect/dynamic/msgregistry => github.com/jhump/protoreflect/v2/protoresolve/remotereg
  This one has the most significant changes to its API. It now provides an interface compatible with types in the new github.com/jhump/protoreflect/v2/protoresolve package.

• github.com/jhump/protoreflect/grpcreflect => github.com/jhump/protoreflect/v2/grpcreflect
  (This one has not really been moved; the only difference in the import path is the obligatory v2 component.)

New Packages

• github.com/jhump/protoreflect/v2/protodescs
  This package provides a few random helpers for working with descriptors and descriptor protos.

• github.com/jhump/protoreflect/v2/protomessage
  This package provides some helpers for working with proto messages.

• github.com/jhump/protoreflect/v2/protoresolve
  This package provides numerous nominal interface types for kinds of "resolvers". Resolvers are types that can answer queries about descriptors. This package also provides a *Registry type, which builds on the standard API's *protoregistry.Files and *protoregistry.Types types and provides new functionality:

  • It is thread-safe and can be used from multiple goroutines.
  • It can track the original descriptor proto from which a descriptor was built. This allows more convenient and efficient access than have to re-construct the descriptor proto (via the standard runtime's protodesc package).
  • It is a single value that can be used to query for descriptors as well as extension and message types. (The type implementations provided are all dynamic types.)

  This package has the broadest new API and is the most likely to potentially change between this beta and a final v2.0.0 release.

• github.com/jhump/protoreflect/v2/sourceloc
  This package provides several helpers for working with instances of protoreflect.SourceLocation and protoreflect.SourcePath.

v1.17.0

This release deprecates some packages that have newer alternatives in the Protobuf runtime. It also includes some small additions to the grpcreflect and desc/sourceinfo packages. The rest of the changes are bug fixes.

"github.com/jhump/protoreflect/desc"

Changes:

• This package is now deprecated. Though use of this package is still needed in order to use a handful of other packages in this repo, many usages should instead prefer the "google.golang.org/protobuf/reflect/protoreflect" package in the Protobuf runtime. In a v2 of this repo, this package will no longer be present.
• A performance improvement was contributed that speeds up the MessageDescriptor.FindFieldByName method.

"github.com/jhump/protoreflect/desc/protoparse"

Changes and bugfixes:

• This package is now deprecated. Most usages should instead prefer the "github.com/bufbuild/protocompile" package. There are some behavioral differences that we hope to address with additional API and enhancements to protocompile. But the protoparse package here is just a veneer on top of protocompile and we plan to do as little maintenance as possible here. In a v2 of this repo, this package will no longer be present.
• The protocompile dependency has been updated to use a more recent version of that package. This addresses potential panics when calling Parser.ParseButDontLink.

"github.com/jhump/protoreflect/desc/sourceinfo"

Changes and bugfixes:

• A possible panic when calling sourceinfo.GlobalFiles.FindDescriptorByName has been fixed.
• Descriptors returned from the sourceinfo.GlobalFiles registry or from the various Wrap* functions could induce panics in user code if querying for non-existent elements. For example, calling messageDescriptor.Fields().ByNumber(123) could return a typed-nil value on failure, instead of a nil interface. This would result in nil-dereference panics if methods were called on this later. These have been fixed.
• Descriptors returned from the sourceinfo.GlobalFiles registry and from the various Wrap* functions had concrete types defined in this package, not in the Protobuf runtime. But the protoreflect.Descriptor interface has a "do not implement" marker, indicating that concrete types implementing the interface should not be implemented outside of that module. To avoid future issues with the Protobuf runtime, there are no more implementations in this repo. Instead of wrapping other descriptor implementations (via embedding, and then overriding some methods), this package now builds new descriptors, using the protodesc package in the Protobuf runtime module.
• The various Wrap* functions are now deprecated. The name is no longer accurate since they no longer return wrapper values. Also the new implementation, which builds new descriptors, can fail (though unlikely), but the existing signature of these functions did not allow for returning an error. So these functions could possibly panic in the face of malformed input descriptors. Users should use the new AddSourceInfoTo* versions of these functions instead.

Additions:

• Adds AddSourceInfoTo* functions to replace the Wrap* functions. These functions augment the input descriptors by adding source code information to them.

"github.com/jhump/protoreflect/dynamic"

Changes:

• This package is now deprecated. Most usages should instead prefer the "google.golang.org/protobuf/types/dynamicpb" package in the Protobuf runtime. In a v2 of this repo, this package will no longer be present.

"github.com/jhump/protoreflect/grpcreflect"

Additions:

• Adds a new NewClientV1 function to return a client that uses v1 of the server reflection service. This fills the gap in the existing factory functions, NewClientV1Alpha and NewClientAuto.
• Adds a new AllowFallbackResolver method to the *grpcreflect.Client type. This allows the client to be configured with an alternate resolver that can be used as a backup when trying to satisfy dependencies, for cases where the remote server is non-authoritative and unable to provide all of them.

v1.16.0

This release adds support for Protobuf Editions to many of the packages. Note, however, that the desc/protoparse package does not yet work to parse Protobuf Editions source files. This release also includes a handful of bug-fixes.

"github.com/jhump/protoreflect/desc"

Additions:

• Adds an Edition method to *desc.FileDescriptor, which returns non-zero for files that use Protobuf Editions (instead of syntax "proto2" or "proto3").

Changes/fixes:

• Fixes an issue in the various Wrap* functions, that could result in a descriptor hierarchy where the same descriptors were incorrectly present multiple times. This could happen if a file in the transitive dependency graph being wrapped had more than one incoming edge (e.g. same file appearing in import statements from two or more other files).

"github.com/jhump/protoreflect/desc/builder"

Additions:

• Adds an Editions field and SetEditions method to *builder.FileBuilder, for creating files that use Protobuf Editions.

"github.com/jhump/protoreflect/desc/protoprint"

Changes/fixes:

• Now correctly prints the contents for files that use Protobuf Editions. Previously, if given a descriptor for a file used Protobuf Editions, it could generate invalid source output.
• Fixed a bug when Printer.CustomSortFunction was set wherein enum values could be printed in incorrect order, resulting in invalid source output. In syntax "proto3" and in open enums in Protobuf Editions, the first value must be the one with a zero numeric value.

"github.com/jhump/protoreflect/grpcreflect"

Additions:

• Adds a new AllowMissingFileDescriptors method to *grpcreflect.Client that allows users to opt-in to a lenient mode that will try to return schemas even when some files are unavailable. This will only be successful in cases where the missing files only provide custom options or are unused imports. This restores some lenience that was inadvertently provided (due to bugs and missing validation checks) when versions of this repo prior to v1.15 were combined with versions of the google.golang.org/grpc module prior to v1.61.

v1.15.6

This is a bugfix release to address recently identified regressions in the desc/protoparse package -- things that worked in v1.14.1 but did not work in v1.15.x.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• The documentation for the Parser.ImportPaths field specifies that the field is not used for calls to the Parser.ParseFilesButDoNotLink method. This is because import paths do not need to be resolved if not linking. But as of v1.15.0, the ImportPaths field was being used for that method, and the difference in behavior could break existing programs. This has been fixed, and the behavior once again matches the spec.
• As of v1.15.0, if a path provided to Parser.ParseFiles were resolved as descriptors (via the Parser.LookupImport or Parser.LookupImportProto functions) instead of as source code (via the Parser.Accessor function or the default action of os.Open), the parse operation would fail with a "no such file" error. This would succeed with v1.14.1, with the descriptor being used as the input for the file instead of the file being parsed from source. So working programs could have been broken by an upgrade to v1.15.x. The package has been changed back to the v1.14.1 behavior.

v1.15.5

This is a bugfix release to address bugs in the desc/builder, desc/protoparse, and grpcreflect packages and a compatibility issue with the most recent version of github.com/bufbuild/protocompile.

"github.com/jhump/protoreflect/desc/builder"

Changes/fixes:

• A regression was introduced in v1.15.0 of this package that could cause it to fail to include some imports in a file for custom options. If the custom option in question was in a known transitive dependency, the builder could fail to add the necessary import. This has been fixed.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• As of v1.15.0, this package uses the github.com/bufbuild/protocompile package under the hood as the parser. But that package is still using major version zero, since it's API has not completely stabilized. A recent release (v0.8.0) included backwards-incompatible changes that caused this repo to no longer compile. This release addresses the compile errors: this repo now uses that latest release of protocompile and correctly compiles against it.
• The use of v0.8.0 of protocompile fixes some issues in the desc/protoparse package that could cause it to incorrectly accept or reject some Protobuf source files. See the release notes for protocompile v0.8.0 for more details.

"github.com/jhump/protoreflect/grpcreflect"

Changes/fixes:

• The grpcreflect.NewClientAuto function (added in v1.14.0) returns a client that automatically falls back to using v1alpha of the reflection protocol if the server does not support v1. However, it relied on the server returning an "unimplemented" error to decide to fall back, but this condition has been observed to sometimes instead manifest as an "unavailable" error (in proxies that fail to write a 404 error before closing the connection). In those cases, the client was not correctly falling back to v1alpha. This release remedies that, so it can fallback to v1alpha when this category of error occurs.

v1.15.4

This is a bugfix release to address bugs in the desc/protoprint package and a compatibility issue with the most recent version of github.com/bufbuild/protocompile.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• As of v1.15.0, this package uses the github.com/bufbuild/protocompile package under the hood as the parser. But that package is still using major version zero, since it's API has not completely stabilized. A recent release (v0.7.0) included backwards-incompatible changes that caused this repo to no longer compile. This release addresses the compile errors: this repo now uses that latest release of protocompile and correctly compiles against it.

"github.com/jhump/protoreflect/desc/protoprint"

Changes/fixes:

• When a string literal included a double-quote character ("), it would get printed incorrectly leading to either an incorrect string literal in some cases or an invalid and uncompilable source file in others. This has been fixed.
• When sorting elements in the file, if the file used proto3 syntax and had an enum with negative numeric values, the result would include the negative values first, before the zero value. This is not allowed in proto3 syntax, which requires that the zero value be the first value, so the result would be an invalid and uncompilable source file. This has been fixed.
• When a source file is parsed into a FileDescriptorProto but never linked, the descriptor may be missing the type field of a FieldDescriptorProto (since it is not known whether a named type refers to a message, group, or enum), and it may have options left in the uninterpreted_options field of the various options messages, which has a peculiar representation for aggregate values. When printing a descriptor that had these characteristics, they were not correctly handled, so the output of the printer would be an invalid and uncompilable source file. Both of these cases have been fixed.

v1.15.3

This is a bugfix release to address a panics in the desc/builder and desc/protoparse packages.

"github.com/jhump/protoreflect/desc/builder"

Changes/fixes:

• When building an enum where some values in an EnumBuilder had numbers explicitly assigned and others did not, a panic could occur. This panic has now been fixed. When there is a mix of explicit value numbers and implicit numbers, the implicit values will be auto-assigned available numbers (starting at zero) while the explicit values will use the configured number.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• When the protoparse package was overhauled in v1.15.0, a bug was inadvertently introduced to the functionality related to the Parser.InferImportPaths flag. If the files being processed imported a file for which an AST was not available (which is actually quite common, since they may import a standard file such as "google/protobuf/empty.proto"), a panic would occur. This panic has been fixed.
• This release includes some other improvements related to the same Parser.InferImportPaths flag which now allow it to better detect file layouts and remedy import mismatches in more scenarios.

v1.15.2

This is a bugfix release to address issues found in the desc/protoparse package.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• A fix was added to v1.15.1 so that descriptors returned from this package would use statically known extension types for custom options, and custom options in the source that were not statically known would be represented as unrecognized fields. However, the fix was incomplete: only the files whose names were explicitly given to the Parser were updated in this fashion. Other files in the transitive dependency (i.e. imports) did not have options represented this way. As of this release, the fix is complete and all descriptors returned by this package should have the same representation for custom options as in v1.14 and earlier.
• As of v1.15.0, this package uses the github.com/bufbuild/protocompile package under the hood as the parser. But that package is still using major version zero, since it's API has not completely stabilized. A recent release (v0.6.0) included backwards-incompatible changes that caused this repo to no longer compile. This release addresses the compile errors: this repo now uses that latest release of protocompile and correctly compiles against it.

v1.15.1

This is small bugfix release to address regressions reported in v1.15.0.

It also fixes a panic bug that can occur when a program includes both the grpcreflect package from this module and v1.52.0 or newer of the "google.golang.org/grpc" module.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• As of v1.15.0, descriptors returned from this package would include dynamic extension fields to represent option values for custom options that were defined in the set of files being compiled. This was a regression from all previous versions, where custom options were represented using statically known extension fields (i.e. extensions that were generated to Go code and linked into the calling program) or as unrecognized fields. This breaks some usages, where calling code is using type assertions of the extension value to a generated struct (which fails when the value is a dynamic message value). This has been fixed, and the behavior from v1.14.1 and before has been restored.
• As of v1.15.0, a data race was possible. This is was a regression, where #236 revealed its ugly head again. The issue would occur if a parse operation used the Parser.LookupImport or Parser.LookupImportProto field to supply some dependencies as descriptors instead of as source. The race occurred when the same descriptor was referenced from different, concurrent parse operations: one goroutine would modify the descriptor during linking, while another goroutine was trying to access it (without synchronization). It has now been fixed (via a fix in the underlying protocompile library: bufbuild/protocompile#103).

"github.com/jhump/protoreflect/grpcreflect"

Changes/fixes:

• This package includes its own internal copy of generated Go code corresponding to grpc/reflection/v1/reflection.proto. This was because, historically, there were no released Go packages that included it. But as of v1.52.0 of the grpc-go runtime, another package now provides this. If both packages are linked into the same program, a panic will occur during program initialization due to there being multiple, conflicting definitions of these types. As of v1.57.0 of grpc-go, this init-time panic is guaranteed to happen. This panic is resolved in this version (the internal copy is now registered using a non-conflicting name). An example panic stack-trace follows:

  panic: proto: file "reflection.proto" has a name conflict over grpc.reflection.v1.ErrorResponse
  	previously from: "google.golang.org/grpc/reflection/grpc_reflection_v1"
  	currently from:  "github.com/jhump/protoreflect/grpcreflect/internal/grpc_reflection_v1"
  See https://protobuf.dev/reference/go/faq#namespace-conflict
  
  goroutine 1 [running]:
  google.golang.org/protobuf/reflect/protoregistry.glob..func1({0x1075580e0?, 0xc0005e2800?}, {0x1075580e0?, 0xc0005e2840})
          /.../go/pkg/mod/google.golang.org/protobuf@v1.31.0/reflect/protoregistry/registry.go:56 +0x264
  google.golang.org/protobuf/reflect/protoregistry.(*Files).RegisterFile.func1({0x107569940, 0xc0006fc5b0})
          /.../go/pkg/mod/google.golang.org/protobuf@v1.31.0/reflect/protoregistry/registry.go:154 +0x2ec
  google.golang.org/protobuf/reflect/protoregistry.rangeTopLevelDescriptors({0x10756d6c8, 0xc0004e76c0}, 0xc00071ce28)
          /.../go/pkg/mod/google.golang.org/protobuf@v1.31.0/reflect/protoregistry/registry.go:417 +0x148
  google.golang.org/protobuf/reflect/protoregistry.(*Files).RegisterFile(0xc0005ce258, {0x10756d6c8?, 0xc0004e76c0?})
          /.../go/pkg/mod/google.golang.org/protobuf@v1.31.0/reflect/protoregistry/registry.go:149 +0x708
  google.golang.org/protobuf/internal/filedesc.Builder.Build({{0x1072414ee, 0x45}, {0x108070e00, 0x6bd, 0x6bd}, 0x0, 0x8, 0x0, 0x1, {0x10755ec98, ...}, ...})
          /.../go/pkg/mod/google.golang.org/protobuf@v1.31.0/internal/filedesc/build.go:112 +0x22c
  google.golang.org/protobuf/internal/filetype.Builder.Build({{{0x1072414ee, 0x45}, {0x108070e00, 0x6bd, 0x6bd}, 0x0, 0x8, 0x0, 0x1, {0x0, ...}, ...}, ...})
          /.../go/pkg/mod/google.golang.org/protobuf@v1.31.0/internal/filetype/build.go:138 +0x22c
  github.com/jhump/protoreflect/grpcreflect/internal/grpc_reflection_v1.file_reflection_proto_init()
          /.../go/pkg/mod/github.com/jhump/protoreflect@v1.14.0/grpcreflect/internal/grpc_reflection_v1/reflection.pb.go:947 +0x558
  github.com/jhump/protoreflect/grpcreflect/internal/grpc_reflection_v1.init.0()
          /.../go/pkg/mod/github.com/jhump/protoreflect@v1.14.0/grpcreflect/internal/grpc_reflection_v1/reflection.pb.go:818 +0x24