Discussing v4 API Changes

[ingydotnet]

This discussion started in issue #237.
That issue was copied to #251 using this script https://gist.github.com/ingydotnet/7065adced8e48514a7e41cd96b144349
Then #251 was converted into this discussion.

[ingydotnet]

Originally posted by @wwqgtxx on January 16, 2026 at 01:17 UTC
View original comment

I'd like to understand the specific meaning behind renaming Dump/Load from Marshal/Unmarshal, given that the entire Go ecosystem uses Marshal/Unmarshal as the top-level function name.

Even if it's for adding options, the standard library's encoding/json/v2 can directly add variable arguments opts ...Options to Marshal/Unmarshal: https://pkg.go.dev/encoding/json/v2#Marshal

I've read your latest v3-to-v4-migration documentation, and similar renamings have occurred in Decoder/Encoder. You've also abandoned the widely used naming conventions in the Go standard library and created a new Loader/Dumper. It can be seen that encoding/gob and encoding/xml both use Decoder/Encoder as their names.

At least from the documentation, aside from increasing the migration cost for users and creating inconsistencies with naming conventions other serialization libraries, I don't see any improvement.

So, is this change an attempt to forcibly bring naming conventions from other languages ​​(perhaps Python, whose standard library uses .dumps() and .loads()) ​​into the Golang library?

[andig]

I'd like to understand the specific meaning behind renaming Dump/Load from Marshal/Unmarshal, given that the entire Go ecosystem uses Marshal/Unmarshal as the top-level function name.

I fully agree. This in not idiomatic and makes the library confusing to use. The migration guide claims

v4 introduces a cleaner API with better naming.

I disagree. It creates confusion and even the internal naming is inconsistent:

LoadError represents an error encountered while decoding a YAML document.

Even if it's for adding options, the standard library's encoding/json/v2 can directly add variable arguments opts ...Options to Marshal/Unmarshal: https://pkg.go.dev/encoding/json/v2#Marshal

+1

I don't see any improvement.

+1

[fredbi]

+1

[ingydotnet]

Originally posted by @ingydotnet on January 16, 2026 at 15:26 UTC
View original comment

When we started thinking about options, we actually needed to change the names from Encode/Decode to EncodeWithOptions/DecodeWithOptions. See #162

Loader and Dumper are YAML specification terms and we are refactoring things (#240 was a huge start) to bring them in line with how a full YAML loader and dumper work.

Load and Dump are also not direct equivalents to Encode and Decode. They are a level above them.

A YAML loader stack is shown here: https://github.com/yaml/go-yaml/blob/main/docs/how-go-yaml-works.md#the-big-picture-paired-pipelines

Decode only went up to the Resolve level. v4 will provide support for full Construction and Representation.

This was not a cosmetic change at all. It was simply fortunate that the API names Load and Dump were not already used.

We don't have to deprecate Marshal/Unmarshal and Encode/Decode but we realized they don't offer anything special that you can't do with the Dump/Load API.

[andig]

Loader and Dumper are YAML specification terms and we are refactoring things (#240 was a huge start) to bring them in line with how a full YAML loader and dumper work.

As a Go developer I do not care for the yaml spec. I do care a lot for the Go ecosystem and efficient APIs.

Load and Dump are also not direct equivalents to Encode and Decode. They are a level above them.

This is not clear to me. In Go terminology, marshaling converts between bytes and Go representation.

We don't have to deprecate Marshal/Unmarshal and Encode/Decode but we realized they don't offer anything special that you can't do with the Dump/Load API.

What can the dump/load api do that marshal/unmarshal with options can't? Maybe this needs an example.

[ingydotnet]

Originally posted by @wwqgtxx on January 16, 2026 at 15:38 UTC
View original comment

As I mentioned earlier, Go supports variable arguments, meaning we don't need a new EncodeWithOptions/DecodeWithOptions function to seamlessly accept parameters while maintaining the original Encode/Decode usage. Furthermore, we can see that the standard library's encoding/json/v2 libraries do the same.

Additionally, you mentioned "don't have to deprecate" but based on the migration documentation, this might be unclear. It makes it seem as if the old API is deprecated, and users should or even must migrate to the new Load/Dump APIs.

From a migration perspective, this is also confusing. If simply changing the name can essentially keep the existing code unchanged, then why change the name?

[andig]

These could also have been added to marshal/unmarshal

[ingydotnet]

Originally posted by @ingydotnet on January 16, 2026 at 17:09 UTC
View original comment

As I mentioned earlier, Go supports variable arguments, meaning we don't need a new EncodeWithOptions/DecodeWithOptions function to seamlessly accept parameters while maintaining the original Encode/Decode usage. Furthermore, we can see that the standard library's encoding/json/v2 libraries do the same.

I'll call on @ccoVeille and others to explain why we couldn't simply extend Encode/Decode, or you might find it in #162 (as I mentioned earlier).

Additionally, you mentioned "don't have to deprecate" but based on the migration documentation, this might be unclear. It makes it seem as if the old API is deprecated, and users should or even must migrate to the new Load/Dump APIs.

We don't have to deprecate those the legacy API in v5, but it makes no sense to keep it as it offers nothing extra from the new API (beyond the names you prefer), and we'd have to maintain all that.
It was not my idea to deprecate. Other team members pointed it out, and I concurred that it made sense.

From a migration perspective, this is also confusing. If simply changing the name can essentially keep the existing code unchanged, then why change the name?

I tried to explain above that it is far from a name change.
This library started with just Marshal/Unmarshal but more functionality was needed which required a Encode/Decode. Now we are adding options and plugins, taking go-yaml to a whole new level. Dump/Load fit the bill perfectly.

v4 makes the transition from v3 nearly seamless. It offers options and plugins to take YAML to almost any level imaginable.

• If you switch your code to from v3 to v4 it will work as before.
• If you change Marshal/Unmarshal to Dump/Load it should work the same most of the time, with simple options to use if it doesn't.
• By using Dump/Load you will be able to customize the heck out of YAML behavior.

This go-yaml library wants to be the best YAML library, not only for Go, but also for YAML. I lead the YAML language maintenance effort as a whole, but I want to do right by both YAML and Go (and the other YAMLs I maintain). Fortunately we have a team of people who are as passionate about Go as I am about YAML.

[ingydotnet]

Originally posted by @wwqgtxx on January 16, 2026 at 17:29 UTC
View original comment

For advanced users, migrating from Decoder/Encoder to Loader/Dumper might not be a problem; most ordinary users won't call these functions, and those who do need them are capable of the migration.

However, using Marshal/Unmarshal as the top-level function of a library is more conducive to maintaining the conventions of the entire Go ecosystem, where all serialization libraries use this name as their top-level function (including but not limited to json, xml, protobuf, msgpack, toml). I think it shouldn't be migrated to Load/Dump, but rather the options and functionality included in Load/Dump should simply be added to Marshal/Unmarshal.

[ingydotnet]

We could talk about un-deprecating Marshal/Unmarshal in v5 and leaving them as is. We'll have a long time to sort that out.

[ingydotnet]

I read through https://github.com/yaml/go-yaml/blob/main/docs/v3-to-v4-migration.md and I agree it could be improved.
I'll make a PR for that soon.

[ingydotnet]

@wwqgtxx One other thing that Dump/Load allowed us to do was change the default options without breaking existing code.

[wwqgtxx]

Based on the current changes, the only style difference between V3 and V4 is in 2-space, compact, and yaml.V3 is already provided as an option to revert to the old behavior with one click. Therefore, wouldn't a more reasonable approach in the migration documentation be:

Inform users that we have modified the default output behavior. If these changes affect existing tests or other needs, they can manually change it to yaml.Marshal(&data, yaml.V3). This would minimize changes for everyone and allow everyone to quickly adopt the new default values.

I agree with you that a new function name for Dump/Load can make old code completely unmodifiable. But what if we release V5 in the future? Will we have to create a new set of function names again and again?

Given Go's sumver system, migrating from V3 to V4 is practically impossible without any manual intervention (at least manual modification of the import path is required). Therefore, it's unlikely that users will unknowingly use the V4 library due to passive updates. Modifying some default values ​​doesn't seem surprising; it just needs to be clearly stated in the migration documentation. (The only exception is for users who already rely on the RC version of the V4 library, but they should be aware of the risks of using the RC version.)

[ingydotnet]

I was hoping that @ccoVeille would explain here why we can't extend Encode/Decode or Marshal/Unmarshal with variadic options. @ccoVeille created an earlier API options attempt that used new API names like EncodeWithOptions etc.
I remember the reasoning being explained to me but I couldn't reexplain it here myself.
I asked Claude for help and got this explanation which I believe is very close to what was explained to me back then.

To your point on v5, by then our options system will be well understood and we can default to new v5 options, requiring people to use yaml.v4 if they upgraded but didn't want the new v5 defaults.

[ccoVeille]

The idea to not add variadic to existing method was brought when we started working on option idea.

The reason behind the fact it's a breaking change are the one in the Claude explanation you shared.

I had understood that v4 was supposed to be a version without breaking changes.

So I raised my concern, that it could break things.

The same way it was dogmatticaly decided that call to Unmarshal and Marshal should provide exactly what the v3 was doing.

Later, Load, Dump and co where introduced to follow the YAML logic. Even if it was going against the Go habit. Unmarshal, Marshal, Decode and Encode were deprecated.

I wouldn't have done these choices, but the dogma of v4 doesn't introduce changes led to this, and I accepted them.

Now, there is a debate.

I would prefer to have:

• Dump/Load in libyaml, so here the YAML concepts are respected
• Unmarshal, Marshal, Decode and Encode at the public API
  • no wording changes
  • having saner defaults
  • supporting variadic option
  • the ability to pass yaml.V3 as an option

It would be a breaking changes, but it would be assumed.

This is how I would imagine the library to be, and here I feel it means it could be achieved via a v5, if we stick with the idea
v4 has to be almost breaking changes free.

But then why having a v4? If the v5 has breaking changes? Why maintaining a version that is almost the v3?

[wwqgtxx]

I think @ccoVeille's idea is basically the same as mine: Dump/Load should reside in libyaml as a low-level or internal API, while externally, Unmarshal, Marshal, Decode, and Encode should remain unchanged, conforming to the Go style.

Regarding the question of whether to introduce break changes, my view might be a bit radical: since users will need to manually modify the import path in the codebase regardless, minor changes to function signatures are acceptable. Compared to the extensive function name replacements in the current migration documentation, this migration cost might be smaller and more easily accepted.

I still maintain my previous view: the current naming convention feels like forcibly introducing other languages' conventions into the Go library. This design might lead users to feel that since they're going to incur migration costs anyway, why not migrate to a more Go-style goccy/go-yaml?

[ingydotnet]

My current, slightly adjusted, stance is that:

• We DON'T DEPRECATE the existing v3 API in v4
  • Deprecating is not something that was asked for. We just noticed that we could.
  • I concur with @wwqgtxx that we don't want to give the impression that this is a major departure. It isn't.
• We DON'T CHANGE the existing API in v4
  • Leaving the API as-is is easiest to explain, easily defendable, and easily maintainable
  • We could still add some new options for the old API if necessary, without the WithFoo pattern
• We continue on with the new Dump/Load API in v4 as planned

TL;DR ^^

---

To be clear Unmarshal in go-yaml is not a complete YAML Load operation.
So from that PoV Load is a new, more complete, API.
One that matches the YAML spec and the PyYAML family of YAMLs, the lineage of which contains most important YAML implementations including go-yaml. (PyYAML -> libyaml -> go-yaml; is the lineage)

I am leading maintenance of PyYAML, libyaml, go-yaml, serde-yaml and am interested in helping support others in the family, including SnakeYAML and YamlDotNet. The YAML test suite failures for the family are myriad and very very similar (for obvious reasons); and can often be addressed in parallel.

For the upcoming v4 release, I still have a lot to show you, so it's hard (for you) to see the complete picture. I'm working around the clock on it. I want to make the new go-yaml a blueprint for the whole PyYAML family, as I try to plan out the future for YAML on the whole.

I've also started the YAMLStar project which will be a full featured YAML framework for every language where YAML is used.
While still in its infancy it ships a YAML loader library to seven languages including Go, and uses a parser that passes 100% of the YAML test suite.
In the not too distant future it will be configurable to expose experimental YAM features which the core team has wanted to change in the spec for years.
Obviously it would be a foolish thing to put out a new version of the YAML spec and expect people to implement it in any reasonable time frame.
YAMLStar solves that by having a complete, full-featured YAML implementation that is fully compliant with a new spec before that spec is even released.

I mention that because my thinking/visions for go-yaml is melded with my overall visions for YAML as a whole.

I see go-yaml as equally serving Go and YAML.
I want the best for both.
Sadly the YAML community is microscopic in comparison to the Go community.
In this project, so far, the YAML side falls almost entirely on me.

Apologies for this long post, but I wanted to get some of my thinking out there.
I appreciate this discussion, but I don't want to bikeshed while there's more important work to get out that shows the whole picture more clearly.
I hope you can trust my vision for a while while I get more of the v4 tasks implemented.

[ingydotnet]

Submitted #258 to not deprecate the existing APIs.

[andig]

This introduces a situation where confusion will be long-term.

[ingydotnet]

@ccoVeille We need to release our main development under a v4 tag because we made very strong and explicit commitments to major adopters at the start of this project, that the v2 and v3 branches and releases would only contain necessary security level changes.

Given the level of new additions to the library, there are many more reasons why we should do v4.

[ccoVeille]

I feel like my post was misunderstood.

Maybe this sentence caused the confusion:

But then why having a v4? If the v5 has breaking changes? Why maintaining a version that is almost the v3?

I suggested to directly ship the v5 that will have breaking changes as a v4 and forget the retro compatible v4.

v2 and v3 would have been of course unchanged.

Anyway, we can chat about this directly out of GitHub discussion

[luke-hagar-sp]

@ingydotnet Are you open to the idea of incremental parsing support? I am trying to use this library to implement LSP functionality, and Tree Sitter style byte offset handling, partial parsing and reparsing, dirty nodes in a tree, etc... would all be HUGE for downstream logic.

[luke-hagar-sp]

I realize now this discussion is for a specific set of changes, not changes in general.