Allow defining CRDs from a single version

[howardjohn]

Part of #3127. Goes with a
corresponding tools change; this will fail until that merges.

This just shows DR. The tool will support both the new and old way (we
can remove the old way if we want), so we don't have to move everything
at once. We will, though. I kept it to one so its easy to review first.

[istio-testing]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[whitneygriffith]

LGTM! I like that you focused on removing the CRD generation from each version for now.

Once this is merged, do we want to wait on anything in particular before we remove the repeated protos and the old way of generation?

[whitneygriffith]

Also does it make sense to keep a central source of each API and all its API version? And maybe even template the versions value in the protos based on the central source of APIs and their versions.

[howardjohn]

Also does it make sense to keep a central source of each API and all its API version? And maybe even template the versions value in the protos based on the central source of APIs and their versions.

Seems reasonable, or could be the other way and derived from the proto as the source of truth.

Once this is merged, do we want to wait on anything in particular before we remove the repeated protos and the old way of generation?

I don't think in terms of time. I have it implemented in #3188. I split it out for reviewing sake

[howardjohn]

Discussed in TOC today; there were no objections that came up there. Feel free to comment here if you were not there/disagree/changed your mind/etc

[costinm]

For my curiosity: why don't we use the client-go ( which I think is generated from the CRD) in the pilot code ?
If I understand the code correctly, for k8s gateway we use their generated client, and most of k8s ecosystem does the same. If there are problems parsing in client-go - anyone else using that package would be impacted, I can't see how our proto translations help. There is the (uncommon) case where we send the proto over XDS - but I suspect it's a far less used code path and if needed we can convert to proto only when that feature is used, before sending over XDS.

The 'source of truth' is likely the CRD in K8S.

+1 on the PR, I can see the tradeoffs and benefits - but I still think all the proto use is too complex and not worth it in this case.

[howardjohn]

For my curiosity: why don't we use the client-go ( which I think is generated from the CRD) in the pilot code ?

Maybe I am not parsing this correctly, but I don't think that is how it works.

Client-go depends on Go structs which follow a certain shape (they can DeepCopy, have object meta, etc). It will call json.(Un)marshal on these to go from api-server to Go.

CRD schema is entirely orthogonal and not used at all on the client side (only internally in api-server).

For our types. json.Unmarshal calls protojson.Unmarshal.

For K8s core types, they have Go structs that use standard json.Unmarshal. They also generate protos from the go structs, but don't use it for the json path -- only if you use the protobuf encoding on api-server (not available for CRDs, though. We use this for the core types).

So all of the core of Istio (ignoring MCP), doesn't use protobuf wire format, but does use it for JSON/deep copy/etc.

[linsun]

LGTM

I don't see any objections from other reviewers either.

[keithmattix]

This LGTM - all of the CRDs are the same anyway, so this better reflects the current state

[keithmattix]

To move v1 as the storage version (maybe in 3 releases or so), do we just change the order?

[howardjohn]

Yep

[howardjohn]

TBH we might need something more complex, but its fairly straightforward to tweak the tooling if that comes