Kubeflow ACLs sync is failing with 422 error

[andreyvelich]

Our regular ACLs sync is failing:

{"component":"peribolos","error":"status code 422 not one of [200], body: {\"message\":\"The request could not be processed.\",\"documentation_url\":\"https://docs.github.com/rest/orgs/members#set-organization-membership-for-a-user\",\"status\":\"422\"}","file":"k8s.io/test-infra/prow/cmd/peribolos/main.go:477","func":"main.configureOrgMembers.func1","level":"warning","msg":"UpdateOrgMembership(kubeflow, prodonjs, false) failed","severity":"warning","time":"2025-02-10T20:30:42Z"}

{"component":"peribolos","file":"k8s.io/test-infra/prow/cmd/peribolos/main.go:194","func":"main.main","level":"fatal","msg":"Configuration failed: failed to configure kubeflow members: status code 422 not one of [200], body: {\"message\":\"The request could not be processed.\",\"documentation_url\":\"https://docs.github.com/rest/orgs/members#set-organization-membership-for-a-user\",\"status\":\"422\"}","severity":"fatal","time":"2025-02-10T20:30:42Z"}

However, this GitHub handler still exists (@prodonjs) https://github.com/prodonjs

Any thoughts on this error @petr-muller @matthyx @droslean @krzyzacy @smg247 @cblecker @cjwagner @MadhavJivrajani @nikhita @Priyankasaggu11929 ?

@jlewi Did you see similar errors before with peribolos other than 404: kubernetes/test-infra#14321 ?

cc @kubeflow/kubeflow-steering-committee @franciscojavierarceo @juliusvonkohout

[jlewi]

I don't recall any problems.

[petr-muller]

Hrm, the GH API message is not really helpful for that 422. I recall we had some issues with inviting people who removed themselves from the org previously (I guess as an anti-harassment protection in GH) but I dug and the symptom was different back then (GH responded with 403, not 422):

kubernetes/test-infra#22723

But back then GH was apparently showing a better message if you used its UI, so trying to add them manually would be the thing I would try.

[cjwagner]

I don't recall seeing this error. Petr's advice sounds good though.