Skip to content

feat(google): fetch groups in parallel#4777

Open
codrutpanea wants to merge 1 commit into
dexidp:masterfrom
codrutpanea:google-groups-parallel
Open

feat(google): fetch groups in parallel#4777
codrutpanea wants to merge 1 commit into
dexidp:masterfrom
codrutpanea:google-groups-parallel

Conversation

@codrutpanea
Copy link
Copy Markdown

Overview

What this PR does / why we need it

Fetches nested (transitive) group membership in parallel (with a concurrency limit) instead of strictly sequential API calls, so large/deep group graphs complete faster.

In my tests the loading times went form 10-15 seconds down to 2-3 seconds. I tested the changes with 20 nested groups. And I'm using this fork for over a month with no problems.

Closes #3929

Special notes for your reviewer

Copy link
Copy Markdown
Member

@nabokihms nabokihms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tackling this. Direction is right, but I'd like a few things addressed before merging.

Comment thread connector/google/google.go Outdated
Comment thread connector/google/google.go Outdated
Comment thread connector/google/google.go Outdated
Comment thread connector/google/google.go Outdated
@codrutpanea
Copy link
Copy Markdown
Author

I implemented the suggestions. I also deployed them in my environment to test the changes. Thanks for the feedback.
Also feel free to change anything in this PR if you think it will be faster and easier to merge it.

Copy link
Copy Markdown
Member

@nabokihms nabokihms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My feedback is mostly addressed - errgroup, ctx, config knob, dead param gone. Nice.
One blocker: recursive g.Go under SetLimit deadlocks. Need a regression test for it.

Other minor stuff inline. Also please squash to one commit and rename to feat(google): - sort order changed and there's a new config field, not really a refactor anymore (and do not forget to sign the commit, otherwise we cannot merge this).

Comment thread connector/google/google.go Outdated
Comment thread connector/google/google.go
Comment thread connector/google/google_test.go
Comment thread connector/google/google.go Outdated
Signed-off-by: Codrut Panea <codrut@flowx.ai>
@codrutpanea codrutpanea force-pushed the google-groups-parallel branch from 84d941d to ac189b6 Compare May 12, 2026 20:59
@codrutpanea codrutpanea changed the title refactor(google): fetch groups in parallel feat(google): fetch groups in parallel May 12, 2026
@codrutpanea
Copy link
Copy Markdown
Author

I think i addressed all the feedback, thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Recursive fetching for Google Groups causes timeouts ( >1m auth req times)

2 participants