Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support scaling up ext proc deployment #82

Closed
yuzisun opened this issue Jan 13, 2025 · 3 comments · Fixed by #98
Closed

Support scaling up ext proc deployment #82

yuzisun opened this issue Jan 13, 2025 · 3 comments · Fixed by #98
Assignees
@mathetake
Milestone
v.0.1.0

Comments

@yuzisun
Copy link
Contributor

yuzisun commented Jan 13, 2025

Currently replica number is not set in reconcileExtProcDeployment, we need to make it configurable or attach HPA.

ai-gateway/internal/controller/llmroute.go

Line 168 in a45438b

func (c *llmRouteController) reconcileExtProcDeployment(ctx context.Context, llmRoute *aigv1a1.LLMRoute, ownerRef []metav1.OwnerReference) error {

@mathetake
Copy link
Member

i will take care of this with #90 in mind

@mathetake mathetake self-assigned this Jan 14, 2025
@mathetake mathetake added this to the v.0.1.0 milestone Jan 14, 2025
@mathetake mathetake mentioned this issue Jan 14, 2025
Closed
2 tasks
@yuzisun
Copy link
Contributor Author

yuzisun commented Jan 14, 2025 •
edited
Loading

We might not be able wait for the dynamic plugin to be ready, so we can take this to support scaling for the extproc.

@mathetake
Copy link
Member

Yeah

@mathetake mathetake mentioned this issue Jan 14, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mathetake mathetake

Labels
None yet
Projects
None yet
Milestone
v.0.1.0
Development

Successfully merging a pull request may close this issue.

Adds deployment configuration for extproc envoyproxy/ai-gateway
2 participants
@yuzisun @mathetake