-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remote Build Cache #1431
Comments
I think it's already possible to instruct kustomize to use a specific directory by using the environment variables used by golang itself, see here and there However, I would too like to see some remote build cache implemented.
Would it be acceptable to have a shared temp base folder to handle cleanup globally ? |
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
Stale issues rot after 30d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
Rotten issues close after 30d of inactivity. Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
@fejta-bot: Closing this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/reopen Please re-open. The build times are excruciating. |
@dsyer: You can't reopen an issue/PR unless you authored it or you are a collaborator. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
The option to specify a map of |
Looking at https://github.com/kubernetes-sigs/kustomize/blob/master/api/internal/git/cloner.go, an alternative would be the ability to specify extra Right now this only appears to be possible using The only workaround I can see so far is to put a wrapper for the Ideally
|
/reopen |
@edrandall: You can't reopen an issue/PR unless you authored it or you are a collaborator. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
The rotten issue robot is a backlog antipattern, particularly when a project is clearly underresourced. |
Also, note that while it's tempting to think you can add a |
There might be a truly nasty way to make this work without patching Use the Here's a demo showing how the alternates mechanism works:
Or closer to how
Note it's nearly instant. Unfortunately I haven't found any improvement when testing with I suspect it would work much better if implemented in https://github.com/kubernetes-sigs/kustomize/blob/master/api/internal/git/cloner.go with a caching cloner option or something, since then it could set up a It could also populate the cache if it doesn't already exist. |
FWIW we are working on #3980, which I believe will resolve the use cases described in this issue. |
@natasha41575 That sounds cool and useful. However it will not solve the use cases here IMO. In particular, for CI systems, pre-commit hooks, and other automation, you often have almost the same kustomizations being built running repeatedly, but with a few small variations. A fully localized, cacheable tree doesn't help much with that unless the cache can be used as a base to be refreshed with only more recent updates. In those cases what's really desirable is:
Of the two, git repo caching will have the biggest benefits, and is easiest. I'm experimenting with a hacky PoC in a local kustomize branch to see if I can make it work. |
Agree w/ ^^^. I'd like to use a kustomize repo where we develop and version shared bases and components in a conventional way. e.g. resources:
- https://github.com/acme/kustomize//bases/service?ref=v1
components:
- https://github.com/acme/kustomize//components/ingress-public/overlays/dev?ref=v1
- https://github.com/acme/kustomize//components/secretstore/overlays/dev?ref=v1
- ../../components/secrets ... and to avoid the maintenance burden and duplicate code (wet not DRY?) of vendoring. Not sure this issue is closed, can the bot re-open ? =) |
Currently ClonerUsingGitExec creates a new directory within /tmp and checks out remote git repositories to that location. This is not ideal for many reasons:
I would like to propose some way to instruct kustomize to use a specific directory to check out remotes persistently, potentially via an environment variable. This would resolve all of the above and make kustomize a lot less painful to use on large projects.
This would potentially mitigate #1132 and #1147
The text was updated successfully, but these errors were encountered: