Local Git Repos

[criztovyl]

I'd like to build kustomizations that refer to refs in local git repositories

resources:
- git::/absolute/path/to/my/repos/a-repo/.git/path/in/repo?ref=some-ref

from my understanding git natively supports file:// URLs and I would also be happy with

resources:
- git::file:///absolute/path/to/my/repos/a-repo/.git/path/in/repo?ref=some-ref

The expected outcome would be that Kustomize accepts I passed in absolute path to a .git repository and uses the git loader for it.

The current way I am using involves (ab-)using the special handling of _git in URLs and git's url.*.insteadOf:

# ~/git.config
[url "/absolute/path/to/my/repos"]
    insteadOf = _git/

resource:
- git::_git/a-repo/path/in/repo?ref=some-ref

URLs tried so far (path to repo is /absolute/path/to/repo/.git, path in repo is path/in/repo, ref is a-ref):

1. git::/absolute/path/to/repo/.git/path/in/repo?ref=a-ref
2. file:///absolute/path/to/repo/.git/path/in/repo?ref=a-ref
3. file://localhost/absolute/path/to/repo/.git/path/in/repo?ref=a-ref
4. git::file:///absolute/path/to/repo/.git/path/in/repo?ref=a-ref
5. git::file://localhost/absolute/path/to/repo/.git/path/in/repo?ref=a-ref

[k8s-ci-robot]

@criztovyl: This issue is currently awaiting triage.

SIG CLI takes a lead on issue triage for this repo, but any Kubernetes member can accept issues by applying the triage/accepted label.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[thatsmydoing]

Just out of curiosity, why not just refer to them as a standard path instead of using a git url? Is the ref part important?

[ciis0]

Yes, the ref is the important part.

For example while in one project we distribute the project's base via git, the customer should not use our git repo in live systems to reduce the number of (moving) parts that can fail.
we cannot assume customer has an existing git "server". Setting that up again involves more moving parts that can break, and, as the project is quite simple, setting one up also is overkill. ^^

thus, the simplest way to just have a defined location on local system where the (base) repo is and can be referenced by customer in it's overlays. (having multiple versions / different refs, e.g. one in staging, one in prod.)

[criztovyl]

@KnVerey you self-assigned but did not update triage status, what does that mean process-wise? :)

[KnVerey]

you self-assigned but did not update triage status, what does that mean process-wise? :)

In this case it means I claimed the issue to read carefully and comment during a bug scrub, then lost track of it for a bit. Sorry for the confusion. 😅

I empathize with the desire to remove dependencies from the mix, and indeed I'd say the best practice is to commit the hydrated build results themselves to git, in cases where that is an option, eliminating the entire build from the release process. If that's not an option, two alternatives come to mind based on what I understand of your situation so far:

1. A standard practice is to have the customer clone the base directory as a sibling to the overlays they create and rebase periodically as described here: https://kubectl.docs.kubernetes.io/guides/config_management/offtheshelf/ (old doc, but conceptually accurate)
2. This open proposal that is still looking for buy-in / contributors to implement it: kustomize localize command to pull all remote data #3980

Generally speaking, Kustomize intends for configuration to be entirely declarative, encapsulated by the root Kustomization directory, and not subject to side-effects based on the state of the machine on which the build is run (same Kustomization directory = same result). It's true that there are ways to bypass this, most notably by disabling the load restrictor with a flag. The remote git source feature itself also contradicts this to some extent. But I would like to promote those best practices to the extent possible, which makes me very hesitant about a new feature that would add another way to bypass the intended structure. I hope that explanation makes sense, and please let me know what you think of the above alternatives.

/unassign

[criztovyl]

Thank you for the explanation, it does make sense to me.

For me the question comes up how far the same-dir-same-result goes.
For example to me it seems bases are the core violation of that rule, because they change independently of root dir (same for components), but see further below.

Alternative 1 assumes users quite familiar and comfortable with Git: It will mean you need different branches for prod and staging if you have different versions on them. That will get quite hairy with users only knowing git basics. We are not willing to assume experienced git users, thus this does not fit us.

What I have grasped from alternative 2 it does not to really change the status quo we're having trouble with.
The dirs to be localized would still be subject to the current limitations where you can reference out-of-root residing-in-a-git-repo-bases only via https and ssh.
It would not remove the moving parts required for https/ssh we want to keep out (because to support future base updates you must not edit the localized dir, thus if the customer needs to updates his overlay our git server must be up, even if the updates do not regard the base).

Thinking around lead me to a third alternative.

Normally I would be happy with kustomize just fetching out-of-dir resources from anywhere, me, the overlay author ensuring same-dir-same-result is meaningfully respected by using static refs, e.g. tags or commit hashes. This works fine for git via https and ssh, but not for file scheme.

While you can already shoot yourself in the foot if you use "dynamic" refs (branches), you don't want an additional options for foot-shooting, so you don't want to add git file scheme, agreed.

Thinking further, and speaking strictly, the git loader would need to be dropped completely to prevent foot-shooting. It's kept because you can ensure same-dir-same-result using static refs, but it's still quite easy to shoot yourself in the foot using dynamic refs. (Also dropping it without an alternative would disallow even static refs that do work towards same-dir-same-result.)

Even stricter you would need to drop bases and components as they can change independently of the kustomization root. BUT most users will have base, overlays and components together in one SCM repo. Thus, to shoot yourself in the foot you'll need to be quite deliberate about that. So, not being overly strict, they are kept.

My thoughts now went into the direction of how to be strict, but still ensure users already using static refs will not be disgruntled.

I thought out a remoteResources field for the kustomization and an update command.

The field will contain resource URLs that do not need to adhere to same-dir-same-result -- but they are not automatically downloaded or updated. Instead kustomize downloads the remote bases only when the user runs kustomize update and places them into an folder inside the root dir, which users are encouraged* to add to SCM, and can be referenced in the normal resources field then. (I thought about remoteResources as a list of objects with name and resource attributes to keep away the complexity to automatically determine a name, not sure if that's neccessary.)
For users with already static refs this would add a step they did not have before, but they only would need it once, so that might not be as bad.
Only users that have dynamic refs would need to run update more often -- but that would only further align them with your same-dir-same-result best-practice.
And added bonus would be that you can reference multiple components from the same remote resource without needing to include the same repo-"prefix" many times.

I'll open a dedicated feature request for remoteResources/kustomize update.

* for remoteResources they are even forced to do so -- build explicitly should not do anything about this field, but unless git loader is completely dropped for resources they can still use automatically-downloaded dynamic refs there.

[criztovyl]

I understand it's not indended to be implement as suggested by me, closing :)