Research how to find the POSIX-compatible shell

[EliahKagan]

Summary 💡

It is a truism that the POSIX-compatible shell is sh and that it is located at /bin/sh. For systems where that is not so, or where it is in some important way not the whole truth, I'd like to figure out what should be done for running commands that should otherwise be run in /bin/sh.

• Common situation on Unix-like systems: it's just /bin/sh!
  • Should we use relative sh instead? POSIX does not require /bin/sh.
• Solaris and illumos: Is the POSIX sh elsewhere than /bin/sh? Which do we use?
• Git for Windows, full installation: (git root)\usr\bin\sh.exe (#1758)
  • (git root)\bin\sh.exe? (No, not SDK-compatible, see below.)
  • "Portable" Git (same as Scoop package except for Scoop's own shim)
• Git for Windows, traditional MinGit: Does it need anything special?
• Git for Windows, BusyBox-based MinGit: Does it need anything special?
• Git for Windows SDK: (git root)\usr\bin\sh.exe (report, repro)
• Windows with other identifiable MSYS2: Do we use its sh as a fallback?
• Windows with an .sh file association: Do we fall back to it if we can prove it's a shell?
• Allow specifying at runtime which POSIX shell to use git-for-windows/git#3301

Motivation 🔦

Rationale

gitoxide runs some commands in shells. It must do so, for some commands, because the semantics of some environment variables and configuration variables, as defined by Git, which gitoxide wishes to be compatible with, stipulates it in some cases.

(It is also often important not to run commands in shells where applicable semantics make that the wrong thing to do. But that is independent of this issue.)

Whatever shell is provided as POSIX sh, and named as such, should generally be used to run shell commands in situations where no #! is applicable because we are running a command rather than a script. For example, core.sshCommand is a shell command. It must, at least in the general case, be run in some shell. (See also #1752 and #1758.)

Relevance

This is not a blocker for #1758. These are ideas that don't belong in the scope of #1758--except for the parts that, as noted above, are already covered there--and that I may not get to immediately. I am opening this so it is not forgotten.

The relationship to running hooks is indirect: hooks with a #!/bin/sh or similar hashbang, on systems like Windows where hashbangs cannot be used directly, could use the sh.exe shell, which #1758 helps find correctly and which some of the items here would help find in even more situations. However, if considering what is discovered here for running hooks that don't specify a hashbang, I recommend making sure to confine it only to cases where that is needed for compatibility with existing Git behavior. (In general, whether scripts with no #! are run with a shell like /bin/sh or simply fail to execute varies, and is implemented, per application. exec-family functions would generally fail to run them.)

This does not directly help, and probably will not help at all, with the problem of getting an environment that picks up things like ssh-agent in setups where that is wanted but does not automatically work.

[Byron]

Thanks a lot for surfacing this problem. Thus far I just tried to solve it quickly, without having a full understanding of it which seems to be required for correctness.

Whatever shell is provided as POSIX sh, and named as such, should generally be used to run shell commands in situations where no #! is applicable because we are running a command rather than a script. For example, core.sshCommand is a shell command. It must, at least in the general case, be run in some shell. (See also #1752 and #1758.)

Actually, Git for Windows applies an optimization that runs the programs directly after performing argument splitting manually, when possible.

I also wonder if this shell selection should actually use the shell() function that was recently added to gix_path::env:: to assure when sh is used, it's the right one. Maybe this fits into the research as well.
My intuition here is to do it as the using the right shell (as opposed to some shell picked up from PATH) is important.

[EliahKagan]

Actually, Git for Windows applies an optimization that runs the programs directly after performing argument splitting manually, when possible.

Thanks--I was unaware that Git for Windows did that. I did know about gitoxide has such functionality, though, which is what motivated me to say "at least in the general case."

[Byron]

I think I copied it from the Git for Windows implementation, but also wanted to say that as part of this research this may be scrutinised. Maybe it's not always the same - probably that pretty much depends on environment variables. My guess is that GUI applications would prefer to use a shell always so there is a chance additional environment is picked up, and as a matter of fact GitButler is already doing it. It does so not just to increase chances of working correctly, not because it could reproduce an issue that occurred when not using a shell. So all of this is still quite vague at least in terms of practical experience.

[rvolgers]

It is probably a good idea to do what existing software on Windows does, since it increases the chances of being compatible with existing configurations. I would also err on the side of using fewer intermediate binaries.

Windows argument passing is an unexpectedly deep rabbit hole. As an example, executing binaries which happen to be compiled to use the ANSI instead of Unicode APIs can cause wildly inconsistent command line parsing, leading to problems: https://devco.re/blog/2025/01/09/worstfit-unveiling-hidden-transformers-in-windows-ansi/

TLDR: Windows will map unicode characters to characters which vaguely resemble them, including ones with special meaning like backslash and double quote. This combined with Windows passing the command line to each program as a single string (each program is responsible for argument splitting) causes problems.