Ensure that parent repos get scanned if cloning submodules fails

[sschuberth]

Currently, it seems that if provenance resolution partly fails due to a submodule of a Git repo not existing anymore, the whole scan for the repo is skipped:

Could not resolve nested provenance for package 'NPM::json3:3.2.6': IOException: Running 'git -c url.https://.insteadOf=git:// submodule update --init --recursive' in '/tmp/ort-DefaultWorkingTreeCache5398368741174589668' failed with exit code 1:
Cloning into '/tmp/ort-DefaultWorkingTreeCache5398368741174589668/vendor/spec'...
fatal: could not read Username for 'https://github.com': No such device or address
fatal: clone of 'git://github.com/kitcambridge/spec.git' into submodule path '/tmp/ort-DefaultWorkingTreeCache5398368741174589668/vendor/spec' failed
Failed to clone 'vendor/spec'. Retry scheduled
Cloning into '/tmp/ort-DefaultWorkingTreeCache5398368741174589668/vendor/spec'...
fatal: could not read Username for 'https://github.com': No such device or address
fatal: clone of 'git://github.com/kitcambridge/spec.git' into submodule path '/tmp/ort-DefaultWorkingTreeCache5398368741174589668/vendor/spec' failed
Failed to clone 'vendor/spec' a second time, aborting

Although https://github.com/bestiejs/json3/blob/v3.2.6/LICENSE of the parent repo contains the MIT license, no such license is present in the scan results for 'NPM::json3:3.2.6'.

We should ensure that the parent repo and generally as much as possible, including any existing submodules do get scanned. Because in this case, spec is just a (vendored) test dependency, for which an issue resolution could be created, if the rest was scanned properly.

[mnonnenmacher]

This requires a change in ORT, so I think this issue should be moved there. Currently, the NestedProvenanceResolver throws an exception if recursive cloning fails:

https://github.com/oss-review-toolkit/ort/blob/4672d8eea433b1aeff9a5054db8a50d873f26c23/scanner/src/main/kotlin/provenance/NestedProvenanceResolver.kt#L78-L82

And the Scanner just records an issue in this case:

https://github.com/oss-review-toolkit/ort/blob/4672d8eea433b1aeff9a5054db8a50d873f26c23/scanner/src/main/kotlin/Scanner.kt#L256-L281

We should ensure that the parent repo and generally as much as possible, including any existing submodules do get scanned. Because in this case, spec is just a (vendored) test dependency, for which an issue resolution could be created, if the rest was scanned properly.

To achieve this, the NestedProvenanceResolver would have to be able to determine which submodule exactly caused the issue and to continue for the other submodules. I assume that this would require a larger refactoring of the updateWorkingTree function and the Git specific implementation of it.