git migration package

[yaswant]

To keep a record of things we plan to do for git migration, I've added a checklist list and how we intend to migrate the repositories from MOSRS to GitHub.

[yaswant]

Issues:

jules revision r1710 mapped to um10.2; but looks like the commit is not in the git log!

{ "um10.2": 1710 }

Fix:

thanks @jennyhickson for the detective work

um10.2 tag in JULES looks to have been a typo and should be 1709. 1710 was a commit to a branch which is why it doesn't exist in git, but JULES vn4.3.1 was tagged as 1709 at the same time so I suspect um10.2 should have been the same

{ "um10.2": 1709 }

[t00sa]

Looks good. I've added a couple of suggestions.

[t00sa]

Results of testing

Interactive test of the script without a test of the update functionality. The elapsed time was 02:49:13.

vdi> ./ssd_svn2git.sh -c $PWD/config.json -w /<data>/sam.clarke/git-migration
/<data>/git-migration /<tmp>/SimSys_Scripts/git-migration
Already up to date.
--------------------------------------------------------------------
gitlify is /<data>/git-migration/gitlify/gitlify
jq is /usr/bin/jq
gh is /usr/bin/gh
--------------------------------------------------------------------
WORKDIR: /<data>/git-migration
CONFIG_FILE: /<tmp>/SimSys_Scripts/git-migration/config.json
UPDATE_GITHUB: 0
--------------------------------------------------------------------
2025-03-31 14:09:12 Processing: casim
2025-03-31 14:10:34 Processing: moci
2025-03-31 14:13:52 Processing: jules
2025-03-31 14:28:39 Processing: socrates
2025-03-31 14:31:02 Processing: ukca
2025-03-31 14:32:05 Processing: shumlib
2025-03-31 14:33:42 Processing: mule
2025-03-31 14:38:32 Processing: um_aux
2025-03-31 14:44:32 Processing: um_doc
2025-03-31 14:54:13 Processing: um_meta
2025-03-31 14:58:39 Processing: um
2025-03-31 16:19:32 Processing: gcom
2025-03-31 16:21:35 Processing: lfric_apps
2025-03-31 16:26:59 Processing: lfric_core
2025-03-31 16:58:25 Done.
vdi> 

[yaswant]

Results of testing

Interactive test of the script without a test of the update functionality. The elapsed time was 02:49:13.

vdi> ./ssd_svn2git.sh -c $PWD/config.json -w /<data>/sam.clarke/git-migration
/<data>/git-migration /<tmp>/SimSys_Scripts/git-migration
Already up to date.
--------------------------------------------------------------------
gitlify is /<data>/git-migration/gitlify/gitlify
jq is /usr/bin/jq
gh is /usr/bin/gh
--------------------------------------------------------------------
WORKDIR: /<data>/git-migration
CONFIG_FILE: /<tmp>/SimSys_Scripts/git-migration/config.json
UPDATE_GITHUB: 0
--------------------------------------------------------------------
2025-03-31 14:09:12 Processing: casim
2025-03-31 14:10:34 Processing: moci
2025-03-31 14:13:52 Processing: jules
2025-03-31 14:28:39 Processing: socrates
2025-03-31 14:31:02 Processing: ukca
2025-03-31 14:32:05 Processing: shumlib
2025-03-31 14:33:42 Processing: mule
2025-03-31 14:38:32 Processing: um_aux
2025-03-31 14:44:32 Processing: um_doc
2025-03-31 14:54:13 Processing: um_meta
2025-03-31 14:58:39 Processing: um
2025-03-31 16:19:32 Processing: gcom
2025-03-31 16:21:35 Processing: lfric_apps
2025-03-31 16:26:59 Processing: lfric_core
2025-03-31 16:58:25 Done.
vdi> 

Thank you @t00sa. The parallel version committed in ecb6101 reduced the runtime by half on Azure SPICE VDI.

2025-04-01 02:18:20 Processing: casim
2025-04-01 02:18:20 Processing: moci
2025-04-01 02:18:20 Processing: jules
2025-04-01 02:18:20 Processing: socrates
2025-04-01 02:18:20 Processing: ukca
2025-04-01 02:18:20 Processing: shumlib
2025-04-01 02:18:20 Processing: mule
2025-04-01 02:18:20 Processing: um_aux
2025-04-01 02:18:20 Processing: um_doc
2025-04-01 02:18:20 Processing: um_meta
2025-04-01 02:18:20 Processing: um
2025-04-01 02:18:20 Processing: gcom
2025-04-01 02:18:20 Processing: lfric_apps
2025-04-01 02:18:20 Processing: lfric_core
2025-04-01 03:47:58 Done.

[yaswant]

I've done a complete refactor of the script with following changes:

• remove the parallel version as it mostly manifests code duplication
• modularise the script into small functions. I hope the z in initialize is accepted in British English?

and additional options:

• -m, --from-mirror use internal SVN mirrors instead of MOSRS - thanks @ericaneininger! This is required for batch jobs where user cant interact with SRS.
• -n, --list-repos list repositories to be migrated in dry run mode.
• -j, --parallel process repositories in parallel batches. By default, number of CPU cores is used as batch size (=number or repos to process in parallel).
• -r, --repository specify a single repository to process. This is useful for testing.

[t00sa]

Very elegant refactor. Happy to approve.

[Pierre-siddall]

Having thoroughly explained each function in the shell script with Copilot my only suggestion is adding comments if the script will be reused and modified in the future. Otherwise, it appears ready for merging as an initial (and elegant) implementation of the SVN to Git migration. Great job Yash :) .

[james-bruten-mo]

Just a minor suggestion on the wording for shumlib