How to apply this at scale?

[JP-Dhabolt]

First, I wanted to start by thanking you for the work you've put into this project! I've played around and got this working in a POC for a single repository. I have questions around how to apply this at scale. I'll use the following scenario to put context around my questions.

Github Org A has 5 repositories that each have a configuration file that needs to be updated in the same way (simple transformation).

From my understanding of AutoTransform, this would require 5 different schema files (one for each repo, as the repo is part of the schema definition). Additionally, AutoTransform would need to be part of each repository, as it appears to apply transformations only on the current repo. The schema repo seems to only come into play when creating a PR, and errors if not referencing the same repo.

Is this understanding correct?

My hope is that something similar to the below diagram would be possible. There would then be a way to scale out a transformation (defined one time) across N number of repos.

sequenceDiagram
    participant AT as AutoTransformRepo
    participant T as TargetRepo
    AT->>+T: Clones base_branch
    AT->>+AT: Applies Transformation to cloned repo
    AT->>+T: Open PR 

Loading

If I'm trying to do something that is out of scope for this tool, please let me know.
Thanks!

JP

[nathro]

This is definitely not out of the realm of things AutoTransform should be able to solve, it's just not been my main focus. There's a world where you could separate the repo from the schema file and have them passed separately to the run script.

I'd want to think about changes to the structure here that we could use to enable this type of behavior. I suspect it would involve an optional argument to runner scripts that would allow you to override the repo that a schema specifies (at least, that would be the simplest way to implement this). We could also go down the road of fully removing the repo from the schema files and have that supplied separately, maybe via a similar approach to the schema map where we map repo names to objects that you could use.

The biggest question is how do we do this in a way that doesn't make the single repo use case difficult to work with.

I come from a mono repo world and this definitely shows in how I built AutoTransform, heh.

[nathro]

As I think about it, what might actually be the cleanest way to do this is via the config. I imagine the changes we would need to make are:

• Remove repos from schema files
• Add a repo option to the config
• Enable the run script to clone the repo
• Add a way to separate where your files are located so you can have schema files separate from the repos themselves

#3 and #4 are where I think we'll run in to the most issues.

[nathro]

And for 3, I think we'll need to actually do a breaking change here, there's no way around it. #83 should be a fix for that (while also giving us a nice SchemaMap util class).

[JP-Dhabolt]

Thanks again for the fast turnaround on this. I'll kick the tires for this on Monday. Have a good Thanksgiving!

[nathro]

I released 1.1.0a1 for you to kick the tires on :)

[JP-Dhabolt]

Thanks! This change works as expected for my use case, using the AUTO_TRANSFORM_REPO_OVERRIDE environment variable.

[nathro]

Glad to hear it, if you encounter any other issues or need some help with something let me know.