`argparse.ArgumentParser` to Styx IR Frontend

[nx10]

Summary

Create a frontend that converts Python argparse.ArgumentParser objects directly into Styx IR, enabling downstream descriptor generation.

Prior Art

• Nipoppy get_descriptor
• Boutiques creator.py

Proposed Architecture

A two-stage approach to maximize compatibility:

1. Standalone serialization package: A minimal, self-contained Python package that serializes argparse.ArgumentParser objects to a JSON representation. This package should support a wide range of Python versions to allow use in legacy environments.

2. Styx frontend: Consumes the JSON output and converts it to Styx IR.

This decoupled design isolates version-sensitive introspection logic from the main Styx toolchain.

[nx10]

I've started a WIP implementation, will link here soon.

[michellewang]

Here are some notes (pain points) from my experience using boutiques.creator.CreateDescriptor, maybe they could be helpful in informing your implementation:

1. Input ID: CreateDescriptor uses the dest attribute as input ID, and sometimes there is a mismatch between the CLI option name and the descriptor's "id" field
   1. This is not ideal:
      1. Can be annoying for users, e.g. if the CLI flag is "--force" but the parser uses "overwrite" internally (example)
      2. Worse, the Boutiques ID can be misleading or outright wrong if processing is applied through the type argument (see example where there is a unit conversion), or if a flag's action is "store_false" (example)
      3. Some pipeline have multiple arguments that share the same dest (example)
   2. I think it makes more sense to use the option strings (ideally with some heuristics to favour double-dashed strings, e.g. "--output" instead of "-o")
2. Default values
   1. Some pipelines dynamically set default values, e.g. from resolving paths (example) or checking the number of cores available on a system (example)
      1. In these cases the default value should not be included in the JSON representation
   2. Some default values (e.g. for the --version flag) are set to argparse.SUPPRESS which resolves to "==SUPPRESS=="
      1. They should also not be included in the JSON representation
   3. Flags have default value True if action="store_false"
      1. Any entry with "type": "Flag" and "default": True is suspicious and probably wrong
3. Lists: CreateDescriptor does not consider an input to be a list if narg is "*", which is probably a bug
4. Flags: args with action "store_false" are not recognized as flags and so have the wrong type in the generated descriptor (in addition to having the wrong default value, see above)
5. Edge cases (here for completeness, I don't think that there are easy ways of handling these)
   1. Custom functions/classes in type field (e.g. PositiveInt) --> CreateDescriptor defaults to string which is somewhat annoying
   2. Nipreps and other similar BIDS Apps have this verbose_count thing (-v/-vv/-vvv --> int) which is not handled correctly: in the descriptor the default is 0 which wrong. My workaround was to make this an optional argument with choices "-v", "-vv", "-vvv", something like this:

      {
          "id": "verbose_count",
          "name": "verbose_count",
          "description": "Increases log verbosity for each occurrence, debug level is -vvv",
          "optional": true,
          "type": "String",
          "value-key": "[VERBOSE_COUNT]",
          "value-choices": [
              "-v",
              "-vv",
              "-vvv"
          ]
      }

I hope this helps, and I'd be happy to test the package/frontend with some of the pipelines that we have already described for Nipoppy!

[nx10]

Amazing! this will help a ton when building and testing the conversion, especially all the concrete examples!

The argparse -> JSON package is done: https://github.com/styx-api/argdump

Now jumping over to build the frontend / boutiques conversion. I'll prioritize getting the main cases solid first, then iterate on the trickier edge cases. Once I have a prototype I will ping back for some real world testing.

Thanks!