Support for *args, positional-only parameters given positionally and function/method groups instantiated as callables

CHANGELOG.rst

@@ -31,6 +31,15 @@ Added
 - New ``docstrings`` and ``typeshed`` extras, which install individually the
   respective optional dependencies of the ``signatures`` extra (`#988
   <https://github.com/mauvilsa/jsonargparse/pull/988>`__).
+- Support for ``*args`` in signatures, added as a list argument named like the
+  parameter, or a positional with ``as_positional=True``. The AST resolver now
+  also follows a ``*args`` forwarded without ``**kwargs`` or to a callable that
+  has its own ``*args`` (`#986
+  <https://github.com/mauvilsa/jsonargparse/pull/986>`__).
+- New ``instantiate`` parameter of ``add_function_arguments`` and
+  ``add_method_arguments``, and links applied on instantiate can target the
+  parameters of these groups (`#986
+  <https://github.com/mauvilsa/jsonargparse/pull/986>`__).
 
 Fixed
 ^^^^^
@@ -82,6 +91,17 @@ Fixed
 - ``--print_config`` failing when a required subcommand is not given, unlike
   other required arguments (`#984
   <https://github.com/mauvilsa/jsonargparse/pull/984>`__).
+- ``instantiate``, ``auto_cli`` and ``from_config`` failing when a signature
+  has positional-only parameters (`#986
+  <https://github.com/mauvilsa/jsonargparse/pull/986>`__).
+- AST resolver failing for a method whose ``self`` is positional-only (`#986
+  <https://github.com/mauvilsa/jsonargparse/pull/986>`__).
+- A typed positional with ``nargs="*"`` set in a config file being reset to
+  empty when no values for it are given in the command line (`#986
+  <https://github.com/mauvilsa/jsonargparse/pull/986>`__).
+- A positional-only parameter that has a default being required, even though
+  the call can omit it (`#986
+  <https://github.com/mauvilsa/jsonargparse/pull/986>`__).
 
 Changed
 ^^^^^^^
@@ -141,6 +161,15 @@ Changed
 - A ``types.ModuleType`` value given as a module object is now normalized to its
   import path, instead of only being normalized when given as a default (`#983
   <https://github.com/mauvilsa/jsonargparse/pull/983>`__).
+- ``instantiate`` now replaces a group added by ``add_function_arguments`` or
+  ``add_method_arguments`` by a ``functools.partial`` with the arguments bound,
+  or by an ``operator.methodcaller`` for a method that is called with an
+  instance, instead of keeping the parsed namespace (`#986
+  <https://github.com/mauvilsa/jsonargparse/pull/986>`__).
+- ``add_subcommands`` now raises an error when the parser already has a
+  positional that accepts a variable number of values, since argparse parses
+  this combination incorrectly, e.g. taking the subcommand name as one of the
+  values (`#986 <https://github.com/mauvilsa/jsonargparse/pull/986>`__).
 
 Removed
 ^^^^^^^
@@ -154,6 +183,19 @@ Removed
   <https://github.com/mauvilsa/jsonargparse/pull/969>`__).
 
 
+v4.53.0 (unreleased)
+--------------------
+
+Deprecated
+^^^^^^^^^^
+- Groups added by ``add_function_arguments`` or ``add_method_arguments`` with a
+  ``nested_key`` are kept as parsed by ``instantiate``. From v5.0.0 they will be
+  replaced by a ``functools.partial`` with the arguments bound, or by an
+  ``operator.methodcaller`` for a method that is called with an instance. Warns
+  only with ``JSONARGPARSE_DEPRECATION_WARNINGS=all`` (`#985
+  <https://github.com/mauvilsa/jsonargparse/pull/985>`__).
+
+
 v4.52.0 (2026-09-01)
 --------------------
 

DOCUMENTATION.rst

@@ -1561,8 +1561,8 @@ and the method run, as follows:
     parser.add_method_arguments(MyClass, "mymethod", "myclass.method")
 
     cfg = parser.parse_args()
-    myclass = MyClass(**cfg.myclass.init.as_dict())
-    myclass.mymethod(**cfg.myclass.method.as_dict())
+    init = parser.instantiate(cfg)
+    init.myclass.method(init.myclass.init)
 
 
 The :meth:`add_class_arguments <.ArgumentParser.add_class_arguments>` call adds
@@ -1574,11 +1574,13 @@ the :meth:`add_method_arguments <.ArgumentParser.add_method_arguments>` call
 adds ``myclass.method.bar`` as a required float and ``myclass.method.baz`` as an
 optional boolean with default false.
 
-Several classes added with :meth:`add_class_arguments
-<.ArgumentParser.add_class_arguments>` are instantiated at once with
-:meth:`instantiate <.ArgumentParser.instantiate>`. In the example above, ``cfg =
-parser.instantiate(cfg)`` makes ``cfg.myclass.init`` an instance of ``MyClass``,
-built from the parsed arguments.
+All the groups added by these methods are instantiated at once with
+:meth:`instantiate <.ArgumentParser.instantiate>`. In the example above,
+``init.myclass.init`` is an instance of ``MyClass`` built from the parsed
+arguments, and ``init.myclass.method`` is an :func:`operator.methodcaller` with
+the arguments of ``mymethod`` bound, which is called with an instance. A function,
+static or class method group becomes a :func:`functools.partial` instead. Give
+``instantiate=False`` to keep a group as parsed.
 
 All values can be given in a single config file (see
 :ref:`configuration-files`). For convenience, the values of each argument group
@@ -1605,13 +1607,22 @@ A wide range of type hints is supported for signature parameters, see
 
 - ``fail_untyped`` decides which parameters without a type annotation raise an
   exception instead: the required ones with the default ``True``, all of them
-  with ``"all"``, and none with ``False``. Positional-only parameters are always
-  required. Use ``"all"`` only for code you own, since one untyped parameter of
-  a dependency would make its signature impossible to add.
+  with ``"all"``, and none with ``False``. Use ``"all"`` only for code you own,
+  since one untyped parameter of a dependency would make its signature
+  impossible to add.
 
 - Parameters whose name starts with ``_`` are considered internal and skipped,
   unless they are required.
 
+- A ``*args`` is added as a list argument with its name, e.g. ``*files: str`` as
+  ``files`` of type ``list[str]``. With ``as_positional=True`` it is a
+  positional that takes zero or more values, which argparse is unable to combine
+  with subcommands, so ``as_positional=False`` is required then. When calling,
+  positional-only parameters, and when ``*args`` has values also the ones before
+  it, are given positionally. Binding such values after positionals given on
+  call, e.g. for a ``Callable`` that returns a class, requires Python 3.14 or
+  later.
+
 - The ``skip`` parameter excludes arguments, e.g.
   ``parser.add_method_arguments(MyClass, 'mymethod', skip={'baz'})``. In a
   subclass spec, a skipped parameter can still be given in ``dict_kwargs``, see
@@ -2017,6 +2028,11 @@ needed because the parser does not know which call will happen at runtime, and
 including them would make :meth:`instantiate <.ArgumentParser.instantiate>` fail
 with unexpected keyword arguments.
 
+A ``*args`` forwarded to calls is replaced by what the calls take positionally,
+including their own ``*args``, when all the calls agree on it. A ``*args`` that
+the code uses itself, or whose use can't be resolved, is added as a list
+argument, and one that is not used is left out.
+
 .. note::
 
     The resolvers log failures and unsupported cases. To see these logs, set the

jsonargparse/_cli.py

@@ -6,6 +6,7 @@
 
 from ._actions import ActionConfigFile, _ActionPrintConfig, remove_actions
 from ._core import ArgumentParser
+from ._instantiation import bind_call, get_call_arguments
 from ._namespace import Namespace, dict_to_namespace
 from ._optionals import get_doc_short_description
 from ._signatures import FailUntyped
@@ -90,7 +91,7 @@ def auto_cli(
             parser.set_defaults(set_defaults)
         cfg = parser.parse_args(args)
         init = parser.instantiate(cfg)
-        return _run_component(components, init)
+        return _run_component(components, init, parser)
 
     elif isinstance(components, list):
         components = {c.__name__: c for c in components}
@@ -110,7 +111,9 @@ def auto_cli(
         else:
             break
     component = components_ns[subcommand]
-    return _run_component(component, init.get(subcommand))
+    for name in subcommand.split("."):
+        parser = parser._subcommands_action._name_parser_map[name]  # type: ignore[union-attr]
+    return _run_component(component, init.get(subcommand), parser)
 
 
 def auto_parser(*args, **kwargs) -> ArgumentParser:
@@ -200,17 +203,23 @@ def _add_component_to_parser(
     return added_args
 
 
-def _run_component(component, cfg):
+def _get_call_values(cfg: Namespace) -> dict:
+    return dict(cfg.items(branches=True, nested=False))
+
+
+def _run_component(component, cfg, parser):
     cfg.pop("config", None)
     subcommand = cfg.pop("subcommand")
     if inspect.isclass(component) and subcommand:
         subcommand_cfg = cfg.pop(subcommand, {})
         subcommand_cfg.pop("config", None)
-        component_obj = component(**cfg)
+        component_obj = bind_call(component, parser._call_layouts[None], _get_call_values(cfg))()
         if isinstance(getattr(component, subcommand), property):
             return getattr(component_obj, subcommand)
         component = getattr(component_obj, subcommand)
+        parser = parser._subcommands_action._name_parser_map[subcommand]
         cfg = subcommand_cfg
+    args, kwargs = get_call_arguments(parser._call_layouts[None], _get_call_values(cfg), component)
     if inspect.iscoroutinefunction(component):
-        return __import__("asyncio").run(component(**cfg))
-    return component(**cfg)
+        return __import__("asyncio").run(component(*args, **kwargs))
+    return component(*args, **kwargs)

jsonargparse/_core.py

@@ -139,6 +139,7 @@ def __init__(self, *args, **kwargs) -> None:
         """Initializer for ActionsContainer instance."""
         super().__init__(*args, **kwargs)
         self._accepted_kwargs = {}
+        self._call_layouts = {}
         self.register("type", None, identity)
         self.register("action", "parsers", ActionSubCommands)
         self.register("action", "config", ActionConfigFile)
@@ -807,6 +808,13 @@ def add_subcommands(self, required: bool = True, dest: str = "subcommand", **kwa
             **kwargs: All options that `argparse.ArgumentParser.add_subparsers
                 <https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser.add_subparsers>`_ accepts.
         """
+        for action in self._actions:
+            if not action.option_strings and action.nargs in {"*", "+", argparse.REMAINDER}:
+                raise ValueError(
+                    f"Positional '{action.dest}' accepts a variable number of values, which argparse is unable to "
+                    "combine with subcommands. It must be an optional argument, in auto_cli by setting "
+                    "as_positional=False."
+                )
         if "description" not in kwargs:
             kwargs["description"] = "For more details of each subcommand, add it as an argument followed by --help."
         default_config_files = self.default_config_files

jsonargparse/_from_config.py

@@ -6,6 +6,7 @@
 
 from ._common import parser_context
 from ._core import ArgumentParser
+from ._instantiation import CallLayout, bind_call
 from ._loaders_dumpers import get_loader_exceptions, load_value
 from ._optionals import _get_config_read_mode
 from ._paths import path_dir_context
@@ -57,12 +58,14 @@ def from_config(cls: type[T], config: str | PathLike | dict) -> T:
         Args:
             config: Path to a config file or a dict with config values.
         """
-        kwargs, cls = _parse_class_kwargs_from_config(cls, config, **cls.__from_config_parser_kwargs__)  # type: ignore[attr-defined]
-        return cls(**kwargs)
+        kwargs, cls, call_layout = _parse_class_kwargs_from_config(cls, config, **cls.__from_config_parser_kwargs__)  # type: ignore[attr-defined]
+        return bind_call(cls, call_layout, kwargs)()
 
 
-def _parse_class_kwargs_from_config(cls: type[T], config: str | PathLike | dict, **kwargs) -> tuple[dict, type[T]]:
-    """Parse the init kwargs for ``cls`` from a config file or dict."""
+def _parse_class_kwargs_from_config(
+    cls: type[T], config: str | PathLike | dict, **kwargs
+) -> tuple[dict, type[T], CallLayout]:
+    """Parse the init kwargs for ``cls`` from a config file or dict, and how they are given in the call."""
     parser = ArgumentParser(exit_on_error=False, **kwargs)
     cfg_path = None
     if not isinstance(config, dict):
@@ -96,7 +99,7 @@ def _parse_class_kwargs_from_config(cls: type[T], config: str | PathLike | dict,
         clear_required(parser, required)
     with load_config_path_context(cfg_path), path_dir_context(cfg_path):
         cfg = parser.parse_object(config, defaults=False)
-    return parser.instantiate(cfg).as_dict(), cls
+    return parser.instantiate(cfg).as_dict(), cls, parser._call_layouts[None]
 
 
 def _override_init_defaults(cls: type[T], parser_kwargs: dict) -> None:
@@ -107,7 +110,7 @@ def _override_init_defaults(cls: type[T], parser_kwargs: dict) -> None:
     if not (isinstance(config, (str, PathLike)) and Path(config).is_file()):
         return
 
-    defaults, cls = _parse_class_kwargs_from_config(cls, config, **parser_kwargs)
+    defaults, cls, _ = _parse_class_kwargs_from_config(cls, config, **parser_kwargs)
     _override_init_defaults_this_class(cls, defaults)
     _override_init_defaults_parent_classes(cls, defaults)
 

jsonargparse/_instantiation.py

@@ -1,11 +1,86 @@
+import dataclasses
+import functools
 import inspect
-from typing import Protocol
+import sys
+from collections.abc import Callable, Mapping
+from functools import partial
+from typing import Any, Protocol
 
 from ._common import ClassType, applied_instantiation_links, get_parsing_setting, is_subclass, parser_context
 from ._namespace import Namespace, get_value_and_parent, split_key
 
 __all__ = ["add_instantiator"]
 
+kinds = inspect._ParameterKind
+
+
+@dataclasses.dataclass(frozen=True)
+class CallLayout:
+    """How the values of the parameters of a signature are given in a call."""
+
+    # parameters that can be given positionally, in order, i.e. positional-only ones and the ones
+    # before a *args
+    positional: tuple[str, ...] = ()
+    # number of leading positional ones that are always given positionally, i.e. up to the last
+    # positional-only one. The rest are only given positionally when *args is not empty.
+    always_positional: int = 0
+    var_positional: str | None = None
+    # defaults of the positional ones, used when one was not given, e.g. because it was skipped
+    defaults: Mapping[str, Any] = dataclasses.field(default_factory=dict)
+    # number of leading positionals that are not bound, but given when calling, e.g. by the
+    # caller of a callable that returns a class
+    open_positionals: int = 0
+
+
+def get_call_layout(params: list, open_positionals: int = 0) -> CallLayout:
+    """Gets the call layout for a list of resolved parameters."""
+    var_idx = next((n for n, p in enumerate(params) if p.kind == kinds.VAR_POSITIONAL), None)
+    positional_only_idxs = [n for n, p in enumerate(params) if p.kind == kinds.POSITIONAL_ONLY]
+    end = var_idx if var_idx is not None else (positional_only_idxs[-1] + 1 if positional_only_idxs else 0)
+    positional = [p for p in params[:end] if p.kind in {kinds.POSITIONAL_ONLY, kinds.POSITIONAL_OR_KEYWORD}]
+    always = [n for n, p in enumerate(positional) if p.kind == kinds.POSITIONAL_ONLY]
+    return CallLayout(
+        positional=tuple(p.name for p in positional),
+        always_positional=always[-1] + 1 if always else 0,
+        var_positional=None if var_idx is None else params[var_idx].name,
+        defaults={p.name: p.default for p in positional if p.default is not inspect._empty},
+        open_positionals=open_positionals,
+    )
+
+
+def get_call_arguments(layout: CallLayout, values: Mapping[str, Any], component: Any) -> tuple[list, dict]:
+    """Splits values into the positional and keyword arguments with which to call a component."""
+    kwargs = dict(values)
+    var_positional = list(kwargs.pop(layout.var_positional, ())) if layout.var_positional else []
+    num_positional = len(layout.positional) if var_positional else layout.always_positional
+    args = []
+    for num, name in enumerate(layout.positional[:num_positional]):
+        if name in kwargs:
+            args.append(kwargs.pop(name))
+        elif name in layout.defaults:
+            args.append(layout.defaults[name])
+        else:
+            following = layout.var_positional if var_positional else layout.positional[num_positional - 1]
+            raise ValueError(
+                f'Calling {component} requires a value for parameter "{name}", since it precedes "{following}" '
+                "which is given positionally."
+            )
+    return args + var_positional, kwargs
+
+
+def bind_call(func: Callable, layout: CallLayout, values: Mapping[str, Any], component: Any = None) -> Callable:
+    """Binds values to func according to a call layout, returning a callable that makes the call."""
+    args, kwargs = get_call_arguments(layout, values, component or func)
+    if layout.open_positionals and args:
+        # the open positionals are given on call, before the bound ones
+        if sys.version_info < (3, 14):
+            raise ValueError(
+                f"Binding values to positionals of {component or func} that follow {layout.open_positionals} "
+                "positionals given on call is only supported in Python 3.14 or later."
+            )
+        args = [functools.Placeholder] * layout.open_positionals + args
+    return partial(func, *args, **kwargs)
+
 
 class InstantiatorCallable(Protocol):
     def __call__(self, class_type: type[ClassType], *args, **kwargs) -> ClassType:
@@ -32,8 +107,8 @@
         - **Class/subclass type arguments** (``add_argument`` with a class type
           or ``add_class_arguments``/``add_subclass_arguments``): An object with
           ``class_path`` and optionally ``init_args`` is replaced by an instance
-          of the referenced class, created by calling
-          ``class_type(**init_args)``. For the case of classes with disabled
+          of the referenced class, created by calling the class with the
+          ``init_args``. For the case of classes with disabled
           subclasses, the namespace can have directly the init args without the
           ``class_path`` + ``init_args`` wrapper.
 
@@ -45,14 +120,20 @@
           call arguments are provided yet — a :func:`functools.partial` bound to
           the given ``init_args``.
 
+        - **Function and method groups** (``add_function_arguments`` or
+          ``add_method_arguments`` with a ``nested_key``): Replaced by a
+          :func:`functools.partial` with the arguments bound, or by an
+          :func:`operator.methodcaller` for a method that is called with an
+          instance.
+
         - **Instantiation order**: Components are processed in the order
           determined by argument links applied on instantiation.
 
         Args:
             namespace: The configuration object to use. Must have been produced
                 by one of the ``parse_*`` methods and not modified in a way that
                 breaks the structure expected by the parser.
-            instantiate_groups: Whether class groups should be instantiated.
+            instantiate_groups: Whether class, function and method groups should be instantiated.
 
         Returns:
             A new configuration object where every registered signature