Feature: Submit apps to App Store review (codemagic-ci-cd#164)

* Extract app store version submission actions into separate action group

* Add action to create new app store version

* Support App Store version parameters from submit to review action

* Include app store review submission as publishing subaction

* Add more options to ASC publishing

* Fix tests

* Add test for App Store submit from publish action

* Reuse arguments from subactions to publish action

* Fix TestFlight and App Store options

* Use custom argument groups in docs

* Adjust custom argument groups descriptions in docs

* Bump version

* Accept build filters in 'app-store-connect apps builds'

* Define missing attribute

* Accept strings for build version number argument

* Log processing message only in case build is still being processed

* Allow using multiple values for single filter

* Add action to get build's app store version

* Add API client method and CLI action to modify App Store version

* Use existing app store version for submission if possible

* Make platform optional for 'app-store-connect publish' action

* Update docs

* Format logs

* Rename CliActionParserBuilder to ArgumentParserBuilder

* Remove pretty_value method from ResourceEnum

* Add test and docstring for Argument.with_custom_argument_group

* Reformat publish_action

* Make AppStoreVersions.read_build return type optional

* Update docstring for 'create_app_store_version'

* Add changelog

* Update test

* Fix error message when using --exclude and --not-exclude together

* Rename CommonArgumentTypes.iso_8601_date to CommonArgumentTypes.iso_8601_datetime

* Update descriptions for --expired and --not-expired flags

* Fix updating existing app store version

* Fix type hint

* Fix tests

* Rename tests/cli/test_argument.py -> tests/cli/argument/test_argument.py

* Make ISO 8601 timestamp parsing more relaxed and add tests

* Fix setting earliest release date for App Store version

* Add more validation steps to earliest release date

* Fix merge conflict in changelog

Author: priitlatt

CHANGELOG.md

@@ -1,3 +1,70 @@
+Version 0.13.0
+-------------
+
+Additions and changes from [pull request #164](https://github.com/codemagic-ci-cd/cli-tools/pull/164).
+
+**Features**
+
+- Add new App Store Connect API client method `AppStoreVersions.create` to create an App Store version. See official API method [documentation](https://developer.apple.com/documentation/appstoreconnectapi/create_an_app_store_version).
+- Add new App Store Connect API client method `AppStoreVersions.read_build` to read associated build from an App Store version. See official API method [documentation](https://developer.apple.com/documentation/appstoreconnectapi/read_the_build_information_of_an_app_store_version).
+- Add new App Store Connect API client method `AppStoreVersions.read_app_store_version_submission` to read associated App Store version submission from an App Store version. See official API method [documentation](https://developer.apple.com/documentation/appstoreconnectapi/read_the_app_store_version_submission_information_of_an_app_store_version).
+- Add new App Store Connect API client method `AppStoreVersions.modify` to edit existing App Store version details. See official API method [documentation](https://developer.apple.com/documentation/appstoreconnectapi/modify_an_app_store_version).
+- Add new App Store Connect API client method `AppStoreVersions.delete` to delete an App Store version. See official API method [documentation](https://developer.apple.com/documentation/appstoreconnectapi/delete_an_app_store_version).
+- Add ability to group optional CLI action arguments into named argument groups.
+- Add new actions group `app-store-versions` to `app-store-connect`.
+- Add new action `app-store-connect app-store-versions create` to create a new App Store version using specified build to an app.
+- Add new action `app-store-connect app-store-versions modify` to update existing App Store version details.
+- Add new action `app-store-connect app-store-versions delete` to remove App Store version.
+- Add option to specify build filters for action `app-store-connect apps builds`.
+- Add new action `app-store-connect builds app-store-version` to get the App Store version of a specific build.
+- Add new action `app-store-connect builds submit-to-app-store` to submit specified build to App Store review. Optionally specify version details and release type.
+- Update `app-store-connect publish` action to allow automatic App Store review submission after binary upload.
+- Use grouped CLI arguments in action `app-store-connect publish` for better help messages and documentation. 
+- Add new option `--skip-package-upload` to `app-store-connect publish`.
+- Add short aliases for CLI flags:
+  - `-su` for `--skip-package-upload`,
+  - `-sv` for `--skip-package-validation`,
+  - `-w` for `--max-build-processing-wait`.
+- 
+
+**Fixes**
+
+- Fix creating `Resource` objects that do not have any attributes.
+- Fix `--max-build-processing-wait` CLI option validation.
+- Allow non-integer version numbers for `--build-version-number`.
+
+**Docs**
+
+- Update docs for tool `app-store-connect`.
+- Create docs for `app-store-connect` actions group `app-store-versions`.
+- Create docs for action `app-store-connect app-store-versions create`.
+- Create docs for action `app-store-connect app-store-versions delete`.
+- Create docs for action `app-store-connect app-store-versions modify`.
+- Create docs for `app-store-connect` actions group `apps`.
+- Update docs for action `app-store-connect apps app-store-versions`.
+- Update docs for action `app-store-connect apps builds`.
+- Update docs for action `app-store-connect apps list`.
+- Create docs for `app-store-connect` actions group `builds`.
+- Create docs for action `app-store-connect builds app-store-version`.
+- Create docs for action `app-store-connect builds submit-to-app-store`.
+- Update docs for action `app-store-connect builds submit-to-testflight`.
+- Update docs for action `app-store-connect list-builds`.
+- Update docs for action `app-store-connect publish`.
+
+**Development**
+
+- Support filtering by multiple values for one parameter in `ResourceManager.filter`.
+- Change `ResourceManager._get_update_payload` to accept `relationships` in addition to `attributes`. Make `attributes` and `relationships` arguments keyword-only.
+- Add missing return type hints to resource manger methods that did not have them.
+- Extract argument parser setup from `CliApp` class into separate module under `ArgumentParserBuilder` class.
+- Move `create_app_store_version_submission` and `delete_app_store_version_submission` methods from `AppStoreConnect` class to `AppStoreVersionSubmissionsActionGroup`.
+- Collect `app-store-connect` actions arguments that are used number of times under argument groups to reduce duplication.
+- Refactor `PublishAction` class:
+  - extract `publish` action arguments validation into separate method,
+  - move decorator arguments into separate tuple, 
+  - add dataclasses for subaction options, 
+  - support App Store review submission.
+
 Version 0.12.1
 -------------
 

doc.py

@@ -3,11 +3,15 @@
 from __future__ import annotations
 
 import argparse
+import operator
 import os
 import re
 import sys
+from collections import defaultdict
+from functools import reduce
 from pathlib import Path
 from typing import Callable
+from typing import Dict
 from typing import Iterable
 from typing import List
 from typing import NamedTuple
@@ -28,6 +32,7 @@ class SerializedArgument(NamedTuple):
     flags: str
     name: str
     required: bool
+    argument_group_name: Optional[str]
     default: str
     nargs: bool
     choices: str
@@ -40,6 +45,7 @@ class Action(NamedTuple):
     description: str
     required_args: List[SerializedArgument]
     optional_args: List[SerializedArgument]
+    custom_args: Dict[str, List[SerializedArgument]]
 
 
 class ActionGroup(NamedTuple):
@@ -61,6 +67,7 @@ def __init__(self, raw_arguments):
         self.raw_arguments = raw_arguments
         self.required_args: List[SerializedArgument] = []
         self.optional_args: List[SerializedArgument] = []
+        self.custom_args: Dict[str, List[SerializedArgument]] = defaultdict(list)
 
     @classmethod
     def _replace_quotes(cls, description: str) -> str:
@@ -91,16 +98,21 @@ def _serialize_argument(self, arg) -> SerializedArgument:
             flags=', '.join(getattr(arg._value_, 'flags', '')),
             name='' if arg_type and arg_type.__name__ == 'bool' else arg._name_,
             required=kwargs.required,
+            argument_group_name=arg._value_.argument_group_name,
             default=kwargs.default,
             nargs=kwargs.nargs,
             choices=kwargs.choices,
             store_boolean=kwargs.store_boolean,
         )
 
     def serialize(self) -> ArgumentsSerializer:
-        args = list(map(self._serialize_argument, self.raw_arguments))
-        self.required_args = [arg for arg in args if arg.required]
-        self.optional_args = [arg for arg in args if not arg.required]
+        for argument in map(self._serialize_argument, self.raw_arguments):
+            if argument.required:
+                self.required_args.append(argument)
+            elif argument.argument_group_name:
+                self.custom_args[argument.argument_group_name].append(argument)
+            else:
+                self.optional_args.append(argument)
         return self
 
     @classmethod
@@ -165,7 +177,7 @@ def generate(self):
             self._write_action_group_page(group)
 
     def _write_tool_command_arguments_and_options(self, writer):
-        writer.write_arguments(f'command `{self.tool_command}`', self.tool_optional_args, self.tool_required_args)
+        writer.write_arguments(f'command `{self.tool_command}`', self.tool_optional_args, self.tool_required_args, {})
         writer.write_options(self.tool_options)
 
     def _write_tool_page(self):
@@ -198,7 +210,12 @@ def _write_action_page(self, action: Action, action_group: Optional[ActionGroup]
         writer = Writer(md)
         writer.write_description(action.description)
         writer.write_command_usage(self, action_group=action_group, action=action)
-        writer.write_arguments(f'action `{action.action_name}`', action.optional_args, action.required_args)
+        writer.write_arguments(
+            f'action `{action.action_name}`',
+            action.optional_args,
+            action.required_args,
+            action.custom_args,
+        )
         self._write_tool_command_arguments_and_options(writer)
         md.create_md_file()
 
@@ -223,6 +240,7 @@ def _serialize_action(action: Callable) -> Action:
                 description=action.__doc__,
                 required_args=action_args_serializer.required_args,
                 optional_args=action_args_serializer.optional_args,
+                custom_args=action_args_serializer.custom_args,
             )
 
         return list(map(_serialize_action, tool.iter_class_cli_actions(action_group=action_group)))
@@ -236,6 +254,7 @@ def _serialize_option(option) -> SerializedArgument:
                 flags=', '.join(option.option_strings),
                 name='',
                 required=False,
+                argument_group_name=None,
                 default='',
                 nargs=False,
                 choices=' | '.join(option.choices) if option.choices else '',
@@ -246,7 +265,7 @@ def _serialize_option(option) -> SerializedArgument:
             description=tool.__doc__,
             formatter_class=cli.cli_help_formatter.CliHelpFormatter,
         )
-        tool.set_default_cli_options(parser)
+        cli.argument.ArgumentParserBuilder.set_default_cli_options(parser)
         return list(map(_serialize_option, parser._actions))
 
 
@@ -263,6 +282,7 @@ def get_command_usage(self,
         action_args = [
             *map(self._get_formatted_flag, action.optional_args),
             *map(self._get_formatted_flag, action.required_args),
+            *map(self._get_formatted_flag, reduce(operator.add, action.custom_args.values(), [])),
         ] if action else ['ACTION']
 
         return [
@@ -350,9 +370,17 @@ def _get_group_doc(group: ActionGroup) -> List[str]:
         self.file.new_header(level=3, title='Action groups', add_table_of_contents='n')
         self.write_table(list(map(_get_group_doc, groups)), ['Action group', 'Description'])
 
-    def write_arguments(self, obj: str, optional: List[SerializedArgument], required: List[SerializedArgument]):
+    def write_arguments(
+            self,
+            obj: str,
+            optional: List[SerializedArgument],
+            required: List[SerializedArgument],
+            custom: Dict[str, List[SerializedArgument]],
+    ):
         self._write_arguments(self.file, f'Required arguments for {obj}', required)
         self._write_arguments(self.file, f'Optional arguments for {obj}', optional)
+        for group_name, custom_arguments in custom.items():
+            self._write_arguments(self.file, f'Optional arguments to {group_name}', custom_arguments)
 
     def write_options(self, options: List[SerializedArgument]):
         self._write_arguments(self.file, 'Common options', options)

docs/app-store-connect/README.md

@@ -102,6 +102,7 @@ Enable verbose logging for commands
 |Action group|Description|
 | :--- | :--- |
 |[`app-store-version-submissions`](app-store-version-submissions.md)|Manage your application's App Store version review process|
+|[`app-store-versions`](app-store-versions.md)|Manage the information related to an App Store version of your app|
 |[`apps`](apps.md)|Manage your apps in App Store Connect|
 |[`beta-app-review-submissions`](beta-app-review-submissions.md)|Manage your application's TestFlight submissions|
 |[`beta-build-localizations`](beta-build-localizations.md)|Manage your beta builds localizations in App Store Connect|

docs/app-store-connect/app-store-versions.md

@@ -0,0 +1,81 @@
+
+app-store-versions
+==================
+
+
+**Manage the information related to an App Store version of your app**
+### Usage
+```bash
+app-store-connect app-store-versions [-h] [--log-stream STREAM] [--no-color] [--version] [-s] [-v]
+    [--log-api-calls]
+    [--json]
+    [--issuer-id ISSUER_ID]
+    [--key-id KEY_IDENTIFIER]
+    [--private-key PRIVATE_KEY]
+    [--certificates-dir CERTIFICATES_DIRECTORY]
+    [--profiles-dir PROFILES_DIRECTORY]
+    ACTION
+```
+### Optional arguments for command `app-store-connect`
+
+##### `--log-api-calls`
+
+
+Turn on logging for App Store Connect API HTTP requests
+##### `--json`
+
+
+Whether to show the resource in JSON format
+##### `--issuer-id=ISSUER_ID`
+
+
+App Store Connect API Key Issuer ID. Identifies the issuer who created the authentication token. Learn more at https://developer.apple.com/documentation/appstoreconnectapi/creating_api_keys_for_app_store_connect_api. If not given, the value will be checked from the environment variable `APP_STORE_CONNECT_ISSUER_ID`. Alternatively to entering `ISSUER_ID` in plaintext, it may also be specified using the `@env:` prefix followed by an environment variable name, or the `@file:` prefix followed by a path to the file containing the value. Example: `@env:<variable>` uses the value in the environment variable named `<variable>`, and `@file:<file_path>` uses the value from the file at `<file_path>`.
+##### `--key-id=KEY_IDENTIFIER`
+
+
+App Store Connect API Key ID. Learn more at https://developer.apple.com/documentation/appstoreconnectapi/creating_api_keys_for_app_store_connect_api. If not given, the value will be checked from the environment variable `APP_STORE_CONNECT_KEY_IDENTIFIER`. Alternatively to entering `KEY_IDENTIFIER` in plaintext, it may also be specified using the `@env:` prefix followed by an environment variable name, or the `@file:` prefix followed by a path to the file containing the value. Example: `@env:<variable>` uses the value in the environment variable named `<variable>`, and `@file:<file_path>` uses the value from the file at `<file_path>`.
+##### `--private-key=PRIVATE_KEY`
+
+
+App Store Connect API private key used for JWT authentication to communicate with Apple services. Learn more at https://developer.apple.com/documentation/appstoreconnectapi/creating_api_keys_for_app_store_connect_api. If not provided, the key will be searched from the following directories in sequence for a private key file with the name `AuthKey_<key_identifier>.p8`: private_keys, ~/private_keys, ~/.private_keys, ~/.appstoreconnect/private_keys, where <key_identifier> is the value of --key-id. If not given, the value will be checked from the environment variable `APP_STORE_CONNECT_PRIVATE_KEY`. Alternatively to entering `PRIVATE_KEY` in plaintext, it may also be specified using the `@env:` prefix followed by an environment variable name, or the `@file:` prefix followed by a path to the file containing the value. Example: `@env:<variable>` uses the value in the environment variable named `<variable>`, and `@file:<file_path>` uses the value from the file at `<file_path>`.
+##### `--certificates-dir=CERTIFICATES_DIRECTORY`
+
+
+Directory where the code signing certificates will be saved. Default:&nbsp;`$HOME/Library/MobileDevice/Certificates`
+##### `--profiles-dir=PROFILES_DIRECTORY`
+
+
+Directory where the provisioning profiles will be saved. Default:&nbsp;`$HOME/Library/MobileDevice/Provisioning Profiles`
+### Common options
+
+##### `-h, --help`
+
+
+show this help message and exit
+##### `--log-stream=stderr | stdout`
+
+
+Log output stream. Default `stderr`
+##### `--no-color`
+
+
+Do not use ANSI colors to format terminal output
+##### `--version`
+
+
+Show tool version and exit
+##### `-s, --silent`
+
+
+Disable log output for commands
+##### `-v, --verbose`
+
+
+Enable verbose logging for commands
+### Actions
+
+|Action|Description|
+| :--- | :--- |
+|[`create`](app-store-versions/create.md)|Add a new App Store version to an app using specified build.|
+|[`delete`](app-store-versions/delete.md)|Delete specified App Store version from Apple Developer portal|
+|[`modify`](app-store-versions/modify.md)|Update the app store version for a specific app.|