Fix duration, import remote plan, dynamic ref parametrization, remove obsoleted execution methods

Highlights

• Fix test duration enforcement
• Support fetching remote repo for shell discover
• Fix default framework, remove old execution methods
• Login after each test using the --test parameter
• Enable context based plan parametrization
• Add support for dynamic ref evaluation
• Add import plan feature

Full Changelog

• Protect args in rsync call
• Set tree root for the default plan tree as well
• Properly set the tmt script shebang on rhel-8
• Use image exists to check for container image
• Updates docs with example on adjust & prepare+
• Fix test duration enforcement
• Skip missing debuginfo packages in recommend
• Explicitly document extending the plan environment
• Fix ownership of a tmp directory propagated to container
• Support fetching remote repo for shell discover
• Fix default framework, remove old execution methods
• Add support for Artemis v0.0.46 upcoming release
• Handle an fmf fetch error in remote plan clone
• Do not truncate RunError output in verbose mode
• Warn user about data erasing after prepare step
• Formalize data package passed to the Result class
• Change order of plugin and guest classes in files
• Add compatible as a new hardware specification key
• Add tpm as a new hardware specification key
• Move the hardware specification into a separate page
• Improve fmf-id processing
• Add test for TTY state in test environment
• Login after each test using the --test parameter
• Mention version where important features were added
• Handle dist-git-sources for gitlab
• Fix getting CentOS via --how virtual
• Capture uncaught exceptions when using testcloud
• Update the overview of essential classes
• When following command line --how, do not iterate over step data
• Convert utils' Run unit tests to class-less tests
• Enable variable expansion for dynamic references
• Support beaker libraries as recommended packages
• Add SpecBasedContainer.to_minimal_spec() method (#1637)
• Enable context based plan parametrization
• Coverage for tests defined under discover.shell
• Fix /tests/run/shell access permission problem
• Add step data classes for provisioning and report
• Adjust support for the dynamic ref evaluation
• Add support for dynamic ref evaluation
• Add a test for fmf id parsing and normalization
• Fix NO_COLOR not being honored by executed command output
• Moves common command options into one place
• Log full chain of exceptions, not just the first cause
• Drop deprecated PluginIndex
• Add message to failure tags in junit report
• Update the pip installation plan
• Remove pointless reimport of tmt.base in discover steps
• Do not use f-string as a docstring
• Fix use of variable before assignment in Plan.go
• Fix variable redefinition in discover/shell.py
• Fix guest distro detection, do not throw results away
• Fix a typo in cpu.sockets hardware requirement
• Make links relative for report html
• Review all uses of type: ignore and link relevant issues
• Enhance ClickOptionDecorator type to announce identity
• Update /tests/execute/upgrade/override duration
• Review all uses of Any and link relevant issues
• Annotate all cast() calls with respective issues
• Move the Result class into a separate file
• Unblock mypy's follow-import setting
• Use set comprehension instead of list-in-set sequence
• Use enumerate() instead of range(len())
• Use dict comprehension instead of tuple-in-list-in-dict
• Replace two more list comprehensions with generators
• Replace GeneralError's "origin" with Python's "raise from"
• Fix normalization of the Plan.context key
• Adds a missing import to polarion plugin
• Update all linters to their most recent versions
• Define CPU HW components with more granularity
• Adjust the support for importing remote plans
• Add import plan feature and tests
• Add type annotations to base.py
• Fix test depending on ordering of elements in junit XML
• Adds type annotations to tmt itself
• Remove custom yet same implementation of step's show()
• Make sure repo_copy is gone before make srpm

Custom results, polarion import, typing, refactoring...

Highlights

• Custom results implementation
• Add Polarion as a source for test case import
• Support to install package from URL
• Refactor internal link handling and storage
• Substantial progress on typing annotations

Full Changelog

• Fix recommended packages handling for rpm-ostree
• Add EFI configuration to the tmt-reboot script
• Fix adjust for precommit test
• Fix provision for coreos image
• Emit only non-default keys when constructing a test from execute
• Add flake8 config file for easier integration with IDEs
• Multiple scripts for CLI prepare -h shell
• Allow mypy to cover the whole tmt.plugins and tmt.steps
• Add typing for steps/execute/upgrade.py
• Fix name & default value of polarion's upload field
• Fixes enhancing of environment by local's guest implementation
• Ignore plan -n when searching for upgrade path
• Document & correct use of class conversion methods
• Print fmf tree location when schema unit test fails
• Custom results implementation
• Refactors internal link handling and storage
• Allow mypy to cover whole tmt.steps.prepare
• Add typing for steps/execute/internal.py
• Use workdir with safe names without special chars
• Adjust support for installing remote packages
• Support to install package from URL
• Make sure short option '-x' is covered
• Add Polarion as a source for test case import
• Print path to the used ssh identity
• Add typing for steps/prepare/__init__.py
• Use generator instead of list comprehension with any/all
• Fixes handling of default of --key in connect plugin
• Update test data for the debuginfo install test
• Add a helper for importing a member from a module
• Fix plan schema to allow custom context dimensions
• Allow mypy to cover whole tmt.steps.discover
• Remove support for the obsoleted detach executor
• Add typing for steps/discover/fmf.py
• Fix importing for pylero
• Allow mypy to cover whole tmt.steps.provision
• Replace blank "type: ignore" with more specific waivers
• Use the SerializableContainer for plugins' data
• Enhance SerializableContainer with default key value inspection
• Moves validation and normalization mixins to utils

Remote playbooks in prepare; Common attributes extended; Nested root warning

Main improvements and fixes:

• Error out if reboot timeout is exceeded
• Initialize workdir thread-safe
• Add support for remote playbooks in prepare
• Print escaped command suitable for manual debugging
• Less eager to disable nitrate case during export
• Move tag and tier to common core attributes
• Hint user if 'tmt init' creates nested root

Whole change log:

• Unify Polarion case searching
• Error out if reboot timeout is exceeded
• Initialize workdir thread-safe
• Add support for remote playbooks in prepare
• Add plan schema for errata and minute plugins
• Correct rhts command names in stories file
• Print escaped command suitable for manual debugging
• Fix report plugin not getting arguments from fmf file
• Less eager to disable nitrate case during export
• Move tag and tier to common core attributes
• Use /bin/bash instead of /bin/sh for execute
• Reorder step and their base plugin classes
• Fix prepare/multihost docs to match implementation
• Teach schema validation tests to peek into other trees
• Clarify motivation for creating id during export
• Add link-polarion option and fix link searching bug
• Ignore race in last-run symlink creation
• Fix polarion tcmscaseid search
• Force order of clean operations
• Convert status/clean argument to option
• Report enabled plans/tests/stories via option
• Hint user if 'tmt init' creates nested root
• Require libvirt-daemon-config-network as well
• Add type annotation for /steps/prepare/install.py
• Encapsulate created file within script's dataclass
• Adds normalization layer to base classes based on fmf
• Fixes data class used for local guest creation
• Fixes Artemis guest data class link
• Making tests/full more usable
• Add typing for tmt/steps/provision/podman.py
• Add typing for tmt/steps/provision/testcloud.py
• Add typing for tmt/steps/provision/local.py
• Remove unused keys parameter from wake() methods
• Adds types describing tmt constructs when as stored in raw fmf data
• Typing /steps/provision/connect.py
• Allow raising an exception on validation errors
• Inject logger object to base node classes inheritance
• Fixes use of SSH keys in testcloud and connect plugins
• Annotate tmt.steps.provision
• Ask mypy to show error codes in its messages
• Testcloud expects disk and memory to be int
• Do not inherit classes from object
• Use keyword-only init in base fmf-backed classes
• Use decorator to register plugin methods
• Demonstrate inheritance on a virtual test example
• Add a simple hint how to write user messages
• Add typing for steps/finish/ansible.py
• Remove unneeded parameters for step load/save (#1428)
• Normalize step data to be always stored a list internally
• Display test/plan/story name in parametrized schema tests
• Allow numbers and booleans to be values of environment
• Give a reasonable error for old data format
• Add typing for tmt/steps/finish/init.py
• add typing for steps/discover/shell.py
• Adds a fmf node validation layer to core classes (Test/Plan/Story)
• Add missing keys role and where to schemas
• Extend plan schema with all known step plugin schemas
• Correcting rhts aliases & adding rhts opt.
• Adds a generic "wait for condition" primitive
• Disallow push/pull/execute if guest is unavailable
• Rename "default how" step attribute to enhance its visibility
• Use textwrap.dedent() to unindent docstrings

Reboot timeout and reconnect improvements; polarion report; library recursion

Main improvements and fixes:

• Backwards compatibility for rstrnt-abort, rstrnt-report-log and rstrnt-report-result.
• Package installation using rpm-ostree.
• Polarion report plugin.
• Implement reboot reconnect timeout configuration and make sure machine was rebooted before reconnect.
• Cut circular dependency of libraries.
• Print result for execute -v.
• Fixing the directory name escaping in 'cd' command.
• Precommit hooks to call tmt * lint

Whole change log:

• Reboot has to check for boot time
• Fix path inside pre-commit test
• Cut circular dependency of libraries
• Update 'Develop' section of contribution docs
• Precommit hooks to call tmt * lint
• Schema loading helpers
• Package schemas in subdirectories too
• Implement reboot reconnect timeout configuration
• Add missing report specifications/docs
• Print result for execute -v
• Correct import assumption about script
• Ask mypy to check whole tmt.steps.report package
• Fixing the directory name escaping in 'cd' command
• Add polarion report plugin
• Add schemas for plans
• Write extra-nitrate as soon as possible
• Retry git clone without --depth=1 if it failed
• Support to lint attribute 'id'
• Do not apply test command filter in upgrade
• Fix export.py typing issues
• Refactor location and signature of Phase's go() method
• Simplify abort handling
• Backwards compatibility for rstrnt-abort
• Add type annotations for tmt/cli.py
• Add typing for tmt/export.py
• add typing for steps/discover/init.py
• Convert guest implementations to use serializable container for load/save
• Detect plugins by entry_point as well
• Add typing for steps/execute/__init__.py
• Add typing for tmt/convert.py
• Remove duplicated dist-git-source/dist-git-type
• Add typing for steps/report/junit.py
• Add typing for steps/report/html.py
• add typing for steps/report/display.py
• Add typing for steps/report/init.py
• tmt-file-submit is a bash script
• Add type annotations for tmt/options.py
• Backwards compatibility for rstrnt-report-log
• Support conditional requires with pip as well
• Remove duplicated short option in tmt lint
• Adjust the rstrnt-report-result implementation
• Backwards compatibility for rstrnt-report-result
• Set the 1777 permision for /var/tmp/tmt
• Adjust the fix for the rpm-ostree intallation
• Fix package installation using rpm-ostree
• Handle empty fmf file as an empty dictionary
• Fix distgit testsuite after tmt packaging change

CoreOS support, typing, result attribute, polarion export

Highlights

• Initial polarion support for test export
• Add CoreOS support to the testcloud provision
• Add support for the rpm-ostree package manager
• Implement the test result attribute

Full changelog

• Require fresh testcloud with coreos support
• Bad substitution in tmt-reboot
• Ignore "certificate verify failed" error when handling retries
• Cache content of each loaded environment file
• Initial polarion support for test export
• Fixes names of Artemis API versions
• Convert FmfIdType from TypedDict to a dataclass
• Add CoreOS support to the testcloud provision
• Run containers with root user
• Retry getting environment file
• Test import --general is default now
• Add typing for steps/finish/shell.py
• Enhance tmt.utils.retry_session with timeout support
• Adjust the rpm-ostree install implementation
• Add support for the rpm-ostree package manager
• Add environment-file to possible Plan keys
• Avoid Library url conflict if repo doesn't exist
• Check changes are pushed before export nitrate
• Add typing for beakerlib.py
• Unbundle template from the report.html plugin
• Rename uuid to identifier to prevent conflicts
• Use must for all mandatory spec requirements
• Fail import for packages starting with minus sign
• Adds support for newer Artemis API versions
• Disable the extra verbose progress in testcloud
• Refactor internal executor scripts
• Adds "missing" imports to help IDEs follow objects
• Add typing for steps/init.py
• Implement the test result attribute
• Add typing for plugins/init.py
• Detect legacy relevancy during import as well
• Implement the new user story key priority
• Implement new class SerializableContainer
• Add schema for stories
• Add typing for steps/prepare/shell.py
• Add typing for steps/prepare/ansible.py
• Require fmf >= 1.1.0 (we need validation support)
• Package fmf root into the source tarball as well
• Add JSON Schema for tests
• Exclude namespaced backup in beakerlib
• Use --depth=1 when cloning git repos by default
• Handle missing nitrate user during export
• Removes unused GuestContainer.container_id attribute
• Every subpackage must require the main tmt package
• Introduce dataclasses as a requirement
• Avoid re-using image/instance for different values by testcloud plugin
• Add typing for multihost.py
• Except nitrate xmlrpc issues during import
• Exclude beakerlib's backup dir from guest.pull()
• Increase duration for the reboot-related tests
• Several release-related tests and docs adjustments

Upgrade plugin, improved discover, identifiers...

Highlights

• Implement upgrade execute plugin
• Dist-git-source for Discover (fmf, shell)
• Add a new core key id for unique identifiers

Full Changelog

• Command 'tmt clean' should not run rsync at all
• Dist-git-source for Discover (fmf, shell)
• Adjust the new id key implementation
• Add a new core key id for unique identifiers
• Recommend qemu emulators for other architectures
• Copy the whole git repo only if necessary
• Reveal hidden errors during testcloud booting
• Use time for timeout handling in Guest.reconnect()
• Split Guest class to separate SSH-capable guests
• Explicitly set the docs language in the config
• Kill the running test when rebooting
• Extend the reboot timeout to allow system upgrades
• Allow selecting tasks directly from upgrade config
• Adjust the new upgrade execute plugin
• Allow specifying command for reboot
• Implement upgrade execute plugin
• Buildrequire python3-docutils, append plan adjust
• Implement tmt tests export --nitrate --link-runs
• Detect component from general plan during import
• Adjust the support for steps in standalone mode
• Add results method to ExecutePlugin
• Implement a common ancestor for Action and Plugin
• Allow abstractly excluding steps from runs
• Correctly handle tests --name '.' shortcut
• Rename WorkdirType to WorkdirArgumentType
• Fix workdir parameter type for tmt.utils.Common
• Allows importing jira issues as link-relates
• Enables mypy coverage for empty-ish Python files
• Adds type annotations to tmt.templates
• Prevent infinite recursion if --id is set
• Enable mypy check for Artemis provision plugin
• Adjust provision dry mode propagation, add a test
• Introduce new _options attribute to Common class
• Add specification for remote plans referencing
• Bootstrap type annotations
• Execute script should not be used with discover
• Add the arch key to the hardware specification
• Fix pip install instructions
• Disable network access when building in copr
• Ignore list for dist-git-source
• Remove the obsoleted detach execute method
• Fix login during execute and prepare step
• Import from Makefile with missing build target

Multiarch testcloud, artemis provision, test export improvements...

• Add multiarch support to testcloud provision
• Consistent summary for test export --nitrate
• Allow dry mode for tests export --nitrate
• Add a nice provisioning progress to Artemis plugin
• Add support for the where keyword to execute
• Adjust support for export of multiple tests
• Add support for exporting multiple tests
• Basic multihost test for the httpd web server
• Update multihost specification with guest groups
• Add a provision plugin for Artemis
• Fix exclude option in fmf discover
• Reduce the number of execute calls for reboot
• Add support for reboot in interactive mode

Plan export, exclude search, import restraint metadata...

Highlights

• New feature: tmt plan export
• Add --exclude search option
• Allow import from restraint's metadata file (#1043)
• Add a command to setup shell completions
• Option to export fmf-id from run discover -h fmf
• Story.example can hold list of strings

Full changelog

• Add a command to setup shell completions
• Use /tmp instead of /run/user/ if not available
• Use separate examples in the test specification
• Add more story examples, simplify examples export
• Story.example can hold list of strings
• Fix traceback when connect plugin is used without hostname.
• Adjust disabled shell expansion in Common.run()
• Disable shell expansion in Common.run() by default
• Build epel9 packages, update install docs
• Adjust the full test wrapper and document it
• Test which compiles tmt and runs its testsuite
• Add --exclude search option
• Correct regex for require read from metadata file
• Update document for creating virtual environment
• Option to export fmf-id from run discover -h fmf
• Allow import from restraint's metadata file (#1043)
• Do not disable building for power arch on Fedora
• Update documentation for plan parametrization
• Make .vscode ignored by git
• Drops basestring and unicode built-ins from utils
• Fix timeout behaviour in testcloud plugin
• Fixes possible test of None-ish CWD when running a command
• Remove workdir only when its defined
• Adjust the new tmt plan export feature
• New feature: tmt plan export

Order key, multihost provision & prepare, rsync handling...

• Prevent koji build failures on unsupported arches
• Check remote git URL if it is accessible
• Implement a generic requires for all plugins
• Run commands in podman provision via bash
• Adjust implementation of the new order attribute
• Implement the Core attribute order
• Fix link generation in report.html
• Improve step name handling
• Enable shared volume mounts in podman provision
• Add support for multihost provision and prepare
• Adjust the dnf support for rsync install
• Add dnf support for rsync install
• Update links and refs after migration to teemtee
• Track output for reboot purposes on per-test basis
• Fix test --name '.' used with multiple plans
• Tweak test suite (duration, centos:8, datadir)
• Use os.pathsep to separate TMT_PLUGINS paths (#1049)
• Document framework:shell exit codes
• Add html2text to the convert pip dependencies

Storing logs, ssh multiplexing, support r/o file systems, ...

Highlights

• Add support for ssh multiplexing
• Ignore read/only file systems reboot script errors
• Handle rsync installation on read-only distros
• Minor modification of test result specification
• Implement variables for storing logs and artifacts

Full Changelog

• Make reboot support a bit more backward compatible
• Ensure that workdir has a correct selinux context
• Use centos:stream8 image instead of centos:8
• Disable X11 forwarding in ssh connections
• Fix traceback for login after last report
• Use TMT_TEST_DATA as location for rlFileSubmit
• Implement variables for storing logs and artifacts
• Adjust rsync installation on read-only distros
• Handle rsync installation on read-only distros
• Add hardware specification for hostname
• Correctly import multiple bugs from Makefile
• Remove dependency on the python3-mock package
• Adjust linting of manual test files
• Check Markdown files in tmt lint if manual=True
• Adjust pulling logs from the guest during finish
• Add guest.pull() to the finish step
• Update virtualization hints for session connection
• Improve error message for empty git repositories
• Minor modification of test result specification
• Use where instead of on in the multihost spec
• Clarify that path is defined from the tree root
• Adjust ansible requires for containers preparation
• Move the reboot scripts to a read/write directory
• Ignore read/only file systems reboot script errors
• Require either ansible or ansible-core
• Set the TMT_TREE variable during test execution
• Clarify that 'until' means until and including
• Update test debugging examples with --force option
• Add bios.method to hardware spec
• Improve environment variables specification a bit
• Adjust the ssh connection multiplexing
• Add support for ssh multiplexing