diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index f8b4052f..781a569f 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -51,7 +51,7 @@ jobs: run: npm run serve & - name: Check for broken links - run: npx broken-link-checker -ro --exclude /docs/managers/ --filter-level 3 --host-requests 8 --user-agent Chrome/90 --exclude https://fonts.gstatic.com/ --exclude https://github.com/galasa-dev/extensions/ http://localhost:9000 + run: npx broken-link-checker -ro --exclude /docs/managers/ --filter-level 3 --host-requests 8 --user-agent Chrome/90 --exclude https://fonts.gstatic.com/ --exclude https://github.com/galasa-dev/extensions/ --exclude https://*.algolia.net/ --exclude https://github.com/ http://localhost:9000 - name: Upload raw site uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0 @@ -208,7 +208,7 @@ jobs: run: npm run serve & - name: Check for broken links - run: npx broken-link-checker -ro --exclude /docs/managers/ --filter-level 3 --host-requests 8 --user-agent Chrome/90 --exclude https://fonts.gstatic.com/ --exclude https://github.com/galasa-dev/extensions/ http://localhost:9000 + run: npx broken-link-checker -ro --exclude /docs/managers/ --filter-level 3 --host-requests 8 --user-agent Chrome/90 --exclude https://fonts.gstatic.com/ --exclude https://github.com/galasa-dev/extensions/ --exclude https://*.algolia.net/ --exclude https://github.com/ http://localhost:9000 - name: Upload raw site uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0 diff --git a/src/data/nav.yaml b/src/data/nav.yaml index 82b3e75e..3dfef58f 100644 --- a/src/data/nav.yaml +++ b/src/data/nav.yaml @@ -87,6 +87,8 @@ path: /docs/writing-own-tests/binding-tests - title: Key principles for writing tests path: /docs/writing-own-tests/key-principles + - title: Controlling code execution after test failure + path: /docs/writing-own-tests/test-result-provider - title: The Galasa Ecosystem path: /docs/ecosystem items: diff --git a/src/markdown-pages/docs/cli-command-reference/installing-cli-tool.md b/src/markdown-pages/docs/cli-command-reference/installing-cli-tool.md index 3455bddc..cee6138a 100644 --- a/src/markdown-pages/docs/cli-command-reference/installing-cli-tool.md +++ b/src/markdown-pages/docs/cli-command-reference/installing-cli-tool.md @@ -32,8 +32,8 @@ On Mac: 3. Then you have a choice. Either install the latest version fo the `galasactl` tool, or install a specific version. 1. To install the latest version of `galasactl`: `brew install --no-quarantine galasactl` - 2. To install a specific version of `galasactl` (version 0.40.0 for example): - `brew install --no-quarantine galasactl@0.40.0` + 2. To install a specific version of `galasactl` (version 0.41.0 for example): + `brew install --no-quarantine galasactl@0.41.0` Note: You can check to see what versions are available using this: `brew tap-info galasa-dev/tap --json` and look in the `"cask_tokens"` part of the json file. diff --git a/src/markdown-pages/docs/ecosystem/ecosystem-installing-k8s.md b/src/markdown-pages/docs/ecosystem/ecosystem-installing-k8s.md index 9ccec473..69c7c385 100644 --- a/src/markdown-pages/docs/ecosystem/ecosystem-installing-k8s.md +++ b/src/markdown-pages/docs/ecosystem/ecosystem-installing-k8s.md @@ -237,14 +237,14 @@ Once you have successfully installed the Ecosystem, you can then deploy your Gal ## Upgrading the Galasa Ecosystem -Get the latest version of the Ecosystem chart and upgrade the Galasa Ecosystem to use the newer version of Galasa - for example version 0.40.0 - by running the following command: +Get the latest version of the Ecosystem chart and upgrade the Galasa Ecosystem to use the newer version of Galasa - for example version 0.41.0 - by running the following command: On Mac or Unix: ```console helm repo update \ helm upgrade galasa/ecosystem --reuse-values \ ---set galasaVersion=0.40.0 --wait +--set galasaVersion=0.41.0 --wait ``` On Windows (Powershell): @@ -252,7 +252,7 @@ On Windows (Powershell): ```console helm repo update ` helm upgrade galasa/ecosystem --reuse-values ` ---set galasaVersion=0.40.0 --wait +--set galasaVersion=0.41.0 --wait ``` where:
diff --git a/src/markdown-pages/docs/ecosystem/ecosystem-role-based-access.md b/src/markdown-pages/docs/ecosystem/ecosystem-role-based-access.md index 6fcb9a4b..ac9b35d3 100644 --- a/src/markdown-pages/docs/ecosystem/ecosystem-role-based-access.md +++ b/src/markdown-pages/docs/ecosystem/ecosystem-role-based-access.md @@ -81,7 +81,7 @@ The system also has some constraints on how these resources can be set up and us - The `owner` role can only be assigned using the Helm property, not via the command-line or REST interface. ## Upgrading to a version of Galasa which supports RBAC -When upgrading your Galasa service to a version 0.40.0 or later, any existing users will be assigned the role of `admin` without any extra action being required. +When upgrading your Galasa service to a version 0.41.0 or later, any existing users will be assigned the role of `admin` without any extra action being required. Under such circumstances you may wish to assign the role of `tester` to anyone who now should not have administration rights. diff --git a/src/markdown-pages/docs/manage-ecosystem/runs-get.md b/src/markdown-pages/docs/manage-ecosystem/runs-get.md index 51ac43ca..c66e227c 100644 --- a/src/markdown-pages/docs/manage-ecosystem/runs-get.md +++ b/src/markdown-pages/docs/manage-ecosystem/runs-get.md @@ -21,6 +21,7 @@ Use the tables provided to view the options for filtering test results, and for | `--result` | Use the [`--result` option](#result) to return test runs based on test run results. You can select more than one result by using a comma-separated list. The `--result` flag cannot be used in conjunction with the `--active` flag. The two flags are mutually exclusive. | | `--active` | Use the `--active` option to query tests that have not finished, so that you can quickly see which tests are currently running. The `--active` flag cannot be used in conjunction with the `--result` flag. The two flags are mutually exclusive.| | `--group` | Use the `--group` flag to get all the test runs associated with the given group name. A custom group name can be assigned to a set of test runs when submitting tests using the [galasactl runs submit](./ecosystem-cli-runs-submit) command, otherwise a random group name will be assigned. `--group` can be used with other flags, like `--age`, `--requestor`, and `--result`, to filter test runs as desired, but it cannot be used alongside the `--name` flag. | +| `--tags` | Use the `--tags` flag to get all the test runs that have the given test tags assigned to them. A custom set of tags can be assigned to a set of test runs when submitting tests using the [galasactl runs submit](./ecosystem-cli-runs-submit) command. `--tags` can be used with other flags, like `--age`, `--requestor`, and `--result`, to filter test runs as desired. Multiple tags can be supplied by either providing `--tags` with a comma-separated list of tags (e.g. `--tags tag1,tag2,tag3`) or by providing multiple `--tags` flags separately (e.g. `--tags tag1 --tags tag2 --tags tag3`). | Table 2: The following table shows the options that you can set on the `galasactl runs get` command to display test run results in different formats: diff --git a/src/markdown-pages/docs/managers.md b/src/markdown-pages/docs/managers.md index 92aafabb..87f43065 100644 --- a/src/markdown-pages/docs/managers.md +++ b/src/markdown-pages/docs/managers.md @@ -30,6 +30,17 @@ Each Manager is tagged with one of the follow readiness indicators: +
+IMS TM Managers + + Name | Description | +| :-------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **[IMS TM Manager](/docs/managers/ims-tm-manager)**
![alpha](../../images/alpha.svg) | Provides configuration information for pre-existing IMS TM systems. Drives provisioning services from other managers, e.g. z/OS 3270 Manager. | + +
+ + +
Cloud Managers diff --git a/src/markdown-pages/docs/managers/galasa-ecosystem-manager.md b/src/markdown-pages/docs/managers/galasa-ecosystem-manager.md index 442055f0..225fe0fe 100644 --- a/src/markdown-pages/docs/managers/galasa-ecosystem-manager.md +++ b/src/markdown-pages/docs/managers/galasa-ecosystem-manager.md @@ -154,7 +154,7 @@ The following are properties used to configure the Galasa Ecosystem Manager. | Required: | No | | Default value: | None | | Valid values: | Valid URL | -| Examples: | galasaecosystem.isolated.mvp.zip=https://github.com/galasa-dev/isolated/releases/download/v0.40.0/galasa-isolated-mvp-0.40.0.zip | +| Examples: | galasaecosystem.isolated.mvp.zip=https://github.com/galasa-dev/isolated/releases/download/v0.41.0/galasa-isolated-mvp-0.41.0.zip |
diff --git a/src/markdown-pages/docs/running-simbank-tests/running-simbank-tests-cli-offline.md b/src/markdown-pages/docs/running-simbank-tests/running-simbank-tests-cli-offline.md index 3c9c5390..f46aa3e9 100644 --- a/src/markdown-pages/docs/running-simbank-tests/running-simbank-tests-cli-offline.md +++ b/src/markdown-pages/docs/running-simbank-tests/running-simbank-tests-cli-offline.md @@ -48,7 +48,7 @@ In order to run the Galasa SimBanks tests you need to add some configuration inf ## Running the SimBank IVT test class by using the CLI -The SimBank tests are located in the `maven` directory of the `isolated.zip` downloadable file. Complete the following steps to run the SimBankIVT test that is provided with Galasa. The following example uses SimBank OBR version `0.40.0`. +The SimBank tests are located in the `maven` directory of the `isolated.zip` downloadable file. Complete the following steps to run the SimBankIVT test that is provided with Galasa. The following example uses SimBank OBR version `0.41.0`. Remember to initialise your local environment by running the `galasactl local init` command and to start the SimPlatform server by running the `run-simplatform.sh` script, as described in the [Running Galasa SimBank using the CLI offline](simbank-cli-offline) documentation. @@ -59,14 +59,14 @@ You are now ready to run a local Galasa test offline with just the contents of t On Mac or Unix: ``` galasactl runs submit local --log - \ ---obr mvn:dev.galasa/dev.galasa.simbank.obr/0.40.0/obr \ +--obr mvn:dev.galasa/dev.galasa.simbank.obr/0.41.0/obr \ --class dev.galasa.simbank.tests/dev.galasa.simbank.tests.SimBankIVT \ --localMaven file:////Users/youruserid/Downloads/isolated/maven ``` On Windows (Powershell): ``` galasactl runs submit local --log - ` ---obr mvn:dev.galasa/dev.galasa.simbank.obr/0.40.0/obr ` +--obr mvn:dev.galasa/dev.galasa.simbank.obr/0.41.0/obr ` --class dev.galasa.simbank.tests/dev.galasa.simbank.tests.SimBankIVT ` --localMaven file:////Users/youruserid/Downloads/isolated/maven ``` @@ -80,7 +80,7 @@ On Mac or Unix: ``` galasactl runs submit local --log - \ ---obr mvn:dev.galasa/dev.galasa.simbank.obr/0.40.0/obr \ +--obr mvn:dev.galasa/dev.galasa.simbank.obr/0.41.0/obr \ --class dev.galasa.simbank.tests/dev.galasa.simbank.tests.BasicAccountCreditTest \ --localMaven file:////Users/youruserid/Downloads/isolated/maven ``` @@ -89,7 +89,7 @@ On Windows (Powershell): ``` galasactl runs submit local --log - ` ---obr mvn:dev.galasa/dev.galasa.simbank.obr/0.40.0/obr ` +--obr mvn:dev.galasa/dev.galasa.simbank.obr/0.41.0/obr ` --class dev.galasa.simbank.tests/dev.galasa.simbank.tests.BasicAccountCreditTest ` --localMaven file:////Users/youruserid/Downloads/isolated/maven ``` diff --git a/src/markdown-pages/docs/running-simbank-tests/running-simbank-tests-cli.md b/src/markdown-pages/docs/running-simbank-tests/running-simbank-tests-cli.md index e9441990..47b06717 100644 --- a/src/markdown-pages/docs/running-simbank-tests/running-simbank-tests-cli.md +++ b/src/markdown-pages/docs/running-simbank-tests/running-simbank-tests-cli.md @@ -50,7 +50,7 @@ In order to run the Galasa SimBanks tests you need to add some configuration inf The SimBank tests are held in the Galasa simplatform repository in GitHub. To start running the tests you need to clone the repository, if you have not already done so. To find out how to clone the cli repository, follow the instruction in the [Running Galasa SimBank online](../running-simbank-tests/simbank-cli) documentation. -After cloning the repository, complete the following steps to run the SimBankIVT test that is provided with Galasa. The following example uses SimBank OBR version `0.40.0` and Galasa uber OBR version `0.40.0`. +After cloning the repository, complete the following steps to run the SimBankIVT test that is provided with Galasa. The following example uses SimBank OBR version `0.41.0` and Galasa uber OBR version `0.41.0`. You can find the version of the `dev.galasa.simbank.obr` that you are using by looking in the `pom.xml` file in the `dev.galasa.simbank.obr` folder. The `dev.galasa.uber.obr` is the OBR that contains all the bundles that are needed for Galasa to work including Managers, any required dependencies, the framework, etc. The version of the `dev.galasa.uber.obr` depends on which version of Galasa you have installed. @@ -84,13 +84,13 @@ If you are a Mac or Linux user, the messages will be almost identical. On Mac or Unix: ``` galasactl runs submit local --log - \ ---obr mvn:dev.galasa/dev.galasa.simbank.obr/0.40.0/obr \ +--obr mvn:dev.galasa/dev.galasa.simbank.obr/0.41.0/obr \ --class dev.galasa.simbank.tests/dev.galasa.simbank.tests.SimBankIVT ``` On Windows (Powershell): ``` galasactl runs submit local --log - ` ---obr mvn:dev.galasa/dev.galasa.simbank.obr/0.40.0/obr ` +--obr mvn:dev.galasa/dev.galasa.simbank.obr/0.41.0/obr ` --class dev.galasa.simbank.tests/dev.galasa.simbank.tests.SimBankIVT ``` where: @@ -107,7 +107,7 @@ On Mac or Unix: ``` galasactl runs submit local --log - \ ---obr mvn:dev.galasa/dev.galasa.simbank.obr/0.40.0/obr \ +--obr mvn:dev.galasa/dev.galasa.simbank.obr/0.41.0/obr \ --class dev.galasa.simbank.tests/dev.galasa.simbank.tests.BasicAccountCreditTest ``` @@ -115,7 +115,7 @@ On Windows (Powershell): ``` galasactl runs submit local --log - ` ---obr mvn:dev.galasa/dev.galasa.simbank.obr/0.40.0/obr ` +--obr mvn:dev.galasa/dev.galasa.simbank.obr/0.41.0/obr ` --class dev.galasa.simbank.tests/dev.galasa.simbank.tests.BasicAccountCreditTest ``` diff --git a/src/markdown-pages/docs/writing-own-tests/test-result-provider.md b/src/markdown-pages/docs/writing-own-tests/test-result-provider.md new file mode 100644 index 00000000..d6727f23 --- /dev/null +++ b/src/markdown-pages/docs/writing-own-tests/test-result-provider.md @@ -0,0 +1,37 @@ +--- +path: "/docs/writing-own-tests/test-result-provider" +title: "Controlling code execution after test failure" +--- + +As a tester, you may want the ability to access the result so far of a Galasa test at different points during the test lifecycle, to control the execution of code based on that result. + +The `ITestResultProvider` can be used to get the test result so far after the first method invokation in a test. It can give you access to if the test is in Passed or Failed state. You can then check the result so far in your test code and choose to call or not call non-test methods based on this. + +The annotation `@TestResultProvider` injects a `ITestResultProvider` object into your test class. The Core Manager updates the provider with the test result so far after each `@BeforeClass`, `@Before`, `@Test`, `@After` and `@AfterClass` method. + +To use this capability, simply include the lines of code below in your test: + +```java +@TestResultProvider +public ITestResultProvider testResultProvider; +``` + +In the example below, the `@AfterClass` method retrieves the result from the `ITestResultProvider` and checks if it is Failed. If the result is Failed, the method `myCustomCleanupMethod` is called which could contain some diagnostic collection or clean up of resources, which the tester only wants to be run if the test Failed. + +```java +@AfterClass +public void afterClassMethod() throws FrameworkException { + if (testResultProvider.getResult().isFailed()) { + myCustomCleanupMethod(); + } +} + +private void myCustomCleanupMethod() { + try { + // Some custom cleanup logic that only happens on failures. + } catch(Exception ex) { + logger.error("Failing while cleaning up in myCustomCleanupMethod()"); + // Ignore the problem. + } +} +``` diff --git a/src/markdown-pages/docs/writing-own-tests/writing-test-classes.md b/src/markdown-pages/docs/writing-own-tests/writing-test-classes.md index c6805d3e..888b7a47 100644 --- a/src/markdown-pages/docs/writing-own-tests/writing-test-classes.md +++ b/src/markdown-pages/docs/writing-own-tests/writing-test-classes.md @@ -18,7 +18,7 @@ Once loaded by the Galasa test runner, there are a few further annotations that **`@Test`**
-This annotation identifies a method as one that contains test code. Such methods are executed by Galasa by the order in which they appear in the test class - from top to bottom. If a test method fails, the following test methods are bypassed to encourage short, sharp parallel testing. This behaviour can be overridden using the `@StopOnError` annotation on the `class` statement. +This annotation identifies a method as one that contains test code. Such methods are executed by Galasa by the order in which they appear in the test class - from top to bottom. If a test method fails, the following test methods are bypassed to encourage short, sharp parallel testing. This behaviour can be overridden using the `@ContinueOnTestFailure` annotation on the `class` statement. When a test method succeeds it is marked as PASSED, and if there is an exception it is marked FAILED. Managers can override this marking - for example, if a test throws a specific exception, a Manager could set the result to DISASTER. diff --git a/src/markdown-pages/highlights.md b/src/markdown-pages/highlights.md index 2bb25a2a..2d0874eb 100644 --- a/src/markdown-pages/highlights.md +++ b/src/markdown-pages/highlights.md @@ -17,7 +17,65 @@ We have the following available Slack channels: Access the Galasa source code in [GitHub](https://github.com/galasa-dev) and open issues in the [project management repository](https://github.com/galasa-dev/projectmanagement). -## 0.40.0 - Release Highlights +## 0.41.0 - Release Highlights + +### Changes affecting the Galasa Service + +- Streams Service: + + - Streams can now be created, updated and deleted using a new set of REST API endpoints. + - Command-line tool list or delete report streams, or create and apply changes to stream resources using the galasactl apply -f command. + +- Resource Management: + + - Custom managers can have their cleanup code executed in a new "Resource Cleanup Monitor" construct, allowing for more cleanup of remote resources after test failures. + - Cleanup monitors load resource management code from a given test stream. + - New REST API commands allow manipulation and query of stream definitions. + - Improved command-line can enable or disable monitors dynamically. + +- Tagging of test run results: + + - Test runs can be submitted with extra tags, which are combined with the annotation tags inside the test source code such that the test results are also tagged with the combined set of tags. + - Extra tagging in test run details can help categorise failures and can contribute to better charting of results over time. + +- Kubernetes CPU and Memory resources: + + - For each pod, including pods that runs tests, can optionally specify resources constraints. Telling Kubernetes how much CPU and memory each pod needs helps Kubernetes spread workload among nodes in the cluster. The default install now specifies such limits. + +- Miscellaneous: + + - Tests which are re-queued or cancelled get marked as finished, rather than appearing to be active forever. + - Querying runs with the --active flag no longer returns cancelled runs. Cancelled runs are marked as finished now. + - Web user interface footer shows the version of the Galasa service, and a health indicator. + - Test runner handles exceptions from rogue manager code better, so test cleanup can be performed. + +### Changes affecting local tests + +- Managers vetoing method execution changes: + + - Managers implementing anyReasonTestMethodShouldBeIgnored will not be called to see whether the manager wants to veto the execution of @Before or @After methods separately to the test method they surround. Managers will only be asked whether the test method should be ignored or not, which will cause the "before" and "after" methods to also be ignored. + +- Tests can find out if the test has failed yet: + + - Test code can ask for a TestStatus provider to be injected into their object. This can be used to find out if the test has failed yet or not, and can be useful when doing complex cleanup of resources allocated during the test. + +- 3270 manager now copes with SSCP-LU-DATA: + + - Unformatted data screens are now processed and the contents is visible to test applications. + +- @ContinueOnTestFailure annotations: + + - These annotations are now respected by the Galasa framework. + +### Other changes + +- Make contributing code easier: + + - Code for the galasactl command-line tool has moved git repositories, to the galasa repository. + This simplifies the build process and makes changes on a fork easier to build and verify. + +
+0.40.0 - Release Highlights - Nameculture: @@ -59,6 +117,8 @@ Access the Galasa source code in [GitHub](https://github.com/galasa-dev) and ope - Added experimental instructions describing how to install the Galasa Service into Minikube. Minikube can be used when developing contributions to Galasa, but is not suitable as a useable run time platform. - Various other component version upgrades and bug fixes. +
+
0.39.0 - Release Highlights diff --git a/src/pages/index.js b/src/pages/index.js index b032fa9f..699bbbe1 100644 --- a/src/pages/index.js +++ b/src/pages/index.js @@ -126,7 +126,7 @@ const IndexPage = () => ( Learn more