Skip to content

Commit a4ab343

Browse files
mefellowsmefellows
committed
feat: initial test template docs and major reorganisation
1 parent 6ff8751 commit a4ab343

File tree

9 files changed

+857
-198
lines changed

9 files changed

+857
-198
lines changed

Diff for: ‎website/docs/docs/ai/index.md

+13-195
Original file line numberDiff line numberDiff line change
@@ -1,205 +1,23 @@
11
---
22
title: Test Generation featuring HaloAI
3-
sidebar_label: AI ⚡️
3+
sidebar_label: Introduction
44
---
55

6-
import { AiDownloadTable } from "../../../src/components/index";
7-
8-
:::note
9-
PactFlow's Test Generation features are currently available through an open beta program. During this stage, all customers can explore the capabilities within fixed usage limits ([more info](#usage-limits)).
10-
11-
If you don’t already have a PactFlow account, get started with our [free Starter plan](https://pactflow.io/try-for-free?&utm_source=pactflow-docs&utm_content=ai).
12-
:::
13-
146
PactFlow's Test Generation feature, is an AI-powered capability designed to help users quickly generate Pact tests. By providing access to source code, OpenAPI descriptions, or traffic capture data, it reduces the time and effort needed to create and maintain contract tests.
157

16-
## Enabling the feature
17-
18-
This feature is disabled by default on all organisations. To enable the feature:
19-
20-
1. In the UI, navigate to "Settings" -> "Preferences" -> "System Preferences".
21-
2. Check the "Test Generation" checkbox.
22-
3. Save the settings.
23-
24-
:::note
25-
This setting is only available in the new user interface.
26-
:::
27-
28-
## Installation
29-
30-
For *nix users (including Windows users running WSL/msys2/mingw), use the following command to download and install:
31-
32-
```sh
33-
curl https://download.pactflow.io/ai/get.sh | sh
34-
```
35-
36-
For Windows PowerShell users, use the following command to download and install:
37-
38-
```pwsh
39-
Invoke-WebRequest -Uri https://download.pactflow.io/ai/get.ps1 | Invoke-Expression
40-
```
41-
42-
### Installation Options
43-
44-
There are some options which you can set during installation.
45-
46-
For a full list, see the `--help`/`-h` command
47-
48-
```sh
49-
curl https://download.pactflow.io/ai/get.sh | sh -s -- -h
50-
```
51-
52-
- `--verbose` / `-v` / `PACTFLOW_AI_VERBOSE`: Enable verbose output
53-
- `--quiet` / `-q` / `PACTFLOW_AI_QUIET`: Disable progress output
54-
- `--yes` / `-y` / `PACTFLOW_AI_YES`: Disable confirmation prompt and assume 'yes'
55-
- `--destination` / `-d` / `PACTFLOW_AI_DESTINATION`: Specify the directory to install the binary
56-
- `--default-host` / `PACTFLOW_AI_DEFAULT_HOST`: Choose a default host triple rather than autodetecting
57-
- `--no-modify-path` / `PACTFLOW_AI_NO_MODIFY_PATH`: Don't configure the PATH environment variable
58-
59-
Verify the installation by running `pactflow-ai` to ensure it executes successfully.
60-
61-
### Manual installation
62-
63-
Alternatively, download the latest version for your OS and architecture from the table below. Be sure to add it to your environment's `PATH`:
64-
65-
<AiDownloadTable />
66-
67-
:::note
68-
Linux GNU users will require glibc version 2.23 or later.
69-
70-
Environments which do not use glibc, or use a version of glibc prior to 2.23, should instead use the musl variant.
71-
:::
72-
73-
## Usage
74-
75-
Running `pactflow-ai --help` will show detailed usage for any subcommands.
76-
77-
### Authentication
78-
79-
Authentication requires valid PactFlow API Tokens, which can be obtained from the `Settings > Tokens` [page](/docs/user-interface/settings/api-tokens) of your PactFlow organisation.
80-
81-
You can authenticate with the CLI in several ways:
82-
83-
#### Environment Variables
84-
85-
Set the following environment variables, and the CLI will use them to communicate with PactFlow:
86-
87-
```
88-
export PACT_BROKER_BASE_URL="https://YOUR_ORG.pactflow.io"
89-
export PACT_BROKER_TOKEN="YOUR_TOKEN"
90-
```
91-
92-
#### Local configuration file
93-
94-
Configure the CLI using the following commands:
95-
96-
```
97-
pactflow-ai config set broker.base-url https://YOUR_ORG.pactflow.io
98-
pactflow-ai config set broker.auth.token YOUR_TOKEN
99-
```
100-
101-
This creates a `.pact.toml` file, which should be added to your `.gitignore`.
102-
103-
#### User configuration file
104-
105-
Alternatively, store the user configuration file in `~/.config/pact/config.toml` (Unix and macOS), and `%APPDATA%\pact\config.toml` (Windows).
106-
107-
Pass the `--user` flag when running `pactflow-ai config set` to write to this file automatically.
108-
109-
## Usage Limits
110-
111-
Access to the feature is metered based on the number of tests generated in a 30-day rolling window. Credits are allocated as follows:
112-
113-
1. Starter Plans: 10 credits per month
114-
2. All other plans: 10 credits per purchased user per month
115-
116-
Credits are allocated to the organisation, not individual users.
117-
118-
### Examples
119-
120-
1. **Starter Plan**
121-
122-
If you have a free plan, you are entitled to 10 tests per month no matter how many active users there are.
123-
124-
2. **Team Plan**
125-
126-
If you are on a Team 10 Plan (5 active users), you receive for 10 credits x 10 users for a total of 100 credits.
127-
128-
3. **Enterprise Plan**
129-
130-
If you are on an Enterprise Plan with 50 users (42 active users), you receive for 10 credits x 50 users for a total of 500 credits.
131-
132-
### Consumption
133-
134-
Each successfully generated test consumes a credit. If your organisation reaches its limit, you will see the following error:
135-
136-
```
137-
ERROR: Client error: 403 Forbidden AI credits are exhausted.
138-
```
139-
140-
:::note
141-
The limits and your usage against these limits are not currently visible on the subscription page. This will be added in due course.
142-
:::
143-
144-
## Role Based Access Control (RBAC)
145-
146-
In addition to the [system preference](#enabling-the-feature) that enables AI within your workspace, PactFlow's RBAC model allows you to manage access to specific AI capabilities for your users. For detailed information, refer to the [permissions documentation](/docs/permissions#ai).
147-
148-
By default, users assigned the [User](/docs/permissions/predefined-roles#user) role have access to all AI permissions ([`ai:*`](/docs/permissions#ai)), unless:
149-
150-
- Your account was created prior to the release of the AI feature (September 2024), or
151-
- You have customized your role configuration.
152-
153-
**Granting Access:**
154-
155-
To enable AI access for users, ensure that the assigned role includes either the [`ai:*`](/docs/permissions#ai) permission (granting access to all AI features) or the specific permissions required for the desired functionality.
156-
157-
## Tutorials
158-
159-
We've created an [in-browser tutorial](https://docs.pactflow.io/docs/tutorials#getting-started-with-pactflow-ai) to help you get started with PactFlow's test generation feature.
160-
161-
## Troubleshooting
162-
163-
### "You do not have the necessary permissions to access this resource"
164-
165-
If you encounter the following error:
166-
167-
```
168-
ERROR: Client error: 403 Forbidden You do not have the necessary permissions to access this resource, Required permissions: ai:generation:request-response
169-
```
170-
171-
This indicates that the user does not have the required permissions to access the feature.
172-
173-
**Resolution:**
174-
175-
Ensure the necessary AI [permission](/docs/permissions#ai) is assigned to the user's role. In this example, the `ai:generation:request-response permission` is required. For more details, refer to the [permissions](/docs/permissions#ai) documentation.
176-
177-
### "AI Disabled in System Preferences"
178-
179-
If the error message states that "AI is disabled in system preferences", this means the AI feature has not been enabled for your account.
180-
181-
**Resolution:**
182-
Enable the AI feature by following the steps outlined [above](#enabling-the-feature).
183-
184-
### "AI credits are exhausted"
185-
186-
If you encounter the following error:
187-
```
188-
ERROR: Client error: 403 Forbidden AI credits are exhausted.
189-
```
190-
191-
This indicates that you have exceeded the AI credits on the account and can no longer use any AI features.
192-
193-
**Resolution:**
194-
Upgrade your plan to increase your credit allocation, or wait until the credits reset after 30 days. For more details on usage limits, see the [documentation](#usage-limits).
195-
196-
## Getting help and providing feedback
8+
## Features
1979

198-
For feedback, feature requests, or assistance with the tool, join our [slack channel](https://pact-foundation.slack.com/archives/C07K2FT0XKK) or speak directly with your Account Manager. For general support, please follow the [usual methods](https://support.smartbear.com/pactflow/message/).
10+
PactFlow AI includes a suite of features to improve the efficiency and quality of your contract tests:
19911

200-
### "AI is not enabled for your plan"
12+
* Generating Pact tests using various combinations of OpenAPI descriptions, code, or request-response pairs
13+
* Customizable prompts and test templates to tailor output to your specific needs
14+
* Seamless integration with PactFlow's Role-based Access Controls feature, enabling you to enforce and manage organizational policies
15+
* Code review (upcoming), enabling you to evaluate the quality of existing Pact tests and suggest best-practice improvements
20116

202-
If the error message states that "AI features are not enabled on your plan", your plan does not include AI features as part of its entitlements.
17+
PactFlow AI is available as a CLI and supports the following programming languages:
20318

204-
**Resolution:**
205-
Upgrade to a plan that has AI enabled.
19+
* Java
20+
* JavaScript / TypeScript
21+
* Golang (coming soon)
22+
* .NET (coming soon)
23+
* Python (coming soon)

Diff for: ‎website/docs/docs/ai/learning.md

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
title: Further Learning
3+
sidebar_label: Further Learning
4+
---
5+
6+
We've created an [in-browser tutorial](https://docs.pactflow.io/docs/tutorials#getting-started-with-pactflow-ai) to help you get started with PactFlow's test generation feature.

Diff for: ‎website/docs/docs/ai/managing.md

+136
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
---
2+
title: Managing PactFlow AI
3+
sidebar_label: Managing PactFlow AI
4+
---
5+
6+
## Usage Limits
7+
8+
Access to the feature is limited by a credit system, where each test generated uses 1 credit. Usage is metered based on the number of tests generated per calendar month, and resets on the first day of each month.
9+
10+
Credits are allotted to your account based on your plan:
11+
12+
1. PactFlow developer plans: 10 credits per month (total)
13+
2. Enterprise Plus or plans with Contract Testing Pro Addon: 30 credits per purchased user per month
14+
3. All other plans (Individual, Team and Enterprise Plans): 10 credits per month
15+
16+
Note that credits are allocated to the organization, not individual users.
17+
18+
### Examples
19+
20+
1. **Starter (Developer) Plan**
21+
22+
If you have a free PactFlow plan, you are entitled to 10 tests per month no matter how many active users there are.
23+
*NOTE: this is a legacy plan and is no longer available for signups.*
24+
25+
2. **Team Plan**
26+
27+
If you are on a Team 10 Plan (5 active users), you receive for 10 credits x 10 users for a total of 100 credits.
28+
29+
3. **Enterprise Plan**
30+
31+
If you are on an Enterprise Plan with 50 users (42 active users), you receive for 30 credits x 50 users for a total of 1500 credits.
32+
33+
### Consumption
34+
35+
Each successfully generated test consumes a credit. If your organization reaches its limit, you will see the following error:
36+
37+
```
38+
ERROR: Client error: 403 Forbidden AI credits are exhausted.
39+
```
40+
41+
:::note
42+
The limits and your usage against these limits are not currently visible on the subscription page. They may be checked using the `pactflow-ai status` command.
43+
:::
44+
45+
46+
## Role Based Access Control (RBAC)
47+
48+
In addition to the [system preference](#managing-the-feature) that enables AI within your workspace, PactFlow's RBAC model allows you to manage access to specific AI capabilities for your users. For detailed information, refer to the [permissions documentation](/docs/permissions#ai).
49+
50+
By default, users assigned the [User](/docs/permissions/predefined-roles#user) role have access to all AI permissions ([`ai:*`](/docs/permissions#ai)), unless:
51+
52+
- Your account was created prior to the release of the AI feature (September 2024), or
53+
- You have customized your role configuration.
54+
55+
**Granting Access:**
56+
57+
To enable AI access for users, ensure that the assigned role includes either the [`ai:*`](/docs/permissions#ai) permission (granting access to all AI features) or the specific permissions required for the desired functionality. Once granted, access may be checked using the `pactflow-ai status` command.
58+
59+
## Configuring PactFlow AI
60+
61+
### Managing the Feature
62+
63+
Access to PactFlow AI can be managed by a PactFlow Administrator (or user with the [`system_preference:manage:*`](/docs/permissions#system_preferencemanage) permission):
64+
65+
1. In the UI, navigate to "Settings" -> "Preferences" -> "System Preferences".
66+
2. Check the "AI Test Generation" checkbox.
67+
3. Save the settings.
68+
69+
### Configuring the CLI
70+
71+
The PactFlow AI CLI supports flexible configuration through environment variables and configuration files. It allows both **user-level** and **local** configuration, with local settings taking precedence.
72+
73+
### Configuration Methods
74+
75+
#### 1. Environment Variables
76+
The CLI automatically picks up configuration from environment variables. For example, to configure authentication, set the following:
77+
78+
```sh
79+
export PACT_BROKER_BASE_URL="https://YOUR_WORKSPACE.pactflow.io"
80+
export PACT_BROKER_TOKEN="YOUR_TOKEN"
81+
```
82+
83+
#### 2. Local Configuration File
84+
To store configuration at the project level, use:
85+
86+
```sh
87+
pactflow-ai config set broker.base-url https://YOUR_WORKSPACE.pactflow.io
88+
pactflow-ai config set broker.auth.token YOUR_TOKEN
89+
```
90+
91+
This creates a `.pact.toml` file in the project directory. Add this file to `.gitignore` to avoid committing sensitive data.
92+
93+
**Recommended setup**
94+
95+
1. Commit `.pact.toml` with shared configuration options for your team.
96+
1. Set sensitive user-level data (such as your API token) through environment variables or the user config (see below).
97+
98+
The CLI will automatically combine the options.
99+
100+
#### 3. User Configuration File
101+
For global configuration shared across multiple projects, use:
102+
103+
```sh
104+
pactflow-ai config set --user broker.base-url https://YOUR_WORKSPACE.pactflow.io
105+
pactflow-ai config set --user broker.auth.token YOUR_TOKEN
106+
```
107+
108+
This stores the configuration in `~/.config/pact/config.toml` (Unix/macOS) or `%APPDATA%\pact\config.toml` (Windows). On unix-like systems, it will also respect the `XDG_CONFIG_HOME` environment variable.
109+
110+
### Managing Configuration
111+
112+
You can inspect or modify settings with the following commands:
113+
114+
- **List all settings**: `pactflow-ai config list`
115+
- **Retrieve a setting**: `pactflow-ai config get broker.base-url`
116+
- **Override with an explicit file**: Use `--config-file <path>`
117+
118+
For advanced options, including certificate management for corporate environments, refer to:
119+
120+
```sh
121+
pactflow-ai config --help
122+
```
123+
<!--
124+
## Monitoring Usage
125+
126+
### Monitoring Usage via the UI
127+
128+
TBC
129+
130+
[screenshot]
131+
132+
### Monitoring Usage via the API
133+
134+
TBC
135+
136+
[Link to API](https://smartbear.portal.swaggerhub.com/pactflow/default/pactflow_saas_api#/metric/get_metrics) -->

0 commit comments

Comments
 (0)