-
Notifications
You must be signed in to change notification settings - Fork 244
docs: add docs for test-set-config #655
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Sarthak160
wants to merge
1
commit into
main
Choose a base branch
from
config-docs
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
87 changes: 87 additions & 0 deletions
87
versioned_docs/version-3.0.0/running-keploy/test-set-configuration-file.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,87 @@ | ||
--- | ||
title: Test-Set Configuration File (config.yaml) | ||
description: How to define per test-set configuration (like overriding appCommand) using config.yaml placed alongside mock.yaml. | ||
--- | ||
|
||
# Test-Set Configuration File | ||
|
||
Each Keploy test-set can have its own lightweight configuration file. Create a file named `config.yaml` in the **same directory** as the corresponding `mock.yaml` (and any captured test data). This lets you override runtime parameters (like the application start command) or supply per test-set scripts/metadata without affecting other sets. | ||
|
||
## File Location | ||
|
||
``` | ||
tests/ | ||
my-test-set/ | ||
mock.yaml # Recorded mocks for this test-set | ||
config.yaml # Test-set specific configuration (optional) | ||
... # Any other assets | ||
``` | ||
|
||
If `config.yaml` is absent, Keploy falls back to global/default settings. | ||
|
||
## Minimal Use Case: Override appCommand | ||
|
||
To run the application under test with a different command (env vars, flags, image, binary path, etc.) only for this test-set, place a `config.yaml` beside `mock.yaml` containing just the `appCommand` key: | ||
|
||
```yaml | ||
appCommand: docker run --rm --name python-api \ | ||
--network python-api-network \ | ||
-e DATABASE_URL=mysql+pymysql://user:password@mysql-db:3306/fastapi_db \ | ||
-e SECRET_KEY=super-secret-key-for-testing \ | ||
-e ACCESS_TOKEN_EXPIRE_MINUTES=5 \ | ||
-p 8000:8000 python-rest-api | ||
``` | ||
|
||
That’s all you need when your only goal is to override how the app starts during replay for this specific test-set. | ||
|
||
## Available (Optional) Keys | ||
|
||
You can extend `config.yaml` later if needed. Common fields (all optional unless noted): | ||
|
||
common example - | ||
|
||
```yaml | ||
preScript: "" | ||
postScript: "" | ||
template: {} | ||
mockRegistry: null | ||
appCommand: <my test-set specific app command> | ||
metadata: | ||
name: flask-mysql-app | ||
``` | ||
|
||
| Key | Purpose | | ||
| -------------- | -------------------------------------------------------------------------------------------------------------------------- | | ||
| `appCommand` | Command Keploy should run to start the application for this test-set. Overrides any global/app-level command. | | ||
| `preScript` | Shell script (inline or path) executed before starting the app / replaying tests. Use for seeding DB, cleaning state, etc. | | ||
| `postScript` | Shell script executed after tests finish (cleanup, exporting artifacts). | | ||
| `mockId` | Explicit identifier if you maintain multiple mock groups in the same folder and need disambiguation. | | ||
| `template` | A map/object for advanced templating or variable substitution (future/advanced usage). | | ||
| `mockRegistry` | (Optional) Registry or remote reference for fetching mocks if they aren’t local. | | ||
| `metadata` | Arbitrary informational fields (e.g., `name`, `owner`, `tags`). Non-functional but useful for organization. | | ||
|
||
> Note: Only include the keys you actually need. Unused keys can be omitted entirely. | ||
|
||
## Tips & Best Practices | ||
|
||
1. Keep the minimal override: if you only change the start command, only keep `appCommand`—simpler diffs and fewer merge conflicts. | ||
2. Avoid storing secrets directly; prefer referencing environment files or secret managers. | ||
3. Use descriptive `metadata.name` to make CI logs clearer when multiple test-sets run. | ||
4. Validate the command manually once before relying on it in automated replay. | ||
|
||
## Troubleshooting | ||
|
||
| Symptom | Likely Cause | Fix | | ||
| ----------------------------------- | ------------------------------------------------------------------- | --------------------------------------------------------- | | ||
| App doesn’t start for this test-set | Typo in `appCommand` or missing image | Run the command locally; ensure image/tag exists. | | ||
| Pre/post script not executed | Wrong key name (`preScript`/`postScript`) or YAML indentation error | Validate YAML (e.g., with `yamllint`) and check spelling. | | ||
| Wrong mocks used | Missing/incorrect `mockId` when multiple sets present | Add/adjust `mockId` or standardize directory structure. | | ||
|
||
## Quick Validation Checklist | ||
|
||
- `config.yaml` sits beside `mock.yaml`. | ||
- YAML parses (`yamllint config.yaml`). | ||
- Only necessary keys included. | ||
- `appCommand` works standalone in your shell. | ||
|
||
--- |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Inconsistent indentation in the directory structure example. Use consistent spacing (either tabs or spaces) for proper formatting in documentation.
Copilot uses AI. Check for mistakes.