diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0967e5f..8e135ed 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -96,15 +96,7 @@ PHPUnit tests can be run using the following command: npm run test:php ``` -To generate a code coverage report, make sure to start the testing environment with coverage mode enabled: - -```bash -npm run env start -- --xdebug=coverage - -npm run test:php -``` - -You should see the html coverage report in the `tests/_output/html` directory and the clover XML report in `tests/_output/php-coverage.xml`. +For detailed testing instructions including running specific tests, generating coverage reports, and troubleshooting, see the **[Testing Guide](docs/guides/testing.md)**. ### Building the plugin for distribution diff --git a/docs/README.md b/docs/README.md index bd152a7..2791a53 100644 --- a/docs/README.md +++ b/docs/README.md @@ -26,6 +26,11 @@ Documentation for the WordPress MCP Adapter - transform WordPress abilities into - **[Common Issues](troubleshooting/common-issues.md)** - Quick fixes and debugging techniques +## Contributing + +- **[Contributing Guide](../CONTRIBUTING.md)** - Development setup, coding standards, and contribution workflow +- **[Testing Guide](guides/testing.md)** - Running and writing tests with wp-env + ## Quick Navigation ### New to MCP Adapter diff --git a/docs/guides/testing.md b/docs/guides/testing.md index ed8a969..e6c151d 100644 --- a/docs/guides/testing.md +++ b/docs/guides/testing.md @@ -1,15 +1,14 @@ # Testing the MCP Adapter -This guide explains how to run and write tests for the MCP Adapter. The suite supports two modes: - -- Fast unit mode (no WordPress DB needed) -- Full WordPress integration mode (uses the WP test suite) +This guide explains how to run and write tests for the MCP Adapter using `wp-env`. ## Prerequisites -- PHP 7.4+ -- Composer -- Optional (for full WP tests): a local MySQL/MariaDB server or a WordPress test DB +- Node.js 20.x (NVM recommended) +- Docker +- Git + +See [CONTRIBUTING.md](../../CONTRIBUTING.md#prerequisites) for full setup requirements. ## Test Layout @@ -19,105 +18,150 @@ This guide explains how to run and write tests for the MCP Adapter. The suite su ## Running Tests -### 1) Fast Unit Mode (recommended for local dev) +The MCP Adapter uses `wp-env` to provide a containerized WordPress environment with all dependencies. This eliminates the need for manual database setup or WordPress installation. -No database required. A lightweight bootstrap provides minimal WordPress shims. +### Starting the Test Environment + +First, ensure the wp-env environment is running: ```bash -# from mcp-adapter/ -MCP_ADAPTER_FAST_UNIT=1 composer test +npm run wp-env start ``` -What this does: -- Loads Composer autoloaders and Abilities API -- Provides shims for `add_filter()`, `apply_filters()`, `wp_set_current_user()`, `is_user_logged_in()`, i18n, etc. -- Runs the entire suite from `tests/Unit` and `tests/Integration` +This starts a WordPress instance at http://localhost:8888 with all required dependencies. + +### Running All Tests -Tip: Run a single file or test via PHPUnit’s filter: +Run the full PHPUnit test suite: ```bash -MCP_ADAPTER_FAST_UNIT=1 vendor/bin/phpunit -c phpunit.xml.dist tests/Unit/Handlers/ToolsHandlerCallTest.php -MCP_ADAPTER_FAST_UNIT=1 vendor/bin/phpunit -c phpunit.xml.dist --filter test_image_result_is_converted_to_base64_with_mime_type +npm run test:php ``` -### 2) Full WordPress Integration Mode +This executes both unit and integration tests in the wp-env container. -This uses the official WP test suite and requires a DB. +### Running Specific Tests + +You can pass PHPUnit arguments to the test script using `--`: ```bash -# Install WP test suite -composer test:install +# Run a specific test by name +npm run test:php -- --filter test_execute_with_public_mcp_filtering + +# Run a specific test file +npm run test:php -- tests/Unit/Handlers/ToolsHandlerCallTest.php -# Run tests (will load WordPress bootstrap) -composer test +# Run tests matching a pattern +npm run test:php -- --filter "Tools.*" ``` -If you see “Error establishing a database connection”, ensure your DB is running and credentials in `bin/install-wp-tests.sh` (or env variables) are correct. +### Test Coverage + +To generate code coverage reports, restart the environment with Xdebug coverage mode enabled: + +```bash +# Enable coverage mode +npm run wp-env start -- --xdebug=coverage + +# Run tests (coverage will be generated) +npm run test:php +``` + +Coverage reports will be generated: +- HTML report: `tests/_output/html/index.html` (open in your browser) +- Clover XML: `tests/_output/php-coverage.xml` (for CI/CD tools) ## Observability and Error Handling -The suite verifies that: -- Request counts, successes, errors, and timings are recorded -- Error envelopes adhere to JSON-RPC shape: `{ jsonrpc, id, error: { code, message, data? } }` - -Fixtures: -- `DummyObservabilityHandler` captures `increment()` and `timing()` calls -- `DummyErrorHandler` collects logs for assertions - -## Test Coverage - -### Handlers and Routing -- Initialize: protocolVersion, serverInfo, capabilities, and instructions -- Tools: list, list-all (available=true), call (permission errors, exceptions, image/text responses) -- Resources: list, read -- Prompts: list, get -- System: ping, setLoggingLevel -- Transport: routing, unknown method, cursor compatibility, metric tags -- Error handling: consistent error response formats and logging -- Session management: state handling and cleanup - -### MCP Transport Compliance (HttpTransport) -- **MCP 2025-06-18 Streamable HTTP specification compliance** -- HTTP methods: GET, POST, DELETE, OPTIONS -- JSON-RPC 2.0 message format validation -- Session management and termination -- CORS preflight and header handling -- Notification handling (202 responses) -- Accept header validation for different request types -- Error response format compliance -- Protocol version header handling -- Batch request processing -- Session state management and cleanup -- HTTP request context validation - -**Test Location**: `tests/Integration/HttpTransportTest.php` (comprehensive test coverage) +The test suite includes fixtures for verifying observability and error handling: + +**DummyObservabilityHandler** (`tests/Fixtures/DummyObservabilityHandler.php`) +- Captures `record_event()` calls with event names, tags, and optional timing data +- Stores events in `$events` array for test assertions +- Used to verify that requests, successes, errors, and timings are properly tracked + +**DummyErrorHandler** (`tests/Fixtures/DummyErrorHandler.php`) +- Captures `log()` calls with messages, context, and error types +- Stores logs in `$logs` array for test assertions +- Used to verify error handling and logging behavior + +Tests verify that error responses adhere to JSON-RPC 2.0 format: `{ jsonrpc, id, error: { code, message, data? } }` ## Writing New Tests - Place unit tests under `tests/Unit/.../*Test.php` -- Favor fast unit mode while developing -- For behavior depending on WordPress state (e.g., current user), in fast mode use: - - `wp_set_current_user(1);` - - `add_filter('mcp_validation_enabled', '__return_false');` +- Place integration tests under `tests/Integration/.../*Test.php` - Use fixtures in `tests/Fixtures` or create your own test doubles +- Follow the Arrange-Act-Assert (AAA) pattern +- Mock external dependencies using PHPUnit mocks +- Test files should mirror the source structure with a `Test.php` suffix -## Coverage +Example test structure: -You can enable coverage locally with Xdebug: +```php +handle($request, $server); + + // Assert + $this->assertSame($expected, $result); + } +} ``` ## Troubleshooting -- DB connection error: use fast unit mode or start a local DB and rerun `composer test:install` -- Class not found in tests: run `composer dump-autoload` -- WP hook callback missing (e.g. `__return_false`): fast unit mode shims these; ensure `MCP_ADAPTER_FAST_UNIT=1` is set +### Environment Issues + +If wp-env fails to start: + +```bash +# Stop and clean the environment +npm run wp-env stop +npm run wp-env clean + +# Restart +npm run wp-env start +``` + +### Test Failures + +- **Class not found**: This typically occurs after adding new classes, pulling changes, or switching branches. Regenerate the Composer autoloader to resolve: + ```bash + npm run wp-env run tests-cli --env-cwd=wp-content/plugins/mcp-adapter/ composer dump-autoload + ``` + The `--env-cwd` flag sets the working directory inside the Docker container to ensure Composer operates on the plugin's `composer.json`. + +- **Permission errors**: Ensure Docker has the necessary permissions to mount volumes +- **Port conflicts**: wp-env uses ports 8888 and 8889 by default. If these are in use, stop other services or configure different ports in `.wp-env.json` + +### Accessing the Test Environment + +- WordPress site: http://localhost:8888 +- Admin dashboard: http://localhost:8888/wp-admin/ (admin/password) +- Run WP-CLI commands: `npm run wp-env run tests-cli YOUR_COMMAND` + +## Continuous Integration -## CI Recommendations +The repository has comprehensive CI testing via GitHub Actions (`.github/workflows/test.yml`): -- Default to fast unit mode for speed -- Optionally add a matrix job that runs the full WP suite on a stable WordPress version +**Test Matrix:** +- PHP versions: 8.4, 8.3, 8.2, 8.1, 8.0, 7.4 +- WordPress versions: latest, trunk +- Coverage: Enabled for PHP 8.4 + WordPress latest (uploaded to Codecov) +**Automated Checks:** +- PHPUnit tests via `npm run test:php` +- PHPCS coding standards +- PHPStan static analysis (Level 8) +All tests run automatically on pull requests and pushes to trunk.