Skip to content
Draft
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
191 changes: 1 addition & 190 deletions README.md
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hyperlinking to each binding readme would also be beneficial.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is a good idea. I was going to do that, but our READMEs aren't consistent right now with what they do and don't contain, and I was thinking it might make sense to update them to mostly point to our website docs, and right now the coding agents we're targeting can't access the internet.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added an issue to track this - #16740

Original file line number Diff line number Diff line change
Expand Up @@ -329,196 +329,7 @@ bazel test //<language>/... --test_tag_filters=this,-not-this
If there are multiple `--test_tag_filters`, only the last one is considered,
so be careful if also using an inherited config

### Java

<details>
<summary>Click to see Java Test Commands</summary>

To run unit tests:

```shell
bazel test //java/... --test_size_filters=small
```
To run integration tests:

```shell
bazel test //java/... --test_size_filters=medium
```
To run browser tests:

```shell
bazel test //java/... --test_size_filters=large --test_tag_filters=<browser>
```

To run a specific test:

```shell
bazel test //java/test/org/openqa/selenium/chrome:ChromeDriverFunctionalTest
```

</details>

### JavaScript

<details>
<summary>Click to see JavaScript Test Commands</summary>

To run the tests run:

```shell
bazel test //javascript/selenium-webdriver:all
```

You can use `--test_env` to pass in the browser name as `SELENIUM_BROWSER`.

```shell
bazel test //javascript/selenium-webdriver:all --test_env=SELENIUM_BROWSER=firefox
```

</details>

### Python

<details>
<summary>Click to see Python Test Commands</summary>

Run unit tests with:
```shell
bazel test //py:unit
```

To run common tests with a specific browser:

```shell
bazel test //py:common-<browsername>
```

To run common tests with a specific browser (include BiDi tests):

```shell
bazel test //py:common-<browsername>-bidi
```

To run tests with a specific browser:

```shell
bazel test //py:test-<browsername>
```

To run all Python tests:

```shell
bazel test //py:all
```

</details>

### Ruby

<details>
<summary>Click to see Ruby Test Commands</summary>

Test targets:

| Command | Description |
| -------------------------------------------------------------------------------- | -------------------------------------------------- |
| `bazel test //rb/...` | Run unit, all integration tests and lint |
| `bazel test //rb:lint` | Run RuboCop linter |
| `bazel test //rb/spec/...` | Run unit and integration tests for all browsers |
| `bazel test //rb/spec/... --test_size_filters small` | Run unit tests |
| `bazel test //rb/spec/unit/...` | Run unit tests |
| `bazel test //rb/spec/... --test_size_filters large` | Run integration tests for all browsers |
| `bazel test //rb/spec/integration/...` | Run integration tests for all browsers |
| `bazel test //rb/spec/integration/... --test_tag_filters firefox` | Run integration tests for local Firefox only |
| `bazel test //rb/spec/integration/... --test_tag_filters firefox-remote` | Run integration tests for remote Firefox only |
| `bazel test //rb/spec/integration/... --test_tag_filters firefox,firefox-remote` | Run integration tests for local and remote Firefox |

Ruby test targets have the same name as the spec file with `_spec.rb` removed, so you can run them individually.
Integration tests targets also have a browser and remote suffix to control which browser to pick and whether to use Grid.

| Test file | Test target |
| ------------------------------------------------------- | ---------------------------------------------------------------- |
| `rb/spec/unit/selenium/webdriver/proxy_spec.rb` | `//rb/spec/unit/selenium/webdriver:proxy` |
| `rb/spec/integration/selenium/webdriver/driver_spec.rb` | `//rb/spec/integration/selenium/webdriver:driver-chrome` |
| `rb/spec/integration/selenium/webdriver/driver_spec.rb` | `//rb/spec/integration/selenium/webdriver:driver-chrome-remote` |
| `rb/spec/integration/selenium/webdriver/driver_spec.rb` | `//rb/spec/integration/selenium/webdriver:driver-firefox` |
| `rb/spec/integration/selenium/webdriver/driver_spec.rb` | `//rb/spec/integration/selenium/webdriver:driver-firefox-remote` |

Supported browsers:

* `chrome`
* `edge`
* `firefox`
* `firefox-beta`
* `ie`
* `safari`
* `safari-preview`

In addition to the [Common Options Examples](#common-options-examples), here are some additional Ruby specific ones:
* `--test_arg "-eTimeouts"` - test only specs which name include "Timeouts"
* `--test_arg "<any other RSpec argument>"` - pass any extra RSpec arguments (see `bazel run @bundle//bin:rspec -- --help`)

Supported environment variables for use with `--test_env`:

- `WD_SPEC_DRIVER` - the driver to test; either the browser name or 'remote' (gets set by Bazel)
- `WD_REMOTE_BROWSER` - when `WD_SPEC_DRIVER` is `remote`; the name of the browser to test (gets set by Bazel)
- `WD_REMOTE_URL` - URL of an already running server to use for remote tests
- `DOWNLOAD_SERVER` - when `WD_REMOTE_URL` not set; whether to download and use most recently released server version for remote tests
- `DEBUG` - turns on verbose debugging
- `HEADLESS` - for chrome, edge and firefox; runs tests in headless mode
- `DISABLE_BUILD_CHECK` - for chrome and edge; whether to ignore driver and browser version mismatches (allows testing Canary builds)
- `CHROME_BINARY` - path to test specific Chrome browser
- `CHROMEDRIVER_BINARY` - path to test specific ChromeDriver
- `EDGE_BINARY` - path to test specific Edge browser
- `MSEDGEDRIVER_BINARY` - path to test specific msedgedriver
- `FIREFOX_BINARY` - path to test specific Firefox browser
- `GECKODRIVER_BINARY` - path to test specific GeckoDriver

To run with a specific version of Ruby you can change the version in `rb/.ruby-version` or from command line:

```shell
echo '<X.Y.Z>' > rb/.ruby-version
```
</details>

### .NET

<details>
<summary>Click to see .NET Test Commands</summary>

.NET tests currently only work with pinned browsers, so make sure to include that.

Run all tests with:

```shell
bazel test //dotnet/test/common:AllTests --pin_browsers=true
```

You can run specific tests by specifying the class name:

```shell
bazel test //dotnet/test/common:ElementFindingTest --pin_browsers=true
```

If the module supports multiple browsers:

```shell
bazel test //dotnet/test/common:ElementFindingTest-edge --pin_browsers=true
```

</details>

### Rust

<details>
<summary>Click to see Rust Test Commands</summary>

Rust tests are run with:

```shell
bazel test //rust/...
```
</details>
Language specific testing guides can be found in a `TESTING.md` file in the applicable directory.

### Linux

Expand Down
134 changes: 134 additions & 0 deletions dotnet/TESTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# .NET Testing Guide

This guide helps contributors write tests in the Selenium .NET codebase.

## Test Framework

* Tests use NUnit.
* All tests inherit from `DriverTestFixture`.
* Test HTML pages accessed via properties like `simpleTestPage`, `javascriptPage`.
* `WaitFor<T>()` provides waiting with 5-second default timeout.

```csharp
[TestFixture]
public class MyFeatureTest : DriverTestFixture
{
[Test]
public void ShouldFindElement()
{
driver.Url = simpleTestPage;
IWebElement element = driver.FindElement(By.Id("foo"));
Assert.That(element.Text, Is.EqualTo("expected"));
}

[Test]
[IgnoreBrowser(Browser.Safari, "Safari doesn't support this")]
public void ShouldDoSomething()
{
// Skipped on Safari
}
}
```

## Running Tests

Bazel creates test targets for each browser. Always use `--pin_browsers`.

```shell
bazel test //dotnet/test/common:AllTests --pin_browsers=true # Default browser (Firefox)
bazel test //dotnet/test/common:AllTests-chrome --pin_browsers=true
bazel test //dotnet/test/common:AllTests-firefox --pin_browsers=true
bazel test //dotnet/test/common:AllTests-edge --pin_browsers=true

# Additional Arguments
bazel test //dotnet/... --flaky_test_attempts=3 --pin_browsers=true
bazel test //dotnet/... --test_output=all --pin_browsers=true
```

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few lines about running tests in the IDE.

Suggested change
### IDE Support
The test infrastructure supports Visual Studio's and Rider's IDE, which can be more convenient in the inner loop when developing features and writing tests. Simply open the solution after building with `bazel` and the IDE should work like normal.
Note that any bugs involving a discrepancy between the `.csproj` and `BUILD.bazel` files would manifest in running the tests through the IDE rather than with `bazel test`; any changes that affect either build systems should be tested with `bazel test` to ensure proper functionality.

## Attributes

### Skipping Tests

| Attribute | When to Use |
|-----------|-------------|
| `[IgnoreBrowser(Browser.X, "reason")]` | Skip test for specific browser |
| `[IgnorePlatform("windows", "reason")]` | Skip test on specific OS |
| `[IgnoreTarget("net8", "reason")]` | Skip test on specific .NET version |
| `[Ignore("reason")]` | Skip test entirely (NUnit built-in) |

```csharp
[Test]
[IgnoreBrowser(Browser.Safari, "Safari doesn't support multiple instances")]
[IgnoreBrowser(Browser.IE, "IE is flaky")]
public void TestWithMultipleDrivers()
{
}

[Test]
[IgnorePlatform("windows", "Thread time not supported")]
public void TestLinuxOnly()
{
}
```

Browser values: `Browser.Chrome`, `Browser.Firefox`, `Browser.Edge`, `Browser.Safari`, `Browser.IE`, `Browser.Remote`, `Browser.All`

### Driver Lifecycle

| Attribute | When to Use |
|-----------|-------------|
| `[NeedsFreshDriver(IsCreatedBeforeTest = true)]` | Fresh driver before test |
| `[NeedsFreshDriver(IsCreatedAfterTest = true)]` | Fresh driver after test |

```csharp
[Test]
[NeedsFreshDriver(IsCreatedBeforeTest = true, IsCreatedAfterTest = true)]
[IgnoreBrowser(Browser.Safari, "Safari doesn't support multiple instances")]
public void TestRequiringFreshDriver()
{
IWebDriver driver2 = EnvironmentManager.Instance.CreateDriverInstance();
try
{
// Test with multiple drivers
}
finally
{
driver2.Quit();
}
}
```

## Helpers

From `DriverTestFixture`:

| Member | Description |
|--------|-------------|
| `driver` | Current WebDriver instance |
| `simpleTestPage`, `javascriptPage`, etc. | Test page URLs |
| `WaitFor<T>(condition, timeout)` | Wait for condition (default 5s) |
| `CreateFreshDriver()` | Create new driver instance |

From `EnvironmentManager.Instance`:

| Member | Description |
|--------|-------------|
| `CreateDriverInstance()` | Create additional driver |
| `CreateDriverInstance(options)` | Create driver with custom options |
| `Browser` | Current browser enum value |

## Test Organization

```
dotnet/test/
├── common/ # Cross-browser tests
│ ├── DriverTestFixture.cs # Base class
│ ├── CustomTestAttributes/ # Custom NUnit attributes
│ └── *Test.cs # Test files
└── support/ # Support library tests
```

## Build Files

* Adding tests shouldn't require Bazel changes—tests are picked up via glob.
* Make sure `*Test.cs` files are in a directory with `dotnet_nunit_test_suite` in BUILD.bazel.
Loading