Skip to content

Integration tests using capybara (with selenium chromedriver) seem flaky #53

@laugmanuel

Description

@laugmanuel

It seems like integration tests using capybara are flaky and fail regularly if not run sequentially.

Currently, I suspect the problem in Github Actions environment and/or the fact, that tests run in parallel. When restarting the failed job one after the other, they complete successfully.

When running the tests with DEBUG_JS_TEST=1, I see the following erorr:

[MinitestRetry] retry 'test_0001_the index page works' count: 1,  msg: Selenium::WebDriver::Error::SessionNotCreatedError: session not created: Chrome failed to start: exited normally.
  (session not created: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /opt/google/chrome/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
    test/integration_test_helper.rb:302:in `login_admin'
    test/test_helper.rb:84:in `before_setup'
    test/integration_test_helper.rb:331:in `before_setup'

I found the following issue in selenium: SeleniumHQ/selenium#12841
However, the recommended fix (adding -remote-debugging-pipe) did not help for me.

I noticed the problem here: dm-drogeriemarkt/foreman_dlm#67
A quick look into foreman_ansible also showed Actions runs showing the same behaviour:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions