Skip to content

Create new test type for accessibility API testing #57696

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
spectranaut wants to merge 8 commits into
base: master
Choose a base branch
from

Use test result 'PRECONDITION_FAILED' for tests on wrong platform

9637525
Select commit
Loading
Failed to load commit list.
Draft

Create new test type for accessibility API testing #57696

Use test result 'PRECONDITION_FAILED' for tests on wrong platform
9637525
Select commit
Loading
Failed to load commit list.
Community-TC Integration / wpt-chrome-dev-results succeeded Feb 26, 2026 in 5m 2s

Community-TC (pull_request)

Collect results for all tests affected by a pull request in chrome.

Details

View task in Taskcluster | View logs in Taskcluster | View task group in Taskcluster

Task Status

Started: 2026-02-26T01:32:33.872Z
Resolved: 2026-02-26T01:36:42.628Z
Task Execution Time: 4 minutes, 8 seconds, 756 milliseconds
Task Status: completed
Reason Resolved: completed
TaskId: cUo-cAByS2SFeJGoknF54Q
RunId: 0

Artifacts

- public/logs/live_backing.log
- public/logs/live.log
- public/results/checkrun.md
- public/results/wpt_report.json.gz
- public/results/wpt_screenshot.txt.gz

WPT Command: python3 ./wpt run --channel=dev --no-fail-on-unexpected --log-wptreport=../artifacts/wpt_report.json --log-wptscreenshot=../artifacts/wpt_screenshot.txt --affected base_head --log-mach-level=info --log-mach=- -y --no-pause --no-restart-on-unexpected --install-fonts --verify-log-full --no-headless --enable-swiftshader --install-browser --install-webdriver chrome

[taskcluster 2026-02-26 01:32:33.962Z] Task ID: cUo-cAByS2SFeJGoknF54Q
[taskcluster 2026-02-26 01:32:33.962Z] Worker ID: 8379717854562491496
[taskcluster 2026-02-26 01:32:33.962Z] Worker Group: us-central1-c
[taskcluster 2026-02-26 01:32:33.962Z] Worker Node Type: projects/757942385826/machineTypes/n2-standard-4
[taskcluster 2026-02-26 01:32:33.962Z] Worker Pool: proj-wpt/ci
[taskcluster 2026-02-26 01:32:33.962Z] Worker Version: 46.1.0
[taskcluster 2026-02-26 01:32:33.962Z] Public IP: 34.61.19.180
[taskcluster 2026-02-26 01:32:33.962Z] Hostname: proj-wpt-ci-fasfd-kyqeekdybg7cv3ka

b1e71f63927a - Started downloading
66587c81b81a - Started downloading
b1e71f63927a - Downloaded in 0.035 seconds
5b82b858ea7e - Started downloading
08c7e224535e - Started downloading
66587c81b81a - Downloaded in 0.216 seconds
08c7e224535e - Downloaded in 0.044 seconds
32bbaaf98e4c - Started downloading
dd5257950bd4 - Started downloading
dd5257950bd4 - Downloaded in 0.078 seconds
af9ac786cec1 - Started downloading

...(1511 lines hidden)...

node       = <Atspi.Accessible object at 0x7fcce8a900c0 (AtspiAccessible at 0x7fcce0213c10)>
session    = <Session 5a058a5298a6cb02903696dd16b2f8ea>
test_name  = 'aria-haspopup-false'

wai-aria-aam/role/button_with_default_values_for_aria-pressed_and_aria-haspopup_tentative.py:16: AssertionError
  PRECONDITION_FAILED test_axapi[no-attributes]
  PRECONDITION_FAILED test_axapi[aria-pressed]
  PRECONDITION_FAILED test_axapi[aria-pressed-and-aria-haspopup]
  PRECONDITION_FAILED test_axapi[aria-haspopup]
  PRECONDITION_FAILED test_axapi[aria-haspopup-false]
  PRECONDITION_FAILED test_ia2[no-attributes]
  PRECONDITION_FAILED test_ia2[aria-pressed]
  PRECONDITION_FAILED test_ia2[aria-pressed-and-aria-haspopup]
  PRECONDITION_FAILED test_ia2[aria-haspopup]
  PRECONDITION_FAILED test_ia2[aria-haspopup-false]
  PRECONDITION_FAILED test_uia[no-attributes]
  PRECONDITION_FAILED test_uia[aria-pressed]
  PRECONDITION_FAILED test_uia[aria-pressed-and-aria-haspopup]
  PRECONDITION_FAILED test_uia[aria-haspopup]
  PRECONDITION_FAILED test_uia[aria-haspopup-false]
/wai-aria-aam/role/button_with_defined_value_for_aria-pressed_tentative.py
  PRECONDITION_FAILED test_axapi[true]
  PRECONDITION_FAILED test_axapi[false]
  PRECONDITION_FAILED test_ia2[true]
  PRECONDITION_FAILED test_ia2[false]
  PRECONDITION_FAILED test_uia[true]
  PRECONDITION_FAILED test_uia[false]
/wai-aria-aam/role/button_with_non_false_value_for_aria-haspopup_tentative.py
  FAIL test_atspi[true] - AssertionError: assert 'push button' == 'button'
atspi = <support.atspi_wrapper.AtspiWrapper object at 0x7fcce8aaed40>
session = <Session 029a31d8ab721d1bd086d2de850716a4>
inline = <function inline.<locals>.inline at 0x7fcce8b345e0>, test_name = 'true'

    @pytest.mark.parametrize("test_name", TEST_HTML.keys())
    def test_atspi(atspi, session, inline, test_name):
        session.url = inline(TEST_HTML[test_name])
        node = atspi.find_node("test", session.url)
>       assert atspi.Accessible.get_role_name(node) == "button"
E       AssertionError: assert 'push button' == 'button'
E         
E         - button
E         + push button

atspi      = <support.atspi_wrapper.AtspiWrapper object at 0x7fcce8aaed40>
inline     = <function inline.<locals>.inline at 0x7fcce8b345e0>
node       = <Atspi.Accessible object at 0x7fccea566c80 (AtspiAccessible at 0x7fcce01b50c0)>
session    = <Session 029a31d8ab721d1bd086d2de850716a4>
test_name  = 'true'

wai-aria-aam/role/button_with_non_false_value_for_aria-haspopup_tentative.py:16: AssertionError
  FAIL test_atspi[menu] - AssertionError: assert 'push button' == 'button'
atspi = <support.atspi_wrapper.AtspiWrapper object at 0x7fcce8a025c0>
session = <Session 029a31d8ab721d1bd086d2de850716a4>
inline = <function inline.<locals>.inline at 0x7fccea5d1fc0>, test_name = 'menu'

    @pytest.mark.parametrize("test_name", TEST_HTML.keys())
    def test_atspi(atspi, session, inline, test_name):
        session.url = inline(TEST_HTML[test_name])
        node = atspi.find_node("test", session.url)
>       assert atspi.Accessible.get_role_name(node) == "button"
E       AssertionError: assert 'push button' == 'button'
E         
E         - button
E         + push button

atspi      = <support.atspi_wrapper.AtspiWrapper object at 0x7fcce8a025c0>
inline     = <function inline.<locals>.inline at 0x7fccea5d1fc0>
node       = <Atspi.Accessible object at 0x7fccea566a00 (AtspiAccessible at 0x7fcce0213b90)>
session    = <Session 029a31d8ab721d1bd086d2de850716a4>
test_name  = 'menu'

wai-aria-aam/role/button_with_non_false_value_for_aria-haspopup_tentative.py:16: AssertionError
  FAIL test_atspi[listbox] - AssertionError: assert 'push button' == 'button'
atspi = <support.atspi_wrapper.AtspiWrapper object at 0x7fcce8a8b490>
session = <Session 029a31d8ab721d1bd086d2de850716a4>
inline = <function inline.<locals>.inline at 0x7fccea736320>
test_name = 'listbox'

    @pytest.mark.parametrize("test_name", TEST_HTML.keys())
    def test_atspi(atspi, session, inline, test_name):
        session.url = inline(TEST_HTML[test_name])
        node = atspi.find_node("test", session.url)
>       assert atspi.Accessible.get_role_name(node) == "button"
E       AssertionError: assert 'push button' == 'button'
E         
E         - button
E         + push button

atspi      = <support.atspi_wrapper.AtspiWrapper object at 0x7fcce8a8b490>
inline     = <function inline.<locals>.inline at 0x7fccea736320>
node       = <Atspi.Accessible object at 0x7fccea565f80 (AtspiAccessible at 0x7fcce02226d0)>
session    = <Session 029a31d8ab721d1bd086d2de850716a4>
test_name  = 'listbox'

wai-aria-aam/role/button_with_non_false_value_for_aria-haspopup_tentative.py:16: AssertionError
  FAIL test_atspi[tree] - AssertionError: assert 'push button' == 'button'
atspi = <support.atspi_wrapper.AtspiWrapper object at 0x7fcce8a483a0>
session = <Session 029a31d8ab721d1bd086d2de850716a4>
inline = <function inline.<locals>.inline at 0x7fcce8a81360>, test_name = 'tree'

    @pytest.mark.parametrize("test_name", TEST_HTML.keys())
    def test_atspi(atspi, session, inline, test_name):
        session.url = inline(TEST_HTML[test_name])
        node = atspi.find_node("test", session.url)
>       assert atspi.Accessible.get_role_name(node) == "button"
E       AssertionError: assert 'push button' == 'button'
E         
E         - button
E         + push button

atspi      = <support.atspi_wrapper.AtspiWrapper object at 0x7fcce8a483a0>
inline     = <function inline.<locals>.inline at 0x7fcce8a81360>
node       = <Atspi.Accessible object at 0x7fcce8a09c80 (AtspiAccessible at 0x7fcce0213d10)>
session    = <Session 029a31d8ab721d1bd086d2de850716a4>
test_name  = 'tree'

wai-aria-aam/role/button_with_non_false_value_for_aria-haspopup_tentative.py:16: AssertionError
  FAIL test_atspi[grid] - AssertionError: assert 'push button' == 'button'
atspi = <support.atspi_wrapper.AtspiWrapper object at 0x7fcce8a03040>
session = <Session 029a31d8ab721d1bd086d2de850716a4>
inline = <function inline.<locals>.inline at 0x7fcce8a829e0>, test_name = 'grid'

    @pytest.mark.parametrize("test_name", TEST_HTML.keys())
    def test_atspi(atspi, session, inline, test_name):
        session.url = inline(TEST_HTML[test_name])
        node = atspi.find_node("test", session.url)
>       assert atspi.Accessible.get_role_name(node) == "button"
E       AssertionError: assert 'push button' == 'button'
E         
E         - button
E         + push button

atspi      = <support.atspi_wrapper.AtspiWrapper object at 0x7fcce8a03040>
inline     = <function inline.<locals>.inline at 0x7fcce8a829e0>
node       = <Atspi.Accessible object at 0x7fcce8ab32c0 (AtspiAccessible at 0x7fcce0213e10)>
session    = <Session 029a31d8ab721d1bd086d2de850716a4>
test_name  = 'grid'

wai-aria-aam/role/button_with_non_false_value_for_aria-haspopup_tentative.py:16: AssertionError
  FAIL test_atspi[dialog] - AssertionError: assert 'push button' == 'button'
atspi = <support.atspi_wrapper.AtspiWrapper object at 0x7fcce8a8b220>
session = <Session 029a31d8ab721d1bd086d2de850716a4>
inline = <function inline.<locals>.inline at 0x7fcce8a82560>
test_name = 'dialog'

    @pytest.mark.parametrize("test_name", TEST_HTML.keys())
    def test_atspi(atspi, session, inline, test_name):
        session.url = inline(TEST_HTML[test_name])
        node = atspi.find_node("test", session.url)
>       assert atspi.Accessible.get_role_name(node) == "button"
E       AssertionError: assert 'push button' == 'button'
E         
E         - button
E         + push button

atspi      = <support.atspi_wrapper.AtspiWrapper object at 0x7fcce8a8b220>
inline     = <function inline.<locals>.inline at 0x7fcce8a82560>
node       = <Atspi.Accessible object at 0x7fcce8a092c0 (AtspiAccessible at 0x7fcce0213f10)>
session    = <Session 029a31d8ab721d1bd086d2de850716a4>
test_name  = 'dialog'

wai-aria-aam/role/button_with_non_false_value_for_aria-haspopup_tentative.py:16: AssertionError
  PRECONDITION_FAILED test_axapi[true]
  PRECONDITION_FAILED test_axapi[menu]
  PRECONDITION_FAILED test_axapi[listbox]
  PRECONDITION_FAILED test_axapi[tree]
  PRECONDITION_FAILED test_axapi[grid]
  PRECONDITION_FAILED test_axapi[dialog]
  PRECONDITION_FAILED test_ia2[true]
  PRECONDITION_FAILED test_ia2[menu]
  PRECONDITION_FAILED test_ia2[listbox]
  PRECONDITION_FAILED test_ia2[tree]
  PRECONDITION_FAILED test_ia2[grid]
  PRECONDITION_FAILED test_ia2[dialog]
  PRECONDITION_FAILED test_uia[true]
  PRECONDITION_FAILED test_uia[menu]
  PRECONDITION_FAILED test_uia[listbox]
  PRECONDITION_FAILED test_uia[tree]
  PRECONDITION_FAILED test_uia[grid]
  PRECONDITION_FAILED test_uia[dialog]
 1:47.98 INFO Got 7 unexpected results, with 0 unexpected passes
 1:47.98 wptserve INFO Stopped http server on 127.0.0.1:8001
 1:47.98 wptserve INFO Stopped http server on 127.0.0.1:8003
 1:47.98 wptserve INFO Stopped http server on 127.0.0.1:8443
 1:48.00 wptserve INFO Stopped http server on 127.0.0.1:8002
 1:48.00 wptserve INFO Stopped http server on 127.0.0.1:8446
 1:48.00 wptserve INFO Stopped http server on 127.0.0.1:8445
 1:48.00 wptserve INFO Stopped http server on 127.0.0.1:8000
 1:48.00 wptserve INFO Stopped http server on 127.0.0.1:8444
 1:48.00 wptserve INFO Stopped http server on 127.0.0.1:9000
 1:48.00 wptserve INFO Stopped WebTransport over HTTP/3 server on 127.0.0.1:11000
 1:48.15 wptserve INFO Close on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8888))
 1:48.15 wptserve INFO Close on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8889))
 1:48.19 INFO Removed font: Ahem.ttf
 1:48.21 INFO Closing logging queue
 1:48.21 INFO queue closed
 1:48.22 INFO Tolerating 7 unexpected results
[taskcluster 2026-02-26 01:36:40.356Z] === Task Finished ===
[taskcluster 2026-02-26 01:36:41.728Z] Successful task run with exit code: 0 completed in 247.768 seconds
View more details on Community-TC Integration