Skip to content

Conversation

@scwhittle
Copy link
Contributor

@scwhittle scwhittle commented Nov 13, 2025

V1 runner supports all state types, enabling MapState, SetState, MultimapState, OrderedListState. Batch v1 was failing with enabling multimap so just disabling for now since I just care about streaming.

V2 runner supports OrderedListState


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

@scwhittle
Copy link
Contributor Author

I believe that this is all supported so removing exclusions and triggering validates runner runs to check.

@scwhittle
Copy link
Contributor Author

Run Java PreCommit

@scwhittle scwhittle changed the title Enable MapState, SetState, MultimapState validates runner tests for Dataflow. Enable validates runner tests for advanced state types for Dataflow. Nov 17, 2025
@scwhittle
Copy link
Contributor Author

Validates runner tests passed on previous run but had to merge trigger file change.

@scwhittle scwhittle marked this pull request as ready for review November 17, 2025 13:22
@github-actions
Copy link
Contributor

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

@scwhittle scwhittle force-pushed the enable_set_and_map_tests branch from 1bfc732 to fbaedc3 Compare November 19, 2025 10:03
@scwhittle scwhittle force-pushed the enable_set_and_map_tests branch from fbaedc3 to cd93d5e Compare November 19, 2025 10:06
@scwhittle
Copy link
Contributor Author

R: @kennknowles I think the local environment tests are unrelated but I don't know how to retrigger them without running the full verify tests again

@github-actions
Copy link
Contributor

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

@kennknowles
Copy link
Member

Since you are a committer, you should have this access. When you click through to the GHA results page, the rerun button is like so:
image

@kennknowles
Copy link
Member

kennknowles commented Nov 20, 2025

I don't know how this error crept in, but it doesn't look like something a rerun will fix:

> Task :runners:local-java:jar
Traceback (most recent call last):

  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 187, in _run_module_as_main
> Task :sdks:python:wordCount FAILED
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 110, in _get_module_details
    __import__(pkg_name)
  File "/Users/runner/work/beam/beam/sdks/python/apache_beam/__init__.py", line 86, in <module>
    from apache_beam import coders
  File "/Users/runner/work/beam/beam/sdks/python/apache_beam/coders/__init__.py", line 17, in <module>
    from apache_beam.coders.coders import *
  File "/Users/runner/work/beam/beam/sdks/python/apache_beam/coders/coders.py", line 60, in <module>
    from apache_beam.coders import coder_impl
  File "/Users/runner/work/beam/beam/sdks/python/apache_beam/coders/coder_impl.py", line 62, in <module>
    from apache_beam.typehints.schemas import named_tuple_from_schema
  File "/Users/runner/work/beam/beam/sdks/python/apache_beam/typehints/__init__.py", line 31, in <module>
    from apache_beam.typehints.pandas_type_compatibility import *
  File "/Users/runner/work/beam/beam/sdks/python/apache_beam/typehints/pandas_type_compatibility.py", line 115, in <module>
    BEAM_TO_PANDAS.update({
  File "/Users/runner/work/beam/beam/sdks/python/apache_beam/typehints/pandas_type_compatibility.py", line 116, in <dictcomp>
    normalize(typehint): pandas_dtype
  File "/Users/runner/work/beam/beam/sdks/python/apache_beam/typehints/typehints.py", line 1468, in normalize
    beam_type = native_type_compatibility.convert_to_beam_type(x)
  File "/Users/runner/work/beam/beam/sdks/python/apache_beam/typehints/native_type_compatibility.py", line 343, in convert_to_beam_type
    if sys.version_info >= (3, 12) and TypeAliasType is not None:
NameError: name 'sys' is not defined

(not related to your PR)

Copy link
Member

@kennknowles kennknowles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I want to try to see if the tests can be green to discourage broken windows syndrome, but this is a clear-cut case.

@scwhittle scwhittle merged commit 3d5d04b into apache:master Nov 21, 2025
20 of 26 checks passed
@scwhittle scwhittle deleted the enable_set_and_map_tests branch November 21, 2025 10:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants