Skip to content

Fix Avro 1.11 test compile breaking publish; bump to 2.45.62 - #188

Merged
cirquare merged 1 commit into
li_trunkfrom
yahao/fix-avro-test-publish
Oct 14, 2026
Merged

cirquare merged 1 commit into
li_trunkfrom
yahao/fix-avro-test-publish

Conversation

@yananhao12

Copy link
Copy Markdown

Problem

The post-merge LI JFrog Artifactory Publish run for 2.45.61 (PR #180), run 37264486966, failed:

sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryAvroUtilsTest.java:147: error: defaultValue() is not public in Field
Execution failed for task ':sdks:java:io:google-cloud-platform:compileTestJava'

PR #180 migrated Avro 1.8.2 -> 1.11.4. In Avro 1.11, Schema.Field#defaultValue() (returns JsonNode) is package-private; public Object defaultVal() replaces it. The publish builds test jars (testJar artifact in BeamModulePlugin.groovy), so test sources must compile. The publish command runs without --continue, so io-google-cloud-platform and the beam-auto-elr umbrella were never published. That left a partial 2.45.61 on JFrog (core, model-pipeline and runners-flink-1.18 are present) and stalled auto-ELR.

Fix (3 files, 4 lines)

  • BigQueryAvroUtilsTest.java: field.defaultValue() -> field.defaultVal(). Upstream made the identical change in update confluent dependency version to 7.6.0 apache/beam#30638.
  • Bump to 2.45.62 with ./update_version.sh 2.45.62 (gradle.properties version/sdk_version, and BeamModulePlugin.groovy project.version). We use a fresh version so the publish does not collide with the partial 2.45.61. Changing BeamModulePlugin.groovy also re-triggers the publish workflow on merge.

No dependency, workflow or auto-elr module-list changes.

Validation

All runs were local on JDK 8 (Zulu 1.8.0_504, matching the workflow's JDK 8), from a clean tree with --no-build-cache.

  • ./gradlew publishMavenJavaPublicationToMavenLocal -PisRelease -PnoSigning -PisLinkedin --continue --no-daemon --no-build-cache -> BUILD SUCCESSFUL in 17m 4s (1052 tasks, 0 failed). It used --continue so that any further Avro 1.11 breakages would surface; none did.
    • beam-auto-elr:2.45.62 was published, and its POM references all 22 modules from auto-elr/build.gradle at 2.45.62. Each of those modules has a 2.45.62 POM, jar and -tests.jar (85 com.linkedin.beam artifacts in total).
    • The remaining error: cannot find symbol lines (Watch.java:174, CoGroup.java:645, JdbcIO, ParquetIO, …) are javadoc-only noise and do not fail the build. They also appear in the CI log.
  • ./gradlew :sdks:java:io:google-cloud-platform:test --tests '*BigQueryAvroUtilsTest*' -> 6/6 passed, including testConvertGenericRecordToTableRow, which exercises defaultVal().

Known follow-up (not in scope)

Publishing compiles tests but does not run them. Upstream apache#30638 also adjusted these tests for Avro 1.11 behavior, so they likely fail when tests are run. They were not run here:

  • PubsubClientTest:208: the expected exception changes from NullPointerException to SchemaParseException.
  • GcpApiSurfaceTest:125: drop classesInPackage("org.codehaus.jackson").

A separate follow-up should also add a compileTestJava PR gate so this class of failure is caught before merge.


  • Mention the appropriate issue in your description, if applicable: no issue filed; this fixes the post-merge publish of Clear all 48 oss-canary CVEs blocking ELR and bump to 2.45.61 #180.
  • Update CHANGES.md with noteworthy changes: N/A, this is a LinkedIn-fork build fix and version bump with no user-facing change.
  • If this contribution is large, please file an Apache ICLA: N/A, the change is small and LinkedIn-internal.

Build health: the post-merge publish result will show under the LI JFrog Artifactory Publish workflow on li_trunk.

The 2.45.61 post-merge publish (run 37264486966) failed at
:sdks:java:io:google-cloud-platform:compileTestJava because Avro 1.11
made Schema.Field#defaultValue() package-private. Publishing builds test
jars, so the failure blocked publication of io-google-cloud-platform and
the beam-auto-elr umbrella, leaving a partial 2.45.61 on JFrog and
stalling auto-ELR. Use the public defaultVal() (same as upstream
apache#30638) and bump to 2.45.62 so the publish re-triggers on a
clean version.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cirquare
cirquare merged commit a8b0e49 into li_trunk Oct 14, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants