[Bug]: Managed BQ read transform fails to attach config row

[chamikaramj]

What happened?

I'm trying out a Managed BQ read pipeline and seems like it's failing to attach the config row preventing Dataflow upgrade.

WARNING: Could not attach the config row for transform BigQueryIO.TypedRead: java.lang.ClassCastException: class [B cannot be cast to class java.lang.String ([B and java.lang.String are in module java.base of loader 'bootstrap')

Pipeline is given below.

    String table =
        String.format(
            "%s.%s.%s",
            options.as(GcpOptions.class).getProject(), BIG_QUERY_DATASET_ID, options.getTable());
    Pipeline pipeline = Pipeline.create(options);
    Map<String, Object> readConfig =
        ImmutableMap.of("query", String.format("SELECT * FROM `%s`", table));
    // read and validate
    PCollection<Row> outputRows =
        pipeline
            .apply(Managed.read(Managed.BIGQUERY).withConfig(readConfig))
            .getSinglePCollection();
    pipeline.run().waitUntilFinish();

Ahmed, any idea what's going on ?

Issue Priority

Priority: 1 (data loss / total loss of function)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[chamikaramj]

It's good if we can get this bug fixed in the 2.63.0 release.

[chamikaramj]

cc: @jrmccluskey

[jrmccluskey]

Noted. Do you have an idea of how long this will take to resolve? If it's < 1 day I can hold the branch cut until it's fixed, otherwise we can aim for a cherrypick

[chamikaramj]

Thanks! It should be fine to cut the branch and cherrypick.

[chamikaramj]

We determined that this warning and the Dataflow upgrade issue are unrelated. So we can just move this out of the release blocking queue.

Thanks @jrmccluskey