Skip to content

CPU matmul silently returns incorrect values for stride-zero broadcast batch #3744

Description

@tomsanbear

Bug

On Candle 0.11.0, the baseline CPU backend silently returns incorrect values when batched matmul receives a stride-zero batch dimension created by broadcast_as. The Accelerate CPU provider returns the correct values for the same tensors.

This is related to #3253, but that report covers broadcast_matmul rejecting a stride-zero layout. This report concerns direct broadcast_as(...).matmul(...) succeeding with incorrect output.

Minimal reproducer

use candle_core::{DType, Device, Result, Tensor};

fn main() -> Result<()> {
    let device = Device::Cpu;
    let left = Tensor::ones((1, 32, 32), DType::F32, &device)?;
    let right = Tensor::ones((32, 32, 32), DType::F32, &device)?;

    let stride_zero = left.broadcast_as((32, 32, 32))?.matmul(&right)?;
    let eager = left.repeat((32, 1, 1))?.matmul(&right)?;

    assert_eq!(
        stride_zero.flatten_all()?.to_vec1::<f32>()?,
        eager.flatten_all()?.to_vec1::<f32>()?,
    );
    Ok(())
}

With baseline candle-core = "0.11.0", the assertion fails. With candle-core = { version = "0.11.0", features = ["accelerate"] } on the same Apple Silicon host, it passes.

Expected behavior

matmul should either honor the stride-zero batch layout and match the eagerly expanded input, or reject the unsupported layout. It should not succeed with incorrect values.

Environment

  • Candle 0.11.0
  • Rust 1.94.1, edition 2024
  • macOS arm64 / Apple M4 Max
  • Baseline CPU provider: incorrect values
  • Accelerate CPU provider: correct values

I can prepare a PR once the desired baseline CPU behavior and regression-test location are confirmed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions