Skip to content

[bug] KVStorageManager._merge_tensors_to_tensordict cannot merge NPU tensors to a nested tensor #105

@dpj135

Description

@dpj135

Description

NPU tensors will be NonTensorStack! Because torch.nested.as_nested_tensor() does not support NPU tensors.

  try:
      return field, torch.nested.as_nested_tensor(chunk, layout=torch.jagged)
  except (RuntimeError, TypeError):
      try:
          return field, torch.nested.as_nested_tensor(chunk, layout=torch.strided)
      except (RuntimeError, TypeError):
          return field, NonTensorStack(*chunk)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions