Matrix Multiplication for 1D

[sociopath00]

In the course, we have tensor = torch.tensor([1,2,3]) which has 1D only. In this case how matrix multiplication works. As per the rules it should have same inner dimensions. When we do torch.matmul(tensor, tensor) how the 2 rules of matrix multiplications are applied here?

[AlphaGeek]

I believe that behind the scenes torch just returns the dot product of the two vectors

[lishav123]

[EDIT: Nov 15, 00:00]

---

QUESTION:
In the course, we have tensor = torch.tensor([1,2,3]) which is 1D only.
In this case how does matrix multiplication work? As per the rules, it should have matching inner dimensions.
When we do torch.matmul(tensor, tensor), how do the rules for matrix multiplication apply here?

---

A:
Great question! The confusion comes down to distinguishing between vectors and matrices in PyTorch.

1. Vector vs. Matrix

• tensor = torch.tensor([1, 2, 3])
  This is a 1D tensor (vector), shape: (3,)
• tensor = torch.tensor([[1, 2, 3]])
  This is a 2D tensor (matrix with 1 row, 3 columns), shape: (1, 3)

---

2. Matrix Multiplication Rules

For matrix multiplication (matmul):

• Matrices must have compatible inner dimensions: (a, b) @ (b, c) → (a, c)
• Vectors are treated as row or column depending on context.

Case 1: torch.matmul(tensor, tensor)

• If both are 1D vectors: dot product is computed
  • Output: a scalar

import torch
tensor = torch.tensor([1, 2, 3])
print(tensor.ndim)             # 1
print(torch.matmul(tensor, tensor))  # 14

Output:

1
tensor(14)

Case 2: Multiplying matrices

• If you try to multiply two matrices with shapes (1, 3) @ (1, 3), you'll get an error (inner dimensions don’t match):

import torch
tensor = torch.tensor([[1, 2, 3]])
print(tensor.ndim)             # 2
# print(torch.matmul(tensor, tensor))   # Error!

• To multiply properly, transpose the second matrix:

print(torch.matmul(tensor, tensor.T))  # shape: (1,3) @ (3,1) → (1,1)

Output:

tensor([[14]])

---

3. Summary

• Vector × Vector (1D): returns a scalar (dot product).
• Matrix × Matrix (2D): returns a matrix, dimensions must be compatible.

Example comparing both:

import torch

# Vector case
vec = torch.tensor([1, 2, 3])
print(vec.ndim, torch.matmul(vec, vec))     # 1, 14

# Matrix case
mat = torch.tensor([[1, 2, 3]])
print(mat.ndim, torch.matmul(mat, mat.T))   # 2, tensor([[14]])

---

Conclusion:
Your understanding is correct:

• For 1D tensors, matmul does a dot product (returns a number).
• For 2D matrices, dimensions must align; transposing may be needed.

Hope this clears things up!