Skip to content

Commit 39538e7

Browse files
committed
docs: 添加 rms normalization 算子定义
Signed-off-by: YdrMaster <[email protected]>
1 parent 88e0712 commit 39538e7

File tree

1 file changed

+27
-0
lines changed

1 file changed

+27
-0
lines changed

src/08-01llm/README.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# 大模型自定义算子
2+
3+
## RMS Normalization
4+
5+
### Summary
6+
7+
```plaintext
8+
___ → →
9+
y = (x^2 + δ)^(-1/2) * w * x
10+
```
11+
12+
### Attributes
13+
14+
- **epsilon - FLOAT** (default is `1e-5`): 防止除 0 异常的小数字 ε。
15+
16+
### Inputs
17+
18+
2 Inputs:
19+
20+
- **X(heterogeneous) - T**: 来自之前算子的输入数据张量。形状为 `N1 x N2 ... D``Nx` 可以为任意维度,将在长度为 `D` 的最后一个维度上标准化。
21+
- **W(heterogeneous) - T**: 权重张量。形状为 `D``D``X` 的最后一个维度的长度。
22+
23+
### Outputs
24+
25+
1 Output:
26+
27+
- **Y(heterogeneous) - T**: 输出张量。形状与 `X` 相同。

0 commit comments

Comments
 (0)