Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Transform batch norm pattern #85

Open
doru1004 opened this issue Apr 14, 2020 · 0 comments
Open

Transform batch norm pattern #85

doru1004 opened this issue Apr 14, 2020 · 0 comments
Assignees
@doru1004
Labels
optimizations optimizations
Milestone
Networks Optimize...

Comments

@doru1004
Copy link
Collaborator

Batch norm is defined in ONNX as

Y = (X - mean) * scale / Sqrt(var+eps) + beta

when mean and var (which are input variable in ONNX) are seen as constant, and scale & beta are parameters, so always constant, we can precompute

a = scale / Sqrt(var+eps)
b = - mean * scale / Sqrt(var+eps) + beta

and then the operation reduces to

Y = a * X + b
@doru1004 doru1004 self-assigned this Apr 14, 2020
@AlexandreEichenberger AlexandreEichenberger added the optimizations optimizations label Jun 25, 2020
cjvolzka added a commit to cjvolzka/onnx-mlir that referenced this issue Oct 30, 2023
@cjvolzka
Merge pull request onnx#85 from C-P2PN897/merge-main-fb9915e
8472704 
Merge onnx/onnx-mlir fb9915e into zosdev/onnx-mlir metis
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@doru1004 doru1004

Labels
optimizations optimizations
Projects
None yet
Milestone
Networks Optimized for CPU execution
Development

No branches or pull requests
2 participants
@doru1004 @AlexandreEichenberger