Skip to content

Fix: prevent variance underflow causing NaN losses#537

Open
Kapadiaaa wants to merge 1 commit intomllam:mainfrom
Kapadiaaa:fix-variance-bug
Open

Fix: prevent variance underflow causing NaN losses#537
Kapadiaaa wants to merge 1 commit intomllam:mainfrom
Kapadiaaa:fix-variance-bug

Conversation

@Kapadiaaa
Copy link
Copy Markdown

Problem
Softplus activation can produce zero values for very negative inputs, leading to division-by-zero and NaN losses during training.

Solution
Added a clamp (min=1e-6) to the predicted standard deviation to ensure numerical stability.

Impact

  • Prevents NaN crashes
  • Ensures stable probabilistic training

@kshirajahere
Copy link
Copy Markdown
Contributor

Hey isnt there a draft PR #527 which resolves this. Given that overlap, I’d suggest closing this in favor of #527, unless there’s some unique piece here that should be folded into that branch.

@Kapadiaaa
Copy link
Copy Markdown
Author

Thanks for pointing that out. I will review PR #527 and see if there’s anything additional I can contribute or improve there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants