Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error in formula using arithmetic operations with float #38

Open
cberdila opened this issue Jan 26, 2017 · 1 comment
Open

Error in formula using arithmetic operations with float #38

cberdila opened this issue Jan 26, 2017 · 1 comment

Comments

@cberdila
Copy link

Calculating 2.3 + 2.4 returns 4.699999999999999 instead of 4.7

About floating-point arithmetic operations go to:
http://floating-point-gui.de/
http://www.w3schools.com/js/js_numbers.asp

Steps to reproduce (on the demo web of calx2 jquery):

  1. go to http://prototype.xsanisty.com/calx2/
  2. introduce 2.3 + 2.4 in the formula textbox
  3. the result shown 4.699999999999999 (the expected result 4.7)

Steps to reproduce (in JavaScript code):
calc.calx ('calculate', "2.3 + 2.4") // returns 4.699999999999999 (instead of 4.7)

I personally use it in a validation and I can not use it with the Excel ROUND function because the formulas are created by the users.
e.g. IF (2.3 + 2.4 = 4.7, "correct", "incorrect") // returns "incorrect" (instead of "correct")

@ikhsan017
Copy link
Member

I think it is the nature way dealing with floating point on most language, the only way to fix this is improve the comparator formula to deal with floating point comparison

https://github.com/xsanisty/jquery-calx/blob/master/jquery-calx-2.2.7.js#L9347-L9371

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants