Skip to content

Commit b8d2982

Browse files
tompngmrkn
authored andcommitted
Fix precision of bigdecimal.power(integer, prec)
Fixes this lack of precision bug BigDecimal('0.1234').power(8, 12) #=> 0.53768e-7 BigDecimal('12.34').power(-8, 12) #=> 0.186e-8
1 parent bf22f51 commit b8d2982

2 files changed

Lines changed: 5 additions & 2 deletions

File tree

ext/bigdecimal/bigdecimal.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3150,7 +3150,7 @@ BigDecimal_power(int argc, VALUE*argv, VALUE self)
31503150
}
31513151
VpPowerByInt(y, x, int_exp);
31523152
if (!NIL_P(prec) && VpIsDef(y)) {
3153-
VpMidRound(y, VpGetRoundMode(), n);
3153+
VpMidRound(y, VpGetRoundMode(), n - VpExponent10(y));
31543154
}
31553155
return VpCheckGetValue(y);
31563156
}

test/bigdecimal/test_bigdecimal.rb

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1709,7 +1709,10 @@ def test_power_with_prec
17091709
assert_equal(pow, pi.power(e, 20))
17101710

17111711
b = BigDecimal('1.034482758620689655172413793103448275862068965517241379310344827586206896551724')
1712-
assert_equal(BigDecimal('0.114523E1'), b.power(4, 5), '[Bug #8818] [ruby-core:56802]')
1712+
assert_equal(BigDecimal('0.11452E1'), b.power(4, 5), '[Bug #8818] [ruby-core:56802]')
1713+
1714+
assert_equal(BigDecimal('0.537676775108E-7'), BigDecimal('0.1234').power(8, 12))
1715+
assert_equal(BigDecimal('0.537676775108E9'), BigDecimal('12.34').power(8, 12))
17131716
end
17141717

17151718
def test_limit

0 commit comments

Comments
 (0)