From b97a0df0f3234b4c1140ba1418d4b96a592afa4a Mon Sep 17 00:00:00 2001 From: Winfried Donkers Date: Sat, 30 Dec 2017 18:25:02 +0100 Subject: tdf#96821 fix corner cases for Calc function ROUND. For very large integer numbers, e.g. 2^52+1, ROUND produced incorrect results because rtl::math::round uses floor( number + 0.5 + small_correction_value ), which reduces the maximum possible mantissa resolution.a Correction of several unit test documents (Calc functions) with rounding errors that only became apparent with this fix. Change-Id: I1769c9939a3d6118d3bfbfdf8e41dd4619997232 Reviewed-on: https://gerrit.libreoffice.org/47179 Tested-by: Jenkins Reviewed-by: Eike Rathke --- sal/rtl/math.cxx | 3 +++ 1 file changed, 3 insertions(+) (limited to 'sal') diff --git a/sal/rtl/math.cxx b/sal/rtl/math.cxx index 4b149b9f99dd..387ae3b2b4aa 100644 --- a/sal/rtl/math.cxx +++ b/sal/rtl/math.cxx @@ -1076,6 +1076,9 @@ double SAL_CALL rtl_math_round(double fValue, int nDecPlaces, if (fValue == 0.0) return fValue; + if ( nDecPlaces == 0 && eMode == rtl_math_RoundingMode_Corrected ) + return std::round( fValue ); + // sign adjustment bool bSign = rtl::math::isSignBitSet( fValue ); if (bSign) -- cgit