diff options
author | Mike Kaganski <mike.kaganski@collabora.com> | 2020-02-25 13:54:12 +0300 |
---|---|---|
committer | Mike Kaganski <mike.kaganski@collabora.com> | 2020-02-27 11:02:16 +0100 |
commit | 1782810f886acd26db211d8fdd7ae8796d203c57 (patch) | |
tree | 2bd0421114294e2f93661cb2927e03310c826358 /sal/qa | |
parent | f3e7004794eded346d98264d3061f4e4aa80ee0a (diff) |
Related: tdf#130725: use strtod also in rtl::math::stringToDouble
Size of buffer on stack is 256 characters. Logging function usage
in make check, of >1 100 000 invocations, the longest string was
80 characters, average being 4.6 characters. So heap allocation
is unlikely in scenarios with intensive function usage.
Several existing unit tests had to be fixed. Usually, the change
is either minimal or getting closer to what Excel returns (for
Calc tests). But in case of AMORDEGRC, I had to change rate value
passed to the function from 0.3 to 0.31. It's because the closest
double value for 0.3 is 0.29999999999999999, which is a bit less
than 0.3; multiplied by 1.5, this gives 0.44999999999999996, and
then rounding the result of multiplication of the latter by cost
gave the result 1 less than before, when 0.3 was imported as
0.30000000000000004. Now the function returns a value 1 less than
Excel for that set of arguments. I don't see how to fix that.
Having rate slightly different gives consistent result between
Calc and Excel.
Change-Id: Icae5ce374fe0c31a1aa10cee815e65ef0014f382
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/89422
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
Diffstat (limited to 'sal/qa')
-rw-r--r-- | sal/qa/rtl/math/test-rtl-math.cxx | 31 |
1 files changed, 31 insertions, 0 deletions
diff --git a/sal/qa/rtl/math/test-rtl-math.cxx b/sal/qa/rtl/math/test-rtl-math.cxx index 5038b45a13c8..5dab92b81ba8 100644 --- a/sal/qa/rtl/math/test-rtl-math.cxx +++ b/sal/qa/rtl/math/test-rtl-math.cxx @@ -149,6 +149,37 @@ public: CPPUNIT_ASSERT_EQUAL(rtl_math_ConversionStatus_Ok, status); CPPUNIT_ASSERT_EQUAL(sal_Int32(5), end); CPPUNIT_ASSERT_EQUAL(1234.0, res); + + // Check that the value is the nearest double-precision representation of the decimal 0.0042 + // (it was 0.0042000000000000006 instead of 0.0041999999999999997) + res = rtl::math::stringToDouble(OUString("0,0042"), ',', ' ', &status, &end); + CPPUNIT_ASSERT_EQUAL(rtl_math_ConversionStatus_Ok, status); + CPPUNIT_ASSERT_EQUAL(0.0042, res); + + // "- 1" is nothing + res = rtl::math::stringToDouble(OUString("- 1"), '.', ',', &status, &end); + CPPUNIT_ASSERT_EQUAL(rtl_math_ConversionStatus_Ok, status); + CPPUNIT_ASSERT_EQUAL(sal_Int32(0), end); + CPPUNIT_ASSERT_EQUAL(0.0, res); + + // "-1E+E" : no exponent + res = rtl::math::stringToDouble(OUString("-1E+E"), '.', ',', &status, &end); + CPPUNIT_ASSERT_EQUAL(rtl_math_ConversionStatus_Ok, status); + CPPUNIT_ASSERT_EQUAL(sal_Int32(2), end); + CPPUNIT_ASSERT_EQUAL(-1.0, res); + + // "-0" is negative zero + res = rtl::math::stringToDouble(OUString("-0"), '.', ',', &status, &end); + CPPUNIT_ASSERT_EQUAL(rtl_math_ConversionStatus_Ok, status); + CPPUNIT_ASSERT_EQUAL(sal_Int32(2), end); + CPPUNIT_ASSERT_EQUAL(0.0, res); + CPPUNIT_ASSERT(rtl::math::isSignBitSet(res)); + + // Compensating: "0.001E311" is 1E308, not overflow/inf + res = rtl::math::stringToDouble(OUString("0.001E311"), '.', ',', &status, &end); + CPPUNIT_ASSERT_EQUAL(rtl_math_ConversionStatus_Ok, status); + CPPUNIT_ASSERT_EQUAL(sal_Int32(9), end); + CPPUNIT_ASSERT_EQUAL(1E308, res); } void test_stringToDouble_bad() { |