Related: tdf#130725: use strtod also in rtl::math::stringToDouble

Size of buffer on stack is 256 characters. Logging function usage
in make check, of >1 100 000 invocations, the longest string was
80 characters, average being 4.6 characters. So heap allocation
is unlikely in scenarios with intensive function usage.

Several existing unit tests had to be fixed. Usually, the change
is either minimal or getting closer to what Excel returns (for
Calc tests). But in case of AMORDEGRC, I had to change rate value
passed to the function from 0.3 to 0.31. It's because the closest
double value for 0.3 is 0.29999999999999999, which is a bit less
than 0.3; multiplied by 1.5, this gives 0.44999999999999996, and
then rounding the result of multiplication of the latter by cost
gave the result 1 less than before, when 0.3 was imported as
0.30000000000000004. Now the function returns a value 1 less than
Excel for that set of arguments. I don't see how to fix that.
Having rate slightly different gives consistent result between
Calc and Excel.

Change-Id: Icae5ce374fe0c31a1aa10cee815e65ef0014f382
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/89422
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>

1 files changed, 18 insertions, 18 deletions

diff --git a/sc/qa/unit/data/functions/array/fods/linest.fods b/sc/qa/unit/data/functions/array/fods/linest.fods
index 1465f7b85e13..fbf27b2053ca 100644
--- a/sc/qa/unit/data/functions/array/fods/linest.fods
+++ b/sc/qa/unit/data/functions/array/fods/linest.fods
@@ -6674,11 +6674,11 @@
      <table:table-cell table:number-columns-repeated="13"/>
     </table:table-row>
     <table:table-row table:style-name="ro2">
-     <table:table-cell table:formula="of:=[.M185]" office:value-type="float" office:value="2484916.49382742" calcext:value-type="float">
-      <text:p>2484916.49382742</text:p>
+     <table:table-cell table:formula="of:=[.M185]" office:value-type="float" office:value="2484916.49382745" calcext:value-type="float">
+      <text:p>2484916.49382745</text:p>
      </table:table-cell>
-     <table:table-cell office:value-type="float" office:value="2484916.49382742" calcext:value-type="float">
-      <text:p>2484916.49382742</text:p>
+     <table:table-cell office:value-type="float" office:value="2484916.49382745" calcext:value-type="float">
+      <text:p>2484916.49382745</text:p>
      </table:table-cell>
      <table:table-cell table:style-name="ce15" table:formula="of:=ROUND([.A171];12)=ROUND([.B171];12)" office:value-type="boolean" office:boolean-value="true" calcext:value-type="boolean">
       <text:p>TRUE</text:p>
@@ -7004,8 +7004,8 @@
       <text:p>=M195</text:p>
      </table:table-cell>
      <table:table-cell table:number-columns-repeated="8"/>
-     <table:table-cell office:value-type="float" office:value="2484916.49382742" calcext:value-type="float">
-      <text:p>2484916.49382742</text:p>
+     <table:table-cell office:value-type="float" office:value="2484916.49382745" calcext:value-type="float">
+      <text:p>2484916.49382745</text:p>
      </table:table-cell>
      <table:table-cell office:value-type="float" office:value="1" calcext:value-type="float">
       <text:p>1</text:p>
@@ -7352,11 +7352,11 @@
      <table:table-cell table:number-columns-repeated="12"/>
     </table:table-row>
     <table:table-row table:style-name="ro2">
-     <table:table-cell table:formula="of:=[.M208]" office:value-type="float" office:value="786024.50000001" calcext:value-type="float">
-      <text:p>786024.50000001</text:p>
+     <table:table-cell table:formula="of:=[.M208]" office:value-type="float" office:value="786024.500000118" calcext:value-type="float">
+      <text:p>786024.500000118</text:p>
      </table:table-cell>
-     <table:table-cell office:value-type="float" office:value="786024.50000001" calcext:value-type="float">
-      <text:p>786024.50000001</text:p>
+     <table:table-cell office:value-type="float" office:value="786024.500000118" calcext:value-type="float">
+      <text:p>786024.500000118</text:p>
      </table:table-cell>
      <table:table-cell table:style-name="ce15" table:formula="of:=ROUND([.A201];12)=ROUND([.B201];12)" office:value-type="boolean" office:boolean-value="true" calcext:value-type="boolean">
       <text:p>TRUE</text:p>
@@ -7533,8 +7533,8 @@
       <text:p>=O205</text:p>
      </table:table-cell>
      <table:table-cell table:number-columns-repeated="8"/>
-     <table:table-cell office:value-type="float" office:value="786024.50000001" calcext:value-type="float">
-      <text:p>786024.50000001</text:p>
+     <table:table-cell office:value-type="float" office:value="786024.500000118" calcext:value-type="float">
+      <text:p>786024.500000118</text:p>
      </table:table-cell>
      <table:table-cell office:value-type="float" office:value="1" calcext:value-type="float">
       <text:p>1</text:p>
@@ -7701,11 +7701,11 @@
      <table:table-cell table:number-columns-repeated="12"/>
     </table:table-row>
     <table:table-row table:style-name="ro2">
-     <table:table-cell table:formula="of:=[.M219]" office:value-type="float" office:value="786024.50000001" calcext:value-type="float">
-      <text:p>786024.50000001</text:p>
+     <table:table-cell table:formula="of:=[.M219]" office:value-type="float" office:value="786024.500000118" calcext:value-type="float">
+      <text:p>786024.500000118</text:p>
      </table:table-cell>
-     <table:table-cell office:value-type="float" office:value="786024.50000001" calcext:value-type="float">
-      <text:p>786024.50000001</text:p>
+     <table:table-cell office:value-type="float" office:value="786024.500000118" calcext:value-type="float">
+      <text:p>786024.500000118</text:p>
      </table:table-cell>
      <table:table-cell table:style-name="ce15" table:formula="of:=ROUND([.A216];12)=ROUND([.B216];12)" office:value-type="boolean" office:boolean-value="true" calcext:value-type="boolean">
       <text:p>TRUE</text:p>
@@ -7789,8 +7789,8 @@
       <text:p>=N217</text:p>
      </table:table-cell>
      <table:table-cell table:number-columns-repeated="8"/>
-     <table:table-cell office:value-type="float" office:value="786024.50000001" calcext:value-type="float">
-      <text:p>786024.50000001</text:p>
+     <table:table-cell office:value-type="float" office:value="786024.500000118" calcext:value-type="float">
+      <text:p>786024.500000118</text:p>
      </table:table-cell>
      <table:table-cell office:value-type="float" office:value="1" calcext:value-type="float">
       <text:p>1</text:p>