Age | Commit message (Collapse) | Author |
|
Change-Id: I6eb9f0431a9402479a2d90d5b6f68b611d52a9f9
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/175957
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Change-Id: I90e26030cb7a002bfd76cbc7aa73a5d3ea7a7f1b
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/164132
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Change-Id: I79636ff9c3b2fe601b0b3c94a111b36af0011775
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/164131
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
And use standard functions in the rtl_math_RoundingMode_HalfEven
handler.
Change-Id: If9f29aa63423db9457a02ed003cfc27cf3df5585
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/164104
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Location:
if (nDecPlaces == 0)
{
switch (eMode)
{
case rtl_math_RoundingMode_Corrected:
return std::round(fValue);
Because the functions are the same as the following
related codes, they are respectively:
if (nDecPlaces >= 0
&& (fValue >= 0x1p52
|| isRepresentableInteger(fValue)))
return fOrigValue;
as well as:
if (fValue < 0x1p52)
{
switch(eMode)
{
case rtl_math_RoundingMode_Corrected:
fValue = rtl::math::approxFloor(fValue + 0.5);
break;
:
:
:
Because some double-precision floating point
numbers cause std::round(fValue) and rtl::math::approxFloor(fValue + 0.5)
to produce different results.
For example, enter 10614.4999999999876 in Calc's
A1 cell (or any operation that will produce a long
decimal number such as .499999999999xyz after the
decimal point).
We expected it to be 10614.4999999999876, but it
was automatically rounded to 10614.5 (note: this
result is also the result of ubiquitous handheld
computers, OpenOffice.org and Excel)
Then, entering =ROUND(A1,0) in B1, we will see 10614,
A1 and B1 clearly don't meet expectations. (My boss
or tax collector might be unhappy :-p)
Although A1's 10614.4999999999876 is still faithfully
recorded, the rendering of 10614.5 is confusing.
Now, there are two views:
1. According to the established convention, B2
displays the expected answer 10615.
or
2. True to the laws of mathematics, A1 displays
10614.4999999999876 or at least 10614.49
Although the second point of view is correct (and I
completely agree with it), when opening
spreadsheets generated by other software (such as
OpenOffice.org or Excel), the results will be different,
which people do not like to see.
So when nDecPlaces == 0 is removed, use
std::round(fValue) and let the existing code below it do the rounding:
if (fValue < 0x1p52)
{
switch(eMode)
{
case rtl_math_RoundingMode_Corrected:
fValue = rtl::math::approxFloor(fValue + 0.5);
This is consistent with the first point.
By the way, in nDecPlaces == 0
case rtl_math_RoundingMode_HalfEven:
This piece of code can be checked to see if it is really
necessary, because rtl_math_round() also has the same purpose code.
If it is not needed, the entire nDecPlaces == 0 can be removed.
Co-authored-by: Firefly <firefly@ossii.com.tw>
Co-authored-by: Franklin Weng <franklin@goodhorse.idv.tw>
Change-Id: If32cdb18c70ec0025c83ba25a99e5d135d66aec9
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/159193
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Change-Id: I0e807118e6a2196d2f2858ed195782a90572a2e9
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/158303
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Change-Id: I11a54c1ddf73c16ce46a0d1c375bf43157870db7
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/155856
Tested-by: Jenkins
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
|
|
Change-Id: Idd3e96d99f13b1e87e2a01de9c9392ead0864de7
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/154323
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
This reverts commit 6a4504bba84dcbaeb71869ec5c9ed6dfdc090619, which is no longer
necessary after 6f75ec6115f0152517be634070607bc61bf96dd0 "tdf#148430 Use atanh
from <cmath> instead of our own" changed the implementation of rtl_math_atanh.
Change-Id: I11094f8c9fb0bb7b635541e7360e460816d3c171
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/134161
Tested-by: Jenkins
Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
|
|
Change wrapper in rtl::math::atanh to use atanh
from <cmath>. Also changed all occurrences of rtl::math::atanh
on files that use this function to directly use the standard
atanh instead.
Change-Id: Idc5c456f67291f5816756f023b61afde844b5ceb
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/133965
Tested-by: Jenkins
Reviewed-by: Bartosz Kosiorek <gang65@poczta.onet.pl>
|
|
... to reduce inter-relations between compilation units, and drop
*StringTraits from sal/rtl/math.cxx.
Change-Id: I0d7544dead03651dc71ec923cab10580f15cf49a
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/131895
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
We don't need it to strip trailing zeroes (we do that ourselves anyway);
also we don't need it to handle negatives.
Makes doubleToString ~5% faster in my testing.
Change-Id: Ie3c4e3fec1899364af54fda3f7141678a95359d0
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/130120
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
This header-only library is accurate in decimal representation of
doubles; provides API that allows to create custom representation
- so it's possible to use custom decimal separators and grouping.
This allows to unify all corner cases: integers, numbers close to
DBL_MAX, up-rounding to the next decade.
Note that Dragonbox creates the shortest decimal representation
of the number, that is unambiguously convertible back to the same
number; thus it may hide trailing digits that are unneeded for
such conversion.
The functional changes are minimal, and beneficial:
1. Rounding numbers close to DBL_MAX now takes into account the
bEraseTrailingDecZeros argument, as it should, allowing to have
"1.8E+308" for rounding DBL_MAX to 2 decimals without trailing
zeroes, instead of previous "1.80E+308".
2. Incorrect rounding is fixed in some cases, e.g. 9.9999999999999929
rounded to 10 previously using rtl_math_DecimalPlaces_Max.
3. Representing the number in the shortest way may change display
of some printed numbers. E.g., 5th greatest double is represented
as "1.797693134862315E+308" instead of a bit longer, but giving
the same double on roundtrip, "1.7976931348623149E+308". This would
generally look better for some numbers similar to the famous 0.1,
where users would likely expect more "round" representation where
it's unambiguous (but we still truncate to 15 significant decimals
anyway - so there's no point in pretending to provide exact digits
for actual binary representation).
These are reflected in the unit tests affected by the change.
Change-Id: I05e20274a30eec499593ee3e9ec070e1269232a2
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/129948
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
... instead of some calculated/hardcoded decimal literals.
The new syntax represents the power of 2 exponent directly.
Change-Id: I826bda6e06ec77f0706bb48abe2934f7289fddc7
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/125359
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Needs 5056 bytes of pre-calculated data.
Change-Id: I138d9dc80c176f675a6854fe906e235c98efcbc0
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122947
Tested-by: Jenkins
Reviewed-by: Noel Grandin <noel.grandin@collabora.co.uk>
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Change-Id: I8ba202232fa42765a8b04113639fdac4b5724aa2
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122941
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Change-Id: I80201148684f6e297ff0c880c0dbbc346129a557
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122864
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
A mistake in eae24a9488814e77254d175c11fc4a138c1dbd30
Change-Id: I0da64366e4c39b3f5559e8a1c757a94d811f041f
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122869
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
An overlook from commit be8da97976658ff19b4dd010bff328cd3f424c1b
Change-Id: I2f639a9d865b43b38e19ad35a0a9e6b5bc1c1c8f
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122863
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Change-Id: Ia88ceaaad700bf3c2c8db9bb19146b75146dad3d
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122861
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Change-Id: I699f22aac871fdcef9ee9326ac30091239fa02be
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122862
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Change-Id: I39ed2485a67ec7a8b24ab90ea0d69a5982374334
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122860
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Change-Id: I9963a527d2323f4df8c3b46c13f5b7993e22f163
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122855
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Fill it in the correct order, starting from the middle
Change-Id: Id35475e391d771d6c23252124a92825b24b55e0c
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122853
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Change-Id: Iee8966eeeaea461e34b5d22b80cb612dfaa57fe4
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122750
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Change-Id: I09ab406a8b7279801ce79b2f9c0a0011f6db05be
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122749
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
The code accepting "NaN" had been introduced with
92dafe9862d693ce9d79269627c3e6832422874e "dba33h: #i112652#: rtl/math.h:
string<->double conversion and XMLSchema-2: [...] rtl_math_stringToDouble and
rtl_math_uStringToDouble support XMLSchema-2 values in addition to deprecated
previously supported values." The "XMLSchema-2" mentioned in that commit and in
the corresponding <https://bz.apache.org/ooo/show_bug.cgi?id=112652> "ORB:
report builder not handle correctly NULL values" presumably references
<https://www.w3.org/TR/xmlschema-2/> "XML Schema Part 2: Datatypes Second
Edition", where section "3.2.5 double" only supports "NaN" without a "+" or a
"-" sign in the lexical representation. So stop accepting those.
(I came across this in the context of 2b2b6405161025678f91a5625e50d0b414597368
"Reliably generate positive or negative NaN again", wondering whether this code
should be updated too. But then decided that it is probably best not to allow
that non-standard signed NaN notation for this case, and just keep producing for
the "NaN" representation whatever std::numeric_limits<double>::quiet_NaN
produces.)
Change-Id: I035e78ca36240317f72f117d2b456fc474d8c08a
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/118647
Tested-by: Jenkins
Reviewed-by: Michael Stahl <michael.stahl@allotropia.de>
|
|
...after e5c80bb69a30dfb0a3daf6061ab127d92f8142d6 "Purge out setNan from
math.cxx" had dropped the use of rtl::math::setNan and sign bit fiddling, and
relied on the implicit assumption that std::numeric_limits<double>::quiet_NaN
would produce a positive NaN (but which does not seem to be guaranteed by the
C++ standard) and on the expressed hope that multiplying such a positive NaN by
-1 would generate a negative NaN (but which does not seem to be guaranteed by
IEEE 754: while it mandates that a NaN's payload is preserved across such an
operation, the result's sign bit appears to be unspecified)
Change-Id: I12215c888a1cb8de6b3f046a836c550cb21b5a85
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/118604
Tested-by: Jenkins
Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
|
|
Change-Id: I243d2998725f09ef533bd06865081d5c415f0d18
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/118282
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Change-Id: I95f9b37b564bb733f44899a8c6c1ea3c36e35694
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/118196
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Reduce operation count by space / speed tradeoff.
This expression is widely used in LO.
Then needs to be fast.
Change-Id: Ic88cf15d451ec95a8ad6da88cd9f601cf2876871
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/117954
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
Tested-by: Jenkins
|
|
C++ floating-point environment has thread storage duration, so
std::fesetround call is thread-safe.
std::nearbyint uses half-to-even rounding with FE_TONEAREST,
as specified in ISO/IEC 9899:1999 F.3, and in Appendix to
ANSI/IEEE 854.
Change-Id: I26995842c30e2663b40af5b5314791c1328037d4
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/117307
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
|
|
Change-Id: I7f9fb45824458d578ee302a668386fe0c3c80974
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/116010
Reviewed-by: Eike Rathke <erack@redhat.com>
Tested-by: Jenkins
|
|
Change-Id: I98cc25267e7a10c34179bab50d19f49436e1c48c
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107929
Tested-by: Jenkins
Reviewed-by: Eike Rathke <erack@redhat.com>
|
|
... to save some cycles as we anyway need only the integer value
of the exponent and even exactly this value for the number of
possible decimals.
Change-Id: I8d462f53cadde6a95d57d1342d8487fbfa001ae9
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107928
Tested-by: Jenkins
Reviewed-by: Eike Rathke <erack@redhat.com>
|
|
Similar to commit 5abb1890ffafe5a2212076208a1c6e226f1ffa4e for
rtl_math_round() use the reciprocal value in an inverse operation
for negative exponents to not use the inexact 1e-1
0.10000000000000001 and so on factors.
Change-Id: I05b852e06f2c31d6e0ce622b07277a81a5690833
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107172
Reviewed-by: Eike Rathke <erack@redhat.com>
Tested-by: Jenkins
|
|
Ditch mostly but not always working correction value and use
approxFloor() instead which yields better results. With this we
now have one single place approxValue() in case more fine grained
tuning is needed.
Unfortunately all numeric spreadsheet function tests use ROUND()
in a manner such that they mostly blindly do a ROUND(...;12)
regardless of magnitude, sometimes effectively rounding to the
14th significant digit that may fail in cases like for
14.2040730851385
^
where the constant (rounded) value is stored as is but the
calculated value is
14.204073085138471
and the old round() yielded
14.204073085139 for both but the new round() more correctly
results in
14.204073085139 and
14.204073085138
so the spreadsheet test case sources had to be changed to
ROUND(...;11) in such places.
Change-Id: I211bb868a4f4dc9e68f4c7dcc2a187b5e175416f
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107135
Reviewed-by: Eike Rathke <erack@redhat.com>
Tested-by: Jenkins
|
|
Change-Id: Ic436d3e9f0c93cb36c5e4377519f2aeb6b7fbd5f
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107034
Reviewed-by: Eike Rathke <erack@redhat.com>
Tested-by: Jenkins
|
|
Change-Id: Ica25747a26d6c2637c46808d1b73aeeed6e1df37
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107001
Reviewed-by: Eike Rathke <erack@redhat.com>
Tested-by: Jenkins
|
|
Decimal negative exponents (powers) are imprecise
1e-1 0.10000000000000001
1e-2 0.01
1e-3 0.001
1e-4 0.0001
1e-5 1.0000000000000001e-05
1e-6 9.9999999999999995e-07
1e-7 9.9999999999999995e-08
1e-8 1e-08
1e-9 1.0000000000000001e-09
1e-10 1e-10
1e-11 9.9999999999999994e-12
1e-12 9.9999999999999998e-13
1e-13 1e-13
1e-14 1e-14
1e-15 1.0000000000000001e-15
1e-16 9.9999999999999998e-17
1e-17 1.0000000000000001e-17
1e-18 1.0000000000000001e-18
1e-19 9.9999999999999998e-20
1e-20 9.9999999999999995e-21
so use the positive exponents instead and swap multiplication and
division when scaling and scaling back the value, which multiplies
the rounded value with a precise integer instead of dividing it by
an imprecise fraction.
For a large (absolute) value check if it is roundable to the
desired decimals at all with the binade's distance precision gap
and if not then diminish the decimals parameter. This prevents
possible inaccuracies due to overly scaling the value.
Change-Id: I41a113078031a552cf98d72f8cb2b10bdc88dea4
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/106830
Reviewed-by: Eike Rathke <erack@redhat.com>
Tested-by: Jenkins
|
|
The mix with SAL_N_ELEMENTS() was confusing and even wrong in one
case.
Change-Id: Ife73342b0efc01ed4e76e217d372fc32500c9b2c
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/106764
Reviewed-by: Eike Rathke <erack@redhat.com>
Tested-by: Jenkins
|
|
Change-Id: I2ae6e3cadc0f182c4798e5d33b0c7f07fbcbbff6
|
|
... and the 4 subsequent nextafters to appropriate strings.
An interim workaround to not write the out-of-range string
1.79769313486232e+308 until we have a proper rounding.
Change-Id: I5f98a7f0a8e0421fd024a8cc37cc6f3a198d02d1
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/106717
Tested-by: Jenkins
Reviewed-by: Eike Rathke <erack@redhat.com>
|
|
This does not solve writing the badly rounded value, which still
has to be fixed, but at least be able to read/convert it back as
double max value.
This is not even used in the bug scenario because the "fake"
condition in that number format is discarded anyway because it was
only written to satisfy ODF, but it helps in case a
1.7976931348623157e+308 value was entered/used on purpose.
Change-Id: I413dd93d5a3c4df609fd42a3133d6d82c34afff0
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/106586
Reviewed-by: Eike Rathke <erack@redhat.com>
Tested-by: Jenkins
|
|
Change-Id: I890d19f5e2177294dc1175c90c98b964347f9e85
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/105751
Tested-by: Jenkins
Reviewed-by: Noel Grandin <noel.grandin@collabora.co.uk>
|
|
Protect against callers using for example rtl_math_StringFormat_F
with rtl_math_DecimalPlaces_Max in worst case..
Change-Id: I9f143df6ae67b22e7732547c0f7a53b498caf2b8
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/104472
Reviewed-by: Eike Rathke <erack@redhat.com>
Tested-by: Jenkins
|
|
+ remove sal_Char check on compilerplugins
Change-Id: I0f7da14e620f0c3d031d038aa8345ba4080fb3e9
Change-Id: Ia6dba4f27b47bc9e0c89159182ad80a5aee17166
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/102499
Tested-by: Jenkins
Reviewed-by: Noel Grandin <noel.grandin@collabora.co.uk>
|
|
Change-Id: I022f5ed37d25f2c8a8870033bab32ff59d4d8da6
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/97648
Tested-by: Jenkins
Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
|
|
GCC appears to support it at least since <https://gcc.gnu.org/git/
?p=gcc.git;a=commit;h=51e2940139d5e3e86590f6e6802ffc3f3010be5b> "Initial
revision" in 1992, and Clang appears to support it since <https://github.com/
llvm/llvm-project/commit/d93abc3bb0acdd430839abdd67bd3920fee87bbc> "Implement
ffs, parity, and popcount builtins" in Clang 2.4. (And if a build used a
compiler that does not support it, there would be no guarantee that it would
support strings.h function ffs from X/Open System Interfaces, either.)
Introducing HAVE_GCC_BUILTIN_FFS in 334a9f16cd1d1f9694f885c759903a41aa3d4833
"tdf#113211: fix calculations with big integers" appears to be due to a
misguided recommendation at <https://gerrit.libreoffice.org/c/core/+/43477/4#
message-899806c724fbdcece0ea9438514a6a5db6a2e645>.
Change-Id: Ib6ee6de548172b3aae25483d03efb86620133933
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/92421
Tested-by: Jenkins
Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
|
|
Change-Id: I5a7bc9378ceacb9116c03e3a9fc01c5675c40908
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/92243
Tested-by: Jenkins
Reviewed-by: Noel Grandin <noel.grandin@collabora.co.uk>
|