summaryrefslogtreecommitdiff
path: root/sal/rtl/math.cxx
AgeCommit message (Collapse)Author
2023-07-12Simplify a bitMike Kaganski
Change-Id: Idd3e96d99f13b1e87e2a01de9c9392ead0864de7 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/154323 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2022-05-11Revert "Disable -fsanitize=float-divide-by-zero in rtl_math_atanh"Stephan Bergmann
This reverts commit 6a4504bba84dcbaeb71869ec5c9ed6dfdc090619, which is no longer necessary after 6f75ec6115f0152517be634070607bc61bf96dd0 "tdf#148430 Use atanh from <cmath> instead of our own" changed the implementation of rtl_math_atanh. Change-Id: I11094f8c9fb0bb7b635541e7360e460816d3c171 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/134161 Tested-by: Jenkins Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
2022-05-11tdf#148430 Use atanh from <cmath> instead of our ownofftkp
Change wrapper in rtl::math::atanh to use atanh from <cmath>. Also changed all occurrences of rtl::math::atanh on files that use this function to directly use the standard atanh instead. Change-Id: Idc5c456f67291f5816756f023b61afde844b5ceb Reviewed-on: https://gerrit.libreoffice.org/c/core/+/133965 Tested-by: Jenkins Reviewed-by: Bartosz Kosiorek <gang65@poczta.onet.pl>
2022-03-21Move implementation of doubleToString to sal/rtl/strtmpl.hxxMike Kaganski
... to reduce inter-relations between compilation units, and drop *StringTraits from sal/rtl/math.cxx. Change-Id: I0d7544dead03651dc71ec923cab10580f15cf49a Reviewed-on: https://gerrit.libreoffice.org/c/core/+/131895 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2022-02-18Define policy to optimize to_decimal to our use caseMike Kaganski
We don't need it to strip trailing zeroes (we do that ourselves anyway); also we don't need it to handle negatives. Makes doubleToString ~5% faster in my testing. Change-Id: Ie3c4e3fec1899364af54fda3f7141678a95359d0 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/130120 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2022-02-17Use Dragonbox to implement doubleTo*String*Mike Kaganski
This header-only library is accurate in decimal representation of doubles; provides API that allows to create custom representation - so it's possible to use custom decimal separators and grouping. This allows to unify all corner cases: integers, numbers close to DBL_MAX, up-rounding to the next decade. Note that Dragonbox creates the shortest decimal representation of the number, that is unambiguously convertible back to the same number; thus it may hide trailing digits that are unneeded for such conversion. The functional changes are minimal, and beneficial: 1. Rounding numbers close to DBL_MAX now takes into account the bEraseTrailingDecZeros argument, as it should, allowing to have "1.8E+308" for rounding DBL_MAX to 2 decimals without trailing zeroes, instead of previous "1.80E+308". 2. Incorrect rounding is fixed in some cases, e.g. 9.9999999999999929 rounded to 10 previously using rtl_math_DecimalPlaces_Max. 3. Representing the number in the shortest way may change display of some printed numbers. E.g., 5th greatest double is represented as "1.797693134862315E+308" instead of a bit longer, but giving the same double on roundtrip, "1.7976931348623149E+308". This would generally look better for some numbers similar to the famous 0.1, where users would likely expect more "round" representation where it's unambiguous (but we still truncate to 15 significant decimals anyway - so there's no point in pretending to provide exact digits for actual binary representation). These are reflected in the unit tests affected by the change. Change-Id: I05e20274a30eec499593ee3e9ec070e1269232a2 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/129948 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-11-17Use C++17 hex-exponent floating point literalsMike Kaganski
... instead of some calculated/hardcoded decimal literals. The new syntax represents the power of 2 exponent directly. Change-Id: I826bda6e06ec77f0706bb48abe2934f7289fddc7 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/125359 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-10-05Optimize the whole range of getN10Exp over IEEE 754 doublesMike Kaganski
Needs 5056 bytes of pre-calculated data. Change-Id: I138d9dc80c176f675a6854fe906e235c98efcbc0 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122947 Tested-by: Jenkins Reviewed-by: Noel Grandin <noel.grandin@collabora.co.uk> Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-10-04Use isRepresentableInteger hereMike Kaganski
Change-Id: I8ba202232fa42765a8b04113639fdac4b5724aa2 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122941 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-10-01Simplify the code a bit to clarify the logicMike Kaganski
Change-Id: I80201148684f6e297ff0c880c0dbbc346129a557 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122864 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-10-01Calculate buffer size correctlyMike Kaganski
A mistake in eae24a9488814e77254d175c11fc4a138c1dbd30 Change-Id: I0da64366e4c39b3f5559e8a1c757a94d811f041f Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122869 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-10-01Consider leading minusMike Kaganski
An overlook from commit be8da97976658ff19b4dd010bff328cd3f424c1b Change-Id: I2f639a9d865b43b38e19ad35a0a9e6b5bc1c1c8f Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122863 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-10-01Use std::memmove instead of loopMike Kaganski
Change-Id: Ia88ceaaad700bf3c2c8db9bb19146b75146dad3d Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122861 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-10-01Drop static_cast syntactic noiseMike Kaganski
Change-Id: I699f22aac871fdcef9ee9326ac30091239fa02be Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122862 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-10-01Always use buffer on stackMike Kaganski
Change-Id: I39ed2485a67ec7a8b24ab90ea0d69a5982374334 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122860 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-09-30Simplify comparisons in the loopMike Kaganski
Change-Id: I9963a527d2323f4df8c3b46c13f5b7993e22f163 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122855 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-09-30Avoid reversing the bufferMike Kaganski
Fill it in the correct order, starting from the middle Change-Id: Id35475e391d771d6c23252124a92825b24b55e0c Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122853 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-09-29Use std::reverse instead of swapping in a loopMike Kaganski
Change-Id: Iee8966eeeaea461e34b5d22b80cb612dfaa57fe4 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122750 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-09-29Simplify integer roundingMike Kaganski
Change-Id: I09ab406a8b7279801ce79b2f9c0a0011f6db05be Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122749 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-07-09Do not support +/-NaN with an explicit signStephan Bergmann
The code accepting "NaN" had been introduced with 92dafe9862d693ce9d79269627c3e6832422874e "dba33h: #i112652#: rtl/math.h: string<->double conversion and XMLSchema-2: [...] rtl_math_stringToDouble and rtl_math_uStringToDouble support XMLSchema-2 values in addition to deprecated previously supported values." The "XMLSchema-2" mentioned in that commit and in the corresponding <https://bz.apache.org/ooo/show_bug.cgi?id=112652> "ORB: report builder not handle correctly NULL values" presumably references <https://www.w3.org/TR/xmlschema-2/> "XML Schema Part 2: Datatypes Second Edition", where section "3.2.5 double" only supports "NaN" without a "+" or a "-" sign in the lexical representation. So stop accepting those. (I came across this in the context of 2b2b6405161025678f91a5625e50d0b414597368 "Reliably generate positive or negative NaN again", wondering whether this code should be updated too. But then decided that it is probably best not to allow that non-standard signed NaN notation for this case, and just keep producing for the "NaN" representation whatever std::numeric_limits<double>::quiet_NaN produces.) Change-Id: I035e78ca36240317f72f117d2b456fc474d8c08a Reviewed-on: https://gerrit.libreoffice.org/c/core/+/118647 Tested-by: Jenkins Reviewed-by: Michael Stahl <michael.stahl@allotropia.de>
2021-07-08Reliably generate positive or negative NaN againStephan Bergmann
...after e5c80bb69a30dfb0a3daf6061ab127d92f8142d6 "Purge out setNan from math.cxx" had dropped the use of rtl::math::setNan and sign bit fiddling, and relied on the implicit assumption that std::numeric_limits<double>::quiet_NaN would produce a positive NaN (but which does not seem to be guaranteed by the C++ standard) and on the expressed hope that multiplying such a positive NaN by -1 would generate a negative NaN (but which does not seem to be guaranteed by IEEE 754: while it mandates that a NaN's payload is preserved across such an operation, the result's sign bit appears to be unspecified) Change-Id: I12215c888a1cb8de6b3f046a836c550cb21b5a85 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/118604 Tested-by: Jenkins Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
2021-07-04Remove unused includesBaiXiaochun
Change-Id: I243d2998725f09ef533bd06865081d5c415f0d18 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/118282 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-07-02Purge out setNan from math.cxxBaiXiaochun
Change-Id: I95f9b37b564bb733f44899a8c6c1ea3c36e35694 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/118196 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-06-29Simplify expresionBaiXiaochun
Reduce operation count by space / speed tradeoff. This expression is widely used in LO. Then needs to be fast. Change-Id: Ic88cf15d451ec95a8ad6da88cd9f601cf2876871 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/117954 Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com> Tested-by: Jenkins
2021-06-18Use std::fesetround / std::nearbyint for half-to-even roundingMike Kaganski
C++ floating-point environment has thread storage duration, so std::fesetround call is thread-safe. std::nearbyint uses half-to-even rounding with FE_TONEAREST, as specified in ISO/IEC 9899:1999 F.3, and in Appendix to ANSI/IEEE 854. Change-Id: I26995842c30e2663b40af5b5314791c1328037d4 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/117307 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-05-23Related: tdf#136794 tdf#137063 Unlimit decimals in rtl_math_round()Eike Rathke
Change-Id: I7f9fb45824458d578ee302a668386fe0c3c80974 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/116010 Reviewed-by: Eike Rathke <erack@redhat.com> Tested-by: Jenkins
2020-12-18Check intermediate for not to be rounded value, tdf#138360 follow-upEike Rathke
Change-Id: I98cc25267e7a10c34179bab50d19f49436e1c48c Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107929 Tested-by: Jenkins Reviewed-by: Eike Rathke <erack@redhat.com>
2020-12-18Replace log2() call with parts.exponent-1023, tdf#138360 follow-upEike Rathke
... to save some cycles as we anyway need only the integer value of the exponent and even exactly this value for the number of possible decimals. Change-Id: I8d462f53cadde6a95d57d1342d8487fbfa001ae9 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107928 Tested-by: Jenkins Reviewed-by: Eike Rathke <erack@redhat.com>
2020-12-03Better accuracy in rtl_math_approxValue(), tdf#138360 relatedEike Rathke
Similar to commit 5abb1890ffafe5a2212076208a1c6e226f1ffa4e for rtl_math_round() use the reciprocal value in an inverse operation for negative exponents to not use the inexact 1e-1 0.10000000000000001 and so on factors. Change-Id: I05b852e06f2c31d6e0ce622b07277a81a5690833 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107172 Reviewed-by: Eike Rathke <erack@redhat.com> Tested-by: Jenkins
2020-12-03Related: tdf#138360 Use approxFloor() in rtl_math_round()Eike Rathke
Ditch mostly but not always working correction value and use approxFloor() instead which yields better results. With this we now have one single place approxValue() in case more fine grained tuning is needed. Unfortunately all numeric spreadsheet function tests use ROUND() in a manner such that they mostly blindly do a ROUND(...;12) regardless of magnitude, sometimes effectively rounding to the 14th significant digit that may fail in cases like for 14.2040730851385 ^ where the constant (rounded) value is stored as is but the calculated value is 14.204073085138471 and the old round() yielded 14.204073085139 for both but the new round() more correctly results in 14.204073085139 and 14.204073085138 so the spreadsheet test case sources had to be changed to ROUND(...;11) in such places. Change-Id: I211bb868a4f4dc9e68f4c7dcc2a187b5e175416f Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107135 Reviewed-by: Eike Rathke <erack@redhat.com> Tested-by: Jenkins
2020-12-02Typo in rounded digit string, tdf#138360 follow-upEike Rathke
Change-Id: Ic436d3e9f0c93cb36c5e4377519f2aeb6b7fbd5f Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107034 Reviewed-by: Eike Rathke <erack@redhat.com> Tested-by: Jenkins
2020-12-02Related: tdf#138360 Rounding integers to decimals is futileEike Rathke
Change-Id: Ica25747a26d6c2637c46808d1b73aeeed6e1df37 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107001 Reviewed-by: Eike Rathke <erack@redhat.com> Tested-by: Jenkins
2020-11-28Resolves: tdf#138360 better accuracy in rtl_math_round()Eike Rathke
Decimal negative exponents (powers) are imprecise 1e-1 0.10000000000000001 1e-2 0.01 1e-3 0.001 1e-4 0.0001 1e-5 1.0000000000000001e-05 1e-6 9.9999999999999995e-07 1e-7 9.9999999999999995e-08 1e-8 1e-08 1e-9 1.0000000000000001e-09 1e-10 1e-10 1e-11 9.9999999999999994e-12 1e-12 9.9999999999999998e-13 1e-13 1e-13 1e-14 1e-14 1e-15 1.0000000000000001e-15 1e-16 9.9999999999999998e-17 1e-17 1.0000000000000001e-17 1e-18 1.0000000000000001e-18 1e-19 9.9999999999999998e-20 1e-20 9.9999999999999995e-21 so use the positive exponents instead and swap multiplication and division when scaling and scaling back the value, which multiplies the rounded value with a precise integer instead of dividing it by an imprecise fraction. For a large (absolute) value check if it is roundable to the desired decimals at all with the binade's distance precision gap and if not then diminish the decimals parameter. This prevents possible inaccuracies due to overly scaling the value. Change-Id: I41a113078031a552cf98d72f8cb2b10bdc88dea4 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/106830 Reviewed-by: Eike Rathke <erack@redhat.com> Tested-by: Jenkins
2020-11-27Consistently use RTL_CONSTASCII_LENGTH(), tdf#136272 follow-upEike Rathke
The mix with SAL_N_ELEMENTS() was confusing and even wrong in one case. Change-Id: Ife73342b0efc01ed4e76e217d372fc32500c9b2c Reviewed-on: https://gerrit.libreoffice.org/c/core/+/106764 Reviewed-by: Eike Rathke <erack@redhat.com> Tested-by: Jenkins
2020-11-27Fix commentEike Rathke
Change-Id: I2ae6e3cadc0f182c4798e5d33b0c7f07fbcbbff6
2020-11-27Resolves: tdf#136272 convert DBL_MAX to 1.7976931348623157e+308Eike Rathke
... and the 4 subsequent nextafters to appropriate strings. An interim workaround to not write the out-of-range string 1.79769313486232e+308 until we have a proper rounding. Change-Id: I5f98a7f0a8e0421fd024a8cc37cc6f3a198d02d1 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/106717 Tested-by: Jenkins Reviewed-by: Eike Rathke <erack@redhat.com>
2020-11-25Related: tdf#136272 accept 1.79769313486232e+308 as DBL_MAXEike Rathke
This does not solve writing the badly rounded value, which still has to be fixed, but at least be able to read/convert it back as double max value. This is not even used in the bug scenario because the "fake" condition in that number format is discarded anyway because it was only written to satisfy ODF, but it helps in case a 1.7976931348623157e+308 value was entered/used on purpose. Change-Id: I413dd93d5a3c4df609fd42a3133d6d82c34afff0 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/106586 Reviewed-by: Eike Rathke <erack@redhat.com> Tested-by: Jenkins
2020-11-16replace std::max(std::min()) with std::clampNoel
Change-Id: I890d19f5e2177294dc1175c90c98b964347f9e85 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/105751 Tested-by: Jenkins Reviewed-by: Noel Grandin <noel.grandin@collabora.co.uk>
2020-10-18Limit nDecPlaces to a sensible value [-20, 20]Eike Rathke
Protect against callers using for example rtl_math_StringFormat_F with rtl_math_DecimalPlaces_Max in worst case.. Change-Id: I9f143df6ae67b22e7732547c0f7a53b498caf2b8 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/104472 Reviewed-by: Eike Rathke <erack@redhat.com> Tested-by: Jenkins
2020-09-12Replace remaining uses of sal_CharJulien Nabet
+ remove sal_Char check on compilerplugins Change-Id: I0f7da14e620f0c3d031d038aa8345ba4080fb3e9 Change-Id: Ia6dba4f27b47bc9e0c89159182ad80a5aee17166 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/102499 Tested-by: Jenkins Reviewed-by: Noel Grandin <noel.grandin@collabora.co.uk>
2020-07-02Upcoming improved loplugin:staticanonymous -> redundantstatic: salStephan Bergmann
Change-Id: I022f5ed37d25f2c8a8870033bab32ff59d4d8da6 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/97648 Tested-by: Jenkins Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
2020-04-17Just use __builtin_ffs on GCC and ClangStephan Bergmann
GCC appears to support it at least since <https://gcc.gnu.org/git/ ?p=gcc.git;a=commit;h=51e2940139d5e3e86590f6e6802ffc3f3010be5b> "Initial revision" in 1992, and Clang appears to support it since <https://github.com/ llvm/llvm-project/commit/d93abc3bb0acdd430839abdd67bd3920fee87bbc> "Implement ffs, parity, and popcount builtins" in Clang 2.4. (And if a build used a compiler that does not support it, there would be no guarantee that it would support strings.h function ffs from X/Open System Interfaces, either.) Introducing HAVE_GCC_BUILTIN_FFS in 334a9f16cd1d1f9694f885c759903a41aa3d4833 "tdf#113211: fix calculations with big integers" appears to be due to a misguided recommendation at <https://gerrit.libreoffice.org/c/core/+/43477/4# message-899806c724fbdcece0ea9438514a6a5db6a2e645>. Change-Id: Ib6ee6de548172b3aae25483d03efb86620133933 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/92421 Tested-by: Jenkins Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
2020-04-15loplugin:buriedassign in salNoel Grandin
Change-Id: I5a7bc9378ceacb9116c03e3a9fc01c5675c40908 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/92243 Tested-by: Jenkins Reviewed-by: Noel Grandin <noel.grandin@collabora.co.uk>
2020-03-14tdf#130975 replace `rtl::math::isNan` with `std::isnan`.Yukio Siraichi
Change-Id: I5d53e6369d35093445b2efd8936bbf8c6775ff47 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/90451 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2020-03-14tdf#130974 replace `rtl::math::isSignBitSet` with `std::signbit`.Yukio Siraichi
Change-Id: I91235eee8c6a9d4a59c1933527b49141f64cd91b Reviewed-on: https://gerrit.libreoffice.org/c/core/+/90478 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2020-03-12tdf#130976 replace `rtl::math::isInf` with `std::inf`.Yukio Siraichi
Change-Id: Id0f0e07b324230d2d69cbf4ab07b0fff5b24474d Reviewed-on: https://gerrit.libreoffice.org/c/core/+/90377 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2020-03-12tdf#130977 replace `rtl::math::isFinite` with `std::isfinite`.Yukio Siraichi
- make all calls look like `std::isfinite`. - change the comments referring `rtl::math::isFinite`. Change-Id: I0cde9ceb9f20150467b454cddde5e62003cfde1a Reviewed-on: https://gerrit.libreoffice.org/c/core/+/90234 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2020-02-27Related: tdf#130725: use strtod also in rtl::math::stringToDoubleMike Kaganski
Size of buffer on stack is 256 characters. Logging function usage in make check, of >1 100 000 invocations, the longest string was 80 characters, average being 4.6 characters. So heap allocation is unlikely in scenarios with intensive function usage. Several existing unit tests had to be fixed. Usually, the change is either minimal or getting closer to what Excel returns (for Calc tests). But in case of AMORDEGRC, I had to change rate value passed to the function from 0.3 to 0.31. It's because the closest double value for 0.3 is 0.29999999999999999, which is a bit less than 0.3; multiplied by 1.5, this gives 0.44999999999999996, and then rounding the result of multiplication of the latter by cost gave the result 1 less than before, when 0.3 was imported as 0.30000000000000004. Now the function returns a value 1 less than Excel for that set of arguments. I don't see how to fix that. Having rate slightly different gives consistent result between Calc and Excel. Change-Id: Icae5ce374fe0c31a1aa10cee815e65ef0014f382 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/89422 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2019-12-19sal_Char->char in remotebridges..saxNoel Grandin
Change-Id: I6d32942960a5e997f16eb1301c45495661cd4cea Reviewed-on: https://gerrit.libreoffice.org/85514 Tested-by: Jenkins Reviewed-by: Noel Grandin <noel.grandin@collabora.co.uk>
2019-08-09Resolves: tdf#126766 fix rounding correction at start of negative valuesEike Rathke
doubleToString() is entered with an inaccuracy afflicted fValue=-9999.9999999999927 for which the rounding into the next magnitude incremented the '-' character to '.' instead of appending a '1' (and '0' and then "000") thus yielded ".0000" instead of "-10000" This seems to have been always the case. Change-Id: I66170defa71fec40ca0b85f68affde8eff0d5ccb Reviewed-on: https://gerrit.libreoffice.org/77208 Reviewed-by: Eike Rathke <erack@redhat.com> Tested-by: Jenkins
generated by cgit (git 2.34.1) at 2025-01-31 17:46:58 +0000