diff options
| author | Eike Rathke <erack@redhat.com> | 2021-01-06 01:24:41 +0100 |
|---|---|---|
| committer | Xisco Fauli <xiscofauli@libreoffice.org> | 2021-01-07 10:31:33 +0100 |
| commit | 072b43a4bdf8392c9a6a8cb1ad5e8a3e61c60117 (patch) | |
| tree | 3668ef5919cd2a92f26c668947d525e6d2810d3b | |
| parent | 9e82c64fa54ce7c0ef18e9de5a06e1243a3a7e44 (diff) | |
Resolves: tdf#139173 One-off error in limits for DEC2HEX()
BIN2HEX() and OCT2HEX() were not affected because the string input
is already limited to 10 characters and the converted decimal
can't even reach the limits.
Change-Id: Iba4212e8fc382287a1a454edf91426ba21497ae2
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/108824
Reviewed-by: Eike Rathke <erack@redhat.com>
Tested-by: Jenkins
(cherry picked from commit 70ea6b36df9ede18b135876d9b9da9945f6c129b)
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/108896
Reviewed-by: Xisco Fauli <xiscofauli@libreoffice.org>
| -rw-r--r-- | scaddins/source/analysis/analysis.cxx | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/scaddins/source/analysis/analysis.cxx b/scaddins/source/analysis/analysis.cxx index 9db4d55fee6f..d5226516000d 100644 --- a/scaddins/source/analysis/analysis.cxx +++ b/scaddins/source/analysis/analysis.cxx @@ -664,7 +664,7 @@ const double SCA_MAX2 = 511.0; // min. val for binary numbe const double SCA_MIN2 = -SCA_MAX2-1.0; // min. val for binary numbers (9 bits + sign) const double SCA_MAX8 = 536870911.0; // max. val for octal numbers (29 bits + sign) const double SCA_MIN8 = -SCA_MAX8-1.0; // min. val for octal numbers (29 bits + sign) -const double SCA_MAX16 = 549755813888.0; // max. val for hexadecimal numbers (39 bits + sign) +const double SCA_MAX16 = 549755813887.0; // max. val for hexadecimal numbers (39 bits + sign) const double SCA_MIN16 = -SCA_MAX16-1.0; // min. val for hexadecimal numbers (39 bits + sign) const sal_Int32 SCA_MAXPLACES = 10; // max. number of places |