diff options
| author | Eike Rathke <erack@redhat.com> | 2021-01-06 01:24:41 +0100 |
|---|---|---|
| committer | Eike Rathke <erack@redhat.com> | 2021-01-06 17:37:40 +0100 |
| commit | 70ea6b36df9ede18b135876d9b9da9945f6c129b (patch) | |
| tree | 7d91779c218d7362eb7b559638140b8aba0a6e36 /scaddins | |
| parent | c7568cfdec8ababa4d73eddb7bed057404a6a009 (diff) | |
Resolves: tdf#139173 One-off error in limits for DEC2HEX()
BIN2HEX() and OCT2HEX() were not affected because the string input
is already limited to 10 characters and the converted decimal
can't even reach the limits.
Change-Id: Iba4212e8fc382287a1a454edf91426ba21497ae2
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/108824
Reviewed-by: Eike Rathke <erack@redhat.com>
Tested-by: Jenkins
Diffstat (limited to 'scaddins')
| -rw-r--r-- | scaddins/source/analysis/analysis.cxx | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/scaddins/source/analysis/analysis.cxx b/scaddins/source/analysis/analysis.cxx index 6fb4e9b958ff..3f7f4317eb55 100644 --- a/scaddins/source/analysis/analysis.cxx +++ b/scaddins/source/analysis/analysis.cxx @@ -661,7 +661,7 @@ const double SCA_MAX2 = 511.0; // min. val for binary numbe const double SCA_MIN2 = -SCA_MAX2-1.0; // min. val for binary numbers (9 bits + sign) const double SCA_MAX8 = 536870911.0; // max. val for octal numbers (29 bits + sign) const double SCA_MIN8 = -SCA_MAX8-1.0; // min. val for octal numbers (29 bits + sign) -const double SCA_MAX16 = 549755813888.0; // max. val for hexadecimal numbers (39 bits + sign) +const double SCA_MAX16 = 549755813887.0; // max. val for hexadecimal numbers (39 bits + sign) const double SCA_MIN16 = -SCA_MAX16-1.0; // min. val for hexadecimal numbers (39 bits + sign) const sal_Int32 SCA_MAXPLACES = 10; // max. number of places |