diff options
author | Stephan Bergmann <sbergman@redhat.com> | 2019-06-27 13:33:27 +0200 |
---|---|---|
committer | Michael Stahl <michael.stahl@cib.de> | 2019-12-23 12:08:26 +0100 |
commit | 9df327e2dda452dd557eb6bcc0b6f54cfe1fef68 (patch) | |
tree | feffe29c80e894f83bbb0acc8a586466a5e54f26 /sw | |
parent | 46b3635facf318d50ba806b8ef9124984d8523c3 (diff) |
tdf#126127: Make nTmp smaller still, avoid -fsanitize=signed-integer-overflow
...after f2e3655255db4032738849cd4b77ce67a6e2c984 "Avoid
-fsanitize=signed-integer-overflow" had already reduced it from using LONG_MAX
to TWIPS_MAX/2 in the past. This time, avoid the computation of
> const sal_uInt64 nCurrentDist = sal_Int64(aDiff.getX()) * sal_Int64(aDiff.getX()) + sal_Int64(aDiff.getY()) * sal_Int64(aDiff.getY()); // opt: no sqrt
in GetFrameOfModify (sw/source/core/layout/frmtool.cxx) from overflowing (where
aDiff.getY() derives from nTmp and can be close to it in magnitude, so computing
its square would overflow on platforms where TWIPS_MAX is a large sal_Int64
value).
(The "empirically shown to be large enough in practice" in the comment is a
successful `make check` on Linux 64-bit with UBSan.)
Change-Id: Ic7f058bd6853ff04ccb50a150509e98f850d12d2
Reviewed-on: https://gerrit.libreoffice.org/74801
Reviewed-by: Michael Stahl <michael.stahl@cib.de>
Tested-by: Jenkins
(cherry picked from commit 8723ac4e20eda87a82393f2f6c7d28ece8514238)
Reviewed-on: https://gerrit.libreoffice.org/85733
Diffstat (limited to 'sw')
-rw-r--r-- | sw/source/core/text/frmform.cxx | 9 |
1 files changed, 8 insertions, 1 deletions
diff --git a/sw/source/core/text/frmform.cxx b/sw/source/core/text/frmform.cxx index 492a738941c3..7e7460dd3812 100644 --- a/sw/source/core/text/frmform.cxx +++ b/sw/source/core/text/frmform.cxx @@ -835,7 +835,14 @@ bool SwTextFrame::CalcPreps() } else { - SwTwips nTmp = TWIPS_MAX/2 - (getFrameArea().Top()+10000); + // nTmp should be very large, but not so large as to cause overflow later (e.g., + // GetFrameOfModify in sw/source/core/layout/frmtool.cxx calculates nCurrentDist + // from, among others, the square of aDiff.getY(), which can be close to nTmp); + // the previously used value TWIPS_MAX/2 (i.e., (LONG_MAX - 1)/2) depended on + // the range of 'long', while the value (SAL_MAX_INT32 - 1)/2 (which matches the + // old value on platforms where 'long' is 'sal_Int32') is empirically shown to + // be large enough in practice even on platforms where 'long' is 'sal_Int64': + SwTwips nTmp = (SAL_MAX_INT32 - 1)/2 - (getFrameArea().Top()+10000); SwTwips nDiff = nTmp - getFrameArea().Height(); { |