tdf#32834 : improve the precision of MDETERM calculation

MDETERM uses lcl_LUP_decompose() which as the name suggests does LUP decomposition.
This patch allows additive cancellations to occur in a cleaner way while doing the *only*
additive operation in LUP decomposition, that is while computing Shur complement.
This patch does not change the high level semantics of the algorithm.
Also note that this change makes improvement only in the case where matrix elements entered
by the user are *integers*. The change allows MDETERM to evaluate to exact 0.0 for
singular integer matrices.

The steps to calculate Shur complement are :

for i = k+1 to n
aik = aik / akk;
for j = k+1 to n
aij = aij - akj*aik

This is now modified as :

for i = k+1 to n
for j = k+1 to n
aij = ( aij*akk - akj*aik ) / akk

Without this change MDETERM() for certain singular matrices used to evaluate to a tiny non zero value,
which also caused MINVERSE() to generate a wrong output.
An example of such a matrix is :
1 2 3
4 5 6
7 8 9

Change-Id: Idd4211ddceab1b758fd05bfd57f7eecd5d4fd1a0
Reviewed-on: https://gerrit.libreoffice.org/19534
Reviewed-by: Eike Rathke <erack@redhat.com>
Tested-by: Eike Rathke <erack@redhat.com>

0 files changed, 0 insertions, 0 deletions