WIN prevent deadlock in SetForegroundWindow

As mentioned in various blogs, like Raymon Chens "The old new
thing", 2008-08-01, "I warned you: The dangers of attaching input
queues", using AttachThreadInput to steal the input from an other
thread, so SetForegroundWindow becomes more reliable, can
deadlock in that call in win32u.dll!NtUserCallHwndLock.

Stackoverflow also has a multitude of suggestions and links in
"Win32 SetForegroundWindow unreliable", to circumvent Windows
focus-stealing prevention mechanisms.

A customer is experiencing these hangs reliably and often when
opening LO windows via Java UNO, because the Window and the UNO
thread are different and trigger this code path. Removing the
calls to AttachThreadInput fixes the problem for them. This has
started lately and nobody really knows why.
I also know other customers with a similar Java UNO setup, which
don't experience them.

For better foreground handling, the calling app eventually should
either use AllowSetForegroundWindow or CoAllowSetForegroundWindow
(for COM servers), to give up the foreground / input handling.

This is plus:
WIN add bootstrap var to re-enable foreground hack

We already found a regressed installation from commit
518c0265efebf39ab6d1e90c4ec4e7cf52b701c6 ("WIN prevent deadlock
in SetForegroundWindow").

Finding a real fix might be impossible, so this just adds the
bootstrap.ini boolean variable Win32.EnableAttachThreadInputHack
to re-enable the previous AttachThreadInput hack on demand.

Reviewed-on: https://gerrit.libreoffice.org/c/core/+/90235

Change-Id: I8de0a17aaaa44c24b1ee728b2ef6ec3aea951c54
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/89518
Tested-by: Thorsten Behrens <Thorsten.Behrens@CIB.de>
Reviewed-by: Thorsten Behrens <Thorsten.Behrens@CIB.de>

1 files changed, 55 insertions, 9 deletions

diff --git a/vcl/win/window/salframe.cxx b/vcl/win/window/salframe.cxx
index 8b0a64232927..fcb7c3250a27 100644
--- a/vcl/win/window/salframe.cxx
+++ b/vcl/win/window/salframe.cxx
@@ -33,6 +33,12 @@
 
 #include <comphelper/windowserrorstring.hxx>
 
+#include <fstream>
+#include <boost/property_tree/ptree.hpp>
+#include <boost/property_tree/ini_parser.hpp>
+#include <osl/file.hxx>
+#include <osl/process.h>
+
 #include <rtl/string.h>
 #include <rtl/ustring.h>
 
@@ -1919,23 +1925,63 @@ void WinSalFrame::SetAlwaysOnTop( bool bOnTop )
     SetWindowPos( mhWnd, hWnd, 0, 0, 0, 0, SWP_NOMOVE | SWP_NOSIZE | SWP_NOACTIVATE );
 }
 
+static bool EnableAttachThreadInputHack()
+{
+    OUString aBootstrapUri;
+    if (osl_getProcessWorkingDir(&aBootstrapUri.pData) != osl_Process_E_None)
+        return false;
+    aBootstrapUri += "/bootstrap.ini";
+
+    OUString aSystemFileName;
+    if (osl::FileBase::getSystemPathFromFileURL(aBootstrapUri, aSystemFileName) != osl::FileBase::E_None)
+        return false;
+    if (aSystemFileName.getLength() > MAX_PATH)
+        return false;
+
+    // this uses the Boost ini parser, instead of tools::Config, as we already use it to read other
+    // values from bootstrap.ini in desktop/win32/source/loader.cxx, because that watchdog process
+    // can't access LO libs. This way the handling is consistent.
+    try
+    {
+        boost::property_tree::ptree pt;
+        std::ifstream aFile(o3tl::toW(aSystemFileName.getStr()));
+        boost::property_tree::ini_parser::read_ini(aFile, pt);
+        const bool bEnabled = pt.get("Win32.EnableAttachThreadInputHack", false);
+        SAL_WARN_IF(bEnabled, "vcl", "AttachThreadInput hack is enabled. Watch out for deadlocks!");
+        return bEnabled;
+    }
+    catch (...)
+    {
+        return false;
+    }
+}
+
 static void ImplSalToTop( HWND hWnd, SalFrameToTop nFlags )
 {
+    static const bool bEnableAttachThreadInputHack = EnableAttachThreadInputHack();
+
     WinSalFrame* pToTopFrame = GetWindowPtr( hWnd );
     if( pToTopFrame && (pToTopFrame->mnStyle & SalFrameStyleFlags::SYSTEMCHILD) )
         BringWindowToTop( hWnd );
 
     if ( nFlags & SalFrameToTop::ForegroundTask )
     {
-        // This magic code is necessary to connect the input focus of the
-        // current window thread and the thread which owns the window that
-        // should be the new foreground window.
-        HWND   hCurrWnd     = GetForegroundWindow();
-        DWORD  myThreadID   = GetCurrentThreadId();
-        DWORD  currThreadID = GetWindowThreadProcessId(hCurrWnd,nullptr);
-        AttachThreadInput(myThreadID, currThreadID,TRUE);
-        SetForegroundWindow_Impl(hWnd);
-        AttachThreadInput(myThreadID,currThreadID,FALSE);
+        // LO used to always call AttachThreadInput here, which resulted in deadlocks
+        // in some installations for unknown reasons!
+        if (bEnableAttachThreadInputHack)
+        {
+            // This magic code is necessary to connect the input focus of the
+            // current window thread and the thread which owns the window that
+            // should be the new foreground window.
+            HWND hCurrWnd = GetForegroundWindow();
+            DWORD myThreadID = GetCurrentThreadId();
+            DWORD currThreadID = GetWindowThreadProcessId(hCurrWnd,nullptr);
+            AttachThreadInput(myThreadID, currThreadID, TRUE);
+            SetForegroundWindow_Impl(hWnd);
+            AttachThreadInput(myThreadID, currThreadID, FALSE);
+        }
+        else
+            SetForegroundWindow_Impl(hWnd);
     }
 
     if ( nFlags & SalFrameToTop::RestoreWhenMin )