Ensure Windows DDE conversations are finished in open_link

Use `ShellExecuteExW` instead of `ShellExecuteW` to pass the additional flag `SEE_MASK_NOASYNC`. This should ensure that DDE (Dynamic data exchange) conversations are finished before the function returns, as it would otherwise be necessary to manually pump messages on the calling thread. DDE conversations may be initiated when shell extensions are loaded via `ShellExecute`. Failing to answer the DDE messages will block the calling UI thread and may cause a dead-lock.

The flag `SEE_MASK_FLAG_NO_UI` is used to prevent native error message popups when the link/file cannot be opened.

Additionally, the `ShellExecute` verb is changed from `"open"` to `NULL`, which will cause the default shell handler to be used instead, as the `"open"` verb may not be available for all targets. Though for most targets it will be the default verb anyway.

References:

- https://learn.microsoft.com/en-us/windows/win32/api/shellapi/nf-shellapi-shellexecutew
- https://learn.microsoft.com/en-us/windows/win32/api/shellapi/ns-shellapi-shellexecuteinfow
- https://devblogs.microsoft.com/oldnewthing/20070430-00/?p=27063
- https://learn.microsoft.com/en-us/windows/win32/dataxchg/about-dynamic-data-exchange

Yet another attempt at solving #5744.
This commit is contained in:
Robert Müller 2022-10-11 21:13:37 +02:00
parent 4bb549b68e
commit c68841a733

View file

@ -3910,11 +3910,26 @@ int open_link(const char *link)
#if defined(CONF_FAMILY_WINDOWS)
WCHAR wBuffer[512];
MultiByteToWideChar(CP_UTF8, 0, link, -1, wBuffer, std::size(wBuffer));
SHELLEXECUTEINFOW info;
mem_zero(&info, sizeof(SHELLEXECUTEINFOW));
info.cbSize = sizeof(SHELLEXECUTEINFOW);
info.lpVerb = NULL; // NULL to use the default verb, as "open" may not be available
info.lpFile = wBuffer;
info.nShow = SW_SHOWNORMAL;
// The SEE_MASK_NOASYNC flag ensures that the ShellExecuteEx function
// finishes its DDE conversation before it returns, so it's not necessary
// to pump messages in the calling thread.
// The SEE_MASK_FLAG_NO_UI flag suppresses error messages that would pop up
// when the link cannot be opened, e.g. when a folder does not exist.
// The SEE_MASK_ASYNCOK flag is not used. It would allow the call to
// ShellExecuteEx to return earlier, but it also prevents us from doing
// our own error handling, as the function would always return TRUE.
info.fMask = SEE_MASK_NOASYNC | SEE_MASK_FLAG_NO_UI;
// Save and restore the FPU control word because ShellExecute might change it
unsigned oldcontrol87 = _control87(0u, 0u);
int status = (uintptr_t)ShellExecuteW(NULL, L"open", wBuffer, NULL, NULL, SW_SHOWDEFAULT) > 32;
BOOL success = ShellExecuteExW(&info);
_control87(oldcontrol87, 0xffffffffu);
return status;
return success;
#elif defined(CONF_PLATFORM_LINUX)
const int pid = fork();
if(pid == 0)