Disclaimer: I don’t know what I’m talking about, I’ve done little Win API (Win32) development and I only have a few years of Java development of which maybe 2 or 3 are developing desktop applications with JavaFX (Dashman being my first fully fledged out JavaFX app).

Disclaimer 2: I have only tested this on my own computer, running Microsoft Windows 10. I hope to soon test it in many others and over time we’ll see whether my solution was correct or not. I’ll update this blog post accordingly (or link to a newer version if necessary).

I started taking the quality of Dashman very seriously and one of the problems I found was that the running instances wouldn’t exit properly during uninstall or upgrades. And as I expected, this turned out into a head-bashing-into-brick-wall task. My solution was for a JavaFX app, but this should work for a Swing or any other kind of apps.

It all started with learning about Windows Restart Manager, something I didn’t know it even existed until a week ago. This is what allows Windows to close applications on uninstall, on reboots, etc. In the Guidelines for Applications, the crucial bit is this:

The Restart Manager queries GUI applications for shutdown by sending a WM_QUERYENDSESSION notification that has the lParam parameter set to ENDSESSION_CLOSEAPP (0x1). Applications should not shut down when they receive a WM_QUERYENDSESSION message because another application may not be ready to shut down. GUI applications should listen for the WM_QUERYENDSESSION message and return a value of TRUE if the application is prepared to shut down and restart. If no application returns a value of FALSE, the Restart Manager sends a WM_ENDSESSION message with the lParam parameter set to ENDSESSION_CLOSEAPP (0x1) and the wparam parameter set to TRUE. Applications should shut down only when they receive the WM_ENDSESSION message. The Restart Manager also sends a WM_CLOSE message for GUI applications that do not shut down on receiving WM_ENDSESSION. If any GUI application responds to a WM_QUERYENDSESSION message by returning a value of FALSE, the shutdown is canceled. However, if the shutdown is forced, the application is terminated regardless.

Simplifying it: when Windows needs your app to close, it will send a message asking if you are ready to close. Your application might respond negatively and then no application will be closed. This could happen for example if there’s some unsaved work and the app needs the consent from the user to either save or discard. This is what happens when you try to shut down your computer and Microsoft Word stops it asking whether you want to save the file or not.

After that your application can receive a message asking it to please close or telling it to close now. I’m not sure what the nuances are between these two. For Dashman I decided to just save the config and close in either of these instances.

Receiving these messages requires interfacing with Windows DLLs, for which I’m using JNA. I don’t know how JNA works, I read the code, sort-of understood it, copied and pasted it. What I think is going on is that you open the user32.dll like this:

User32 user32 = Native.loadLibrary("user32", User32.class, Collections.unmodifiableMap(options))

User32 is an interface that contains all the methods with the proper signatures to be able to call them from Java. options just makes sure we are using the Unicode version of the Win32 API calls. You can see that and all the other missing pieces on the full example at the end of the blog post.

I need a Win32 API callback that will receive the messages and actually implement the guidelines previously quoted:

StdCallLibrary.StdCallCallback proc = new StdCallLibrary.StdCallCallback() {
    public WinDef.LRESULT callback(WinDef.HWND hwnd, int uMsg, WinDef.WPARAM wParam, WinDef.LPARAM lParam) {
        if (uMsg == WM_QUERYENDSESSION && lParam.intValue() == ENDSESSION_CLOSEAPP) {
            return new WinDef.LRESULT(WIN_TRUE);
        } else if ((uMsg == WM_ENDSESSION && lParam.intValue() == ENDSESSION_CLOSEAPP && wParam.intValue() == WIN_TRUE) || uMsg == WM_CLOSE) {
            Application.exit();
            return new WinDef.LRESULT(WIN_FALSE); 
        }
        return user32.DefWindowProc(hwnd, uMsg, wParam, lParam);
 
    }
};

Oh! Lot’s of constants! What are they? I define them in the full example at the bottom of this post. They should be mostly self-evident what they stand for, their actual values are not that important.

Now things get tricky. Apparently Microsoft Windows send these messages to windows, not processes. Dashman can run in the tray bar, with no active window. And even if it had an active window, getting the HWND pointer for that window in JavaFX doesn’t seem trivial (I couldn’t get it to work). So, I create a size 0 invisible window to receive the message:

WinDef.HWND window = user32.CreateWindowEx(0, "STATIC", "Dashman Win32 Restart Manager Window.", WS_MINIMIZE, 0, 0, 0, 0, null, null, null, null);

Then I need to connect that window to the callback:

try {
    user32.SetWindowLongPtr(window, GWL_WNDPROC, proc);
} catch (UnsatisfiedLinkError e) {
    user32.SetWindowLong(window, GWL_WNDPROC, proc);
}

The callback is not magic though, and requires an event loop that will constantly check if there’s a message and trigger the processing when that happens:

WinUser.MSG msg = new WinUser.MSG();
while (user32.GetMessage(msg, null, 0, 0) > 0) {
    user32.TranslateMessage(msg);
    user32.DispatchMessage(msg);
}

Of course, that means you want this to run as its own daemon thread. The reason to make it a daemon thread is so that it won’t hang around preventing the JVM from exiting. 

One of my most useful sources of understanding and inspiration was the source code for Briar. I want to give credit where credit is due. I do think I spotted an issue with their source code in which they are not following the guidelines though. Also, they have a much more complex situation to handle.

And now, the full example with all my comments including links to more information explaining where all the values for constants and logic is coming from:

import com.sun.jna.Native;
import com.sun.jna.Pointer;
import com.sun.jna.platform.win32.WinDef;
import com.sun.jna.platform.win32.WinUser;
import com.sun.jna.win32.StdCallLibrary;
import com.sun.jna.win32.W32APIFunctionMapper;
import com.sun.jna.win32.W32APITypeMapper;

import java.util.Collections;
import java.util.HashMap;
import java.util.Map;

import static com.sun.jna.Library.OPTION_FUNCTION_MAPPER;
import static com.sun.jna.Library.OPTION_TYPE_MAPPER;

// Inspiration can be found at https://code.briarproject.org/akwizgran/briar
public class RestartManager {
    // https://autohotkey.com/docs/misc/SendMessageList.htm
    private static final int WM_CLOSE = 0x10; // https://msdn.microsoft.com/en-us/library/windows/desktop/ms632617
    private static final int WM_QUERYENDSESSION = 0x11; // https://msdn.microsoft.com/en-us/library/windows/desktop/aa376890
    private static final int WM_ENDSESSION = 0x16; // https://msdn.microsoft.com/en-us/library/windows/desktop/aa376889

    // https://msdn.microsoft.com/en-us/library/windows/desktop/aa376890
    // https://msdn.microsoft.com/en-us/library/windows/desktop/aa376889
    private static final int ENDSESSION_CLOSEAPP = 0x00000001;
    private static final int ENDSESSION_CRITICAL = 0x40000000;
    private static final int ENDSESSION_LOGOFF = 0x80000000;

    // https://stackoverflow.com/questions/50409858/how-do-i-return-a-boolean-as-a-windef-lresult
    private static final int WIN_FALSE = 0;
    private static final int WIN_TRUE = 1;

    // https://msdn.microsoft.com/en-us/library/windows/desktop/ms633591(v=vs.85).aspx
    private static final int GWL_WNDPROC = -4;

    // https://msdn.microsoft.com/en-us/library/windows/desktop/ms632600(v=vs.85).aspx
    private static final int WS_MINIMIZE = 0x20000000;

    public static void enable() {
        Runnable evenLoopProc = () -> {
            // Load user32.dll usi the Unicode versions of Win32 API calls
            Map<String, Object> options = new HashMap<>();
            options.put(OPTION_TYPE_MAPPER, W32APITypeMapper.UNICODE);
            options.put(OPTION_FUNCTION_MAPPER, W32APIFunctionMapper.UNICODE);
            User32 user32 = Native.loadLibrary("user32", User32.class, Collections.unmodifiableMap(options));

            // Function that handles the messages according to the Restart Manager Guidelines for Applications.
            // https://msdn.microsoft.com/en-us/library/windows/desktop/aa373651
            StdCallLibrary.StdCallCallback proc = new StdCallLibrary.StdCallCallback() {
                public WinDef.LRESULT callback(WinDef.HWND hwnd, int uMsg, WinDef.WPARAM wParam, WinDef.LPARAM lParam) {
                    if (uMsg == WM_QUERYENDSESSION && lParam.intValue() == ENDSESSION_CLOSEAPP) {
                        return new WinDef.LRESULT(WIN_TRUE); // Yes, we can exit whenever you want.
                    } else if ((uMsg == WM_ENDSESSION && lParam.intValue() == ENDSESSION_CLOSEAPP
                            && wParam.intValue() == WIN_TRUE) || uMsg == WM_CLOSE) {
                        Application.exit();
                        return new WinDef.LRESULT(WIN_FALSE); // Done... don't call user32.DefWindowProc.
                    }
                    return user32.DefWindowProc(hwnd, uMsg, wParam, lParam); // Pass the message to the default window procedure

                }
            };

            // Create a native window that will receive the messages.
            WinDef.HWND window = user32.CreateWindowEx(0, "STATIC",
                    "Dashman Win32 Restart Manager Window.", WS_MINIMIZE, 0, 0, 0,
                    0, null, null, null, null);

            // Register the callback
            try {
                user32.SetWindowLongPtr(window, GWL_WNDPROC, proc); // Use SetWindowLongPtr if available (64-bit safe)
            } catch (UnsatisfiedLinkError e) {
                user32.SetWindowLong(window, GWL_WNDPROC, proc); // Use SetWindowLong if SetWindowLongPtr isn't available
            }

            // The actual event loop.
            WinUser.MSG msg = new WinUser.MSG();
            while (user32.GetMessage(msg, null, 0, 0) > 0) {
                user32.TranslateMessage(msg);
                user32.DispatchMessage(msg);
            }
        };

        Thread eventLoopThread = new Thread(evenLoopProc, "Win32 Event Loop");
        eventLoopThread.setDaemon(true); // Make the thread a daemon so it doesn't prevent Dashman from exiting.
        eventLoopThread.start();
    }

    private interface User32 extends StdCallLibrary {
        // https://msdn.microsoft.com/en-us/library/windows/desktop/ms632680(v=vs.85).aspx
        WinDef.HWND CreateWindowEx(int dwExStyle, String lpClassName, String lpWindowName, int dwStyle, int x, int y, int nWidth, int nHeight, WinDef.HWND hWndParent, WinDef.HMENU hMenu, WinDef.HINSTANCE hInstance, Pointer lpParam);

        // https://msdn.microsoft.com/en-us/library/windows/desktop/ms633572(v=vs.85).aspx
        WinDef.LRESULT DefWindowProc(WinDef.HWND hWnd, int Msg, WinDef.WPARAM wParam, WinDef.LPARAM lParam);

        // https://msdn.microsoft.com/en-us/library/windows/desktop/ms633591(v=vs.85).aspx
        WinDef.LRESULT SetWindowLong(WinDef.HWND hWnd, int nIndex, StdCallLibrary.StdCallCallback dwNewLong);

        // https://msdn.microsoft.com/en-us/library/windows/desktop/ms644898(v=vs.85).aspx
        WinDef.LRESULT SetWindowLongPtr(WinDef.HWND hWnd, int nIndex, StdCallLibrary.StdCallCallback dwNewLong);

        // https://msdn.microsoft.com/en-us/library/windows/desktop/ms644936(v=vs.85).aspx
        int GetMessage(WinUser.MSG lpMsg, WinDef.HWND hWnd, int wMsgFilterMin, int wMsgFilterMax);

        // https://msdn.microsoft.com/en-us/library/windows/desktop/ms644955(v=vs.85).aspx
        boolean TranslateMessage(WinUser.MSG lpMsg);

        // https://msdn.microsoft.com/en-us/library/windows/desktop/ms644934(v=vs.85).aspx
        WinDef.LRESULT DispatchMessage(WinUser.MSG lpmsg);
    }
}

And now, my usual question: do you think this should be a reusable open source library? would you use it?