Hooking Console Applications and the Desktop

• Microsoft Win32 Application Programming Interface (API) included with:

      - Microsoft Windows NT versions 3.5 and 3.51
      - Microsoft Windows 95 version 4.0
  

SUMMARY

Under Windows NT, system hooks are limited in two situations: hooking console windows and hooking the desktop.

Because of the current design of the console and the fact that its user interface runs in the Win32 server, Windows NT does not support hook calls in the context of console applications. Thus, if application A sets a system-wide input hook and text is typed in a console window, application A's input hook will not be called. The same is true for every type of Windows hook except for journal record and journal playback hooks.

Hooking a console application will be enabled in Windows NT 3.51.

MORE INFORMATION

Windows NT supports journaling by forcing the console input thread to communicate with the application that set the hook. In the case of a console, the call to the hook functions are run in the context of the application that installed the hook. This forces Windows NT to synchronously talk to this process in order for it to work; if this process is busy or blocked (as it is when it is sitting at a breakpoint), the console thread is hung.

If console applications were typical Win32-based applications, then this would not be a problem. A design change such as this would require that each console take an extra thread just to process input. This was not acceptable to the designers, and therefore console applications are not implemented in the same way that other Win32-based applications are implemented.

Similarly, if Windows NT allowed other hooks to freely hook any process, then these processes could enter a hanging state as well. The reason that journaling is allowed to hook consoles is that journaling already requires synchronization between all processes in the system, and a mechanism to disengage the journaling process (via the CTRL+ESC, ALT+ESC and CTRL+ALT+DEL keystroke combinations) is provided to prevent hanging the system message queue.

For similar reasons, 16-bit Windows-based applications cannot hook Win32-based applications under Windows NT.

The issues above apply equally well to hooking the desktop. If an application were allowed to hook the desktop, it could potentially hang it. This is completely unacceptable and violates one of the design principles of Windows NT: no application should be allowed to bring down the system or hang the user interface.