Using a DLL for a PM hook

From EDM2
Jump to: navigation, search

By Roger Orr


Presentation manager provides a range of ways to obtain information about messages in the system. One of these ways is to use the 'hook' facility; which means a user provided function to be called by PM during its own processing.

A number of such hook points are provided, and the system can be asked to install a hook function for a specific message queue or for all message queues. In order for the latter case to work, you must provide the hook function in a Dynamic Link Library (which can be loaded into the memory space of each process being hooked) to allow the function to be called in the context of the appropriate application.

I thought that a simple example of one such hook might be instructive, both for itself and also to provide a few points of note about the writing of DLLs.

The Journal Record Hook

The hooks supported by OS/2 are passed a range of parameters; and some also provide a return code. In general any structures pointed to can be altered, and the return code can specify whether or not to continue with the default processing of the message.

For simplicity I decided to install a hook for what is known as the 'journal recording' hook. This hook is designed for the recording of user input (keyboard and mouse) and is often used in conjunction with the 'journal playback' hook for such things as automated testing. This is one of the simplest hooks - the function called returns VOID and no change to the behaviour of PM is supported - but the principles are generally valid for all hooks.

All this sample hook function does is to post on any WM_CHAR messages received to a client (using the UWM_JRN_MSG message number) which in this case is responsible for logging them to a window.

This simple application therefore provides a method of displaying in a window all the keystrokes received by PM. Note that we are NOT handling all the messages passed to the hook, which also receives mouse messages, WM_JOURNALNOTIFY messages and WM_VIOCHAR messages (for keystrokes in the OS/2 command prompt windows). This hook also gives us the keystrokes for the PM hotkey sequences, such as Ctrl+Esc, which are processed by PM and not passed on directly to an actual window procedure.

Our application's interface to the hook function is provided by two the functions JrnStart and JrnStop. Both take one parameter - the client window handle. To improve the resiliency of the DLL we use an exit list internally to ensure the client is tidied up if the application exits without first calling JrnStop.

The main application uses a simple fixed-font list box to display the received journal messages.

Notes on the code

The WinSetHook() API is used to set all the PM hooks. For global hooks it requires the module handle, and one clean way to provide that is to make the DLL responsible for providing it. It is helpfully provided by the initialisation call which is made by the operating system when the DLL is loaded and unloaded - see below for more about this.

JrnStop is called to explicitly remove the hook using WinReleaseHook, and to reset the internal data so another client can be accepted. (This code only supports one client at a time to keep it simple).

Whenever a WM_CHAR message is received (basically one for each key press and key release made) the data is posted to the client window. This window formats the data based on which fields in the input are valid and adds it to a list box. The client window is not visible - it is just used to handle messages and it to keep the size of its child window (the list box) in step with its own size.

The SWP_MOVE and SWP_SIZE flags are BOTH required on the WinSetWindowPos of the list box in the client window procedure (unless CS_SIZEREDRAW is specified) since PM will by default move the child window when the client is resized.

The fields in the character message I have decoded are the character and virtual key code (if valid), the flags word and a 2 letter mnemonic for each bit in the flag word - eg "KU" for KC_KEYUP. Most keys generate either a character (eg 'A') or a virtual key (eg 'PgUp') - some do not generate either (eg 'Ctrl') and some generate BOTH (eg 'Enter').

I have picked WinPostMsg to pass the character message to the application to try and remove the possibility of hanging the PM input processed by some sort of deadly embrace. If I were to use WinSendMsg the input of the keystroke would be held until the client window processed the message sent to it. Alternatively I could pass data using some linked list or array in the DLL itself. For this example I haven't address queue overflow, or what happens if the client window is destroyed without ending the owning application.

Notes about the code in the DLL

The DLL must be written to handle potentially multiple threads all trying to call JrnStart and JrnStop simultaneously. To ensure that only one at a time can access the global data we use a Mutex semaphore. This can be initialised during the function called by the operating system loader at library load and unload, which for the IBM C Set ++ compiler is called "_DLL_InitTerm", and will be called once for every process loading the DLL if we specify 'INITINSTANCE' in the linker file "Journal.DEF". If the handle is zero we create the semaphore, and if not we open it (to ensure that it is available to that process). (We are making use of the shared memory for hMutexSem to pass the semaphore handle between the two processes!)

I do not like using indefinite waits for semaphores since it is too easy to cause deadlock this way. So I have restricted the timeout to 1 second in JrnStop and JrnStart, and the API will pass this semaphore timeout to the caller if it occurs. The program could log an error and then exit gracefully since in this case a timeout indicates a problem.

If the input window's anchor block is destroyed before JrnStop is called to explicitly remove the hook and reset the internal data then PM will remove the hook as part of WinTerminate processing. However the journal DLL will not have cleared its internal data properly and this might stop the application being restarted successfully. To enable the DLL to sort itself out even in this case we add a tidy up function (JrnAbort) to the exit list for the application. This will give us a change to tidy up when the application exits. When the JrnStop API is called we can remove this exit function since the tidy up is no longer required.

An alternative inside a DLL is to make further use of the _DLL_InitTerm function - which is called when the DLL is unloaded as well as loaded. However, depending upon compiler options, etc., the C runtime may not be available during the termination function; exit list processing is done BEFORE this occurs and is therefore safer.

Notes about the data in the DLL, with the IBM compiler

We want all the global data in the module to be shared. This is the default for DLLs, but there are complications depending upon the particular choices made about compiler options and C runtime used.

The IBM C Set ++ compiler provides a number of choices for this. You may refer to the programming guide for a full list and description.

In my opinion there are two straightforward ways to go from the plethora of available options:

(a) Don't use the C runtime at all in your DLL, and compile with the /Rn option.

(b) Use the C runtime packaged as a DLL in its own right. The only problem with this one is that if you want to ship a DLL, or to run different levels of the compiler, you have to use DLLRNAME to rename the C runtime library.

This example is written to use (b) as illustration.

The three things required for this method to work are:

(i) default data must be non-shared

(ii) _CRT_init() must be called in the DLL initialisation function if you provide your own - the default one does this for you - and the DLL must be INITINSTANCE

(iii) compile with /Gd+m+ to get the multithreaded DLL C library

There are additional things you may need to do to get DLLs using C++ to work - see the IBM documentation.

So, firstly, all the global data in this module is to be shared, and so I use the IBM C Set ++ compiler pragma 'data_seg' to explicitly name the segment for three items to be shared. I then use the DATA NONSHARED directive in the linker definition file to make default data items non shared, i.e. one copy for each instance of the DLL loaded; and finally use the SEGMENTS directive to make the data in the 'global_data' segment shared. There are other ways to achieve the same result, but this way has the benefit that it is obvious from the code which data items are shared, with all other items non shared.

Secondly, as I am providing my own _DLL_InitTerm function I ensure it calls _CRT_init before anything else. In actual fact the C runtime DLL will call this anyway as part of its own initialisation but it is safer to put in an explicit call since we cannot force the order in which OS/2 will initialise DLLs.

Thirdly I use the right options in the makefile - and lastly check it all worked by running EXEHDR on the output DLL and looking for one data segment "READABLE, WRITEABLE, SHARED" and one only "READABLE, WRITEABLE".

If you look at the code in the Journal.c you will notice that it in fact doesn't call any C runtime functions - the only use of the C runtime is for the exception handling (see the next paragraph). So we could convert this particular DLL to method (a) as follows: - remove all the #pragma lines - remove the call to _CRT_init from _DLL_InitTerm - remove the DATA NONSHARED and SEGMENTS directive from the DEF file since all our data is to be shared - change /Gd+m+ to /Rn for the compile and link lines for Journal

The last detail I have added is an exception handler for the DLL to allow the C runtime used by the DLL to catch and process any traps or problems occurring during processing. In the IBM C Set ++ environment this is done when the compiler pragma 'handler' is invoked with the name of the function for which to set up an exception handler. A more advanced step would be write our own exception handler to attempt to tidy up - for example releasing the mutex semaphore.


target: Journal.dll Showchar.exe

Journal.obj : Journal.c Journal.h
        icc /Ge- /Gd+m+ /Wall+ppt- /Gs /c /Fo$@ $*.c >$*.err
        type $*.err 

Journal.dll : Journal.obj Journal.def
        icc /Ge- /Gd+m+ $** /B/NOE

Journal.lib : Journal.def
        implib $@ $**

ShowChar.obj : ShowChar.c Journal.h
        icc /c /Wall+ppt- $*.c >$*.err
        type $*.err

ShowChar.exe : ShowChar.obj ShowChar.def Journal.lib
        icc $**

APIRET APIENTRY JrnStart(HWND hwndClient);
#define UWM_JRN_MSG     WM_USER

#define INCL_BASE
#define INCL_WIN
#include <os2.h> 

#include "Journal.h"              /* Interface definition                    */

extern int _CRT_init( void );     /* C runtime initialisation function       */

#pragma data_seg( global_data )   /* All global data must be made shared     */

static HMODULE hJournal = 0;      /* our own module handle                   */
static HWND HookClient = 0;       /* Window currently getting data           */
static HMTX hMutexSem = 0;        /* To ensure multiple threads work OK      */

#pragma data_seg( )

/* Ensure externally callable functions use correct exception handler */
#pragma handler( JournalHookProc )
#pragma handler( JrnAbort )
#pragma handler( JrnStart )
#pragma handler( JrnStop )

/* _DLL_InitTerm is the function that gets called by the operating system    */
/* loader when it loads and frees this DLL for each process using it         */

   ULONG lRet = 1;                /* default non-zero implies success */

   if ( ulFlag == 0 ) /* If ulFlag is zero then the DLL is being loaded */
      hJournal = hModule;

      if ( _CRT_init() == -1 )    /* First initialise the C runtime          */
         lRet = 0;
      else if ( hMutexSem == 0 )
         if ( DosCreateMutexSem( NULL, &hMutexSem, DC_SEM_SHARED, 0 ) != 0 )
            lRet = 0;
      else if ( DosOpenMutexSem( NULL, &hMutexSem ) != 0 )
         lRet = 0;

   return lRet;

/* JournalHookProc: this function is called for each input message.  We      */
/* are only interested in WM_CHAR and ignore the others                      */

static void EXPENTRY JournalHookProc(
   HAB habInstall,
   PQMSG pQmsg )
   habInstall = habInstall; /* Keep compiler happy */

   if ( ( pQmsg->msg == WM_CHAR ) && ( HookClient ) )
      WinPostMsg( HookClient, UWM_JRN_MSG, pQmsg->mp1, pQmsg->mp2 );


/* JrnAbort: cope with abnormal process termination                          */
static VOID APIENTRY JrnAbort( ULONG ulTerm )
   ulTerm = ulTerm; /* Keep compiler happy! */

   /* PM will decently remove the hook when the process/window dies,
      but we must remove the data to free up for another client */
   HookClient = NULLHANDLE;

/* JrnStart: start journal recording for keystrokes - data will be posted    */
/* to the specified window.                                                  */

APIRET APIENTRY JrnStart( HWND hwndClient )
   APIRET rc = 0; 

   rc = DosRequestMutexSem( hMutexSem, 1000 );
   if ( rc == 0 )
      if ( HookClient != NULLHANDLE )
         HookClient = hwndClient;
         if ( ! WinSetHook( WinQueryAnchorBlock( hwndClient ), NULLHANDLE,
                    HK_JOURNALRECORD, (PFN) JournalHookProc, hJournal ) )
            rc = ERROR_ACCESS_DENIED;
            rc = DosExitList( EXLST_ADD, JrnAbort );
            if ( rc != 0 )
               JrnStop( HookClient );
      DosReleaseMutexSem( hMutexSem );

   return rc;

/* JrnStop: stop recording keystrokes                                        */

APIRET APIENTRY JrnStop( HWND hwndClient )
   APIRET rc = 0; 

   rc = DosRequestMutexSem( hMutexSem, 1000 );
   if ( rc == 0 )
      if ( HookClient != hwndClient )
         WinReleaseHook( WinQueryAnchorBlock( hwndClient ), NULLHANDLE,
                      HK_JOURNALRECORD, (PFN) JournalHookProc, hJournal ); 

         DosExitList( EXLST_REMOVE, JrnAbort ); 

         HookClient = 0;

      DosReleaseMutexSem( hMutexSem );

   return 0;
   global_data  CLASS 'DATA' SHARED


#define INCL_BASE
#define INCL_PM
#include <os2.h> 

#include <stdio.h>
#include <string.h>

#include "Journal.h"

#define ID_LIST 100

/* space + 2 chars for each bit in the WM_CHAR 'fs' field - see PMWIN.H */

/* Process WM_CREATE by creating a list box for the output messages          */

static void DoCreate( HWND hwnd )
   char szFont[] = "10.Courier";
   HWND hwndList = WinCreateWindow( hwnd, WC_LISTBOX, NULL,
              0, 0, 0, 0, hwnd,
              HWND_TOP, ID_LIST, NULL, NULL );
   WinSetPresParam( hwndList, PP_FONTNAMESIZE,
                  sizeof( szFont ), szFont );
   WinSendMsg( hwndList, LM_INSERTITEM, 0, "Chr Vkey Bits" SZFLAGS );

/* DoChar: add unpacked WM_CHAR fields into the list box at the top          */

static void DoChar( HWND hwnd, PCHRMSG pChar )
   char szBuff[ 120 ] = "";
   char *ptr = szBuff;
   int i = 0; 

   if ( ( pChar->fs & KC_CHAR ) /* && ( isprint( pChar->chr ) ) */ )
      ptr += sprintf( ptr, "'%c'", pChar->chr );
      ptr += sprintf( ptr, "   " );

   if ( pChar->fs & KC_VIRTUALKEY )
      ptr += sprintf( ptr, " %4u", pChar->vkey );
      ptr += sprintf( ptr, "     " ); 

   ptr += sprintf( ptr, " %4.4x",  pChar->fs ); 

   memset( ptr, ' ', 3*16 );
   ptr[3*16] = '\0';

   for ( i = 0; i < 16; i++ )
      if ( (USHORT)( 1 << i ) & pChar->fs )
         ptr[(i*3)+2] = 'X';

   WinSendDlgItemMsg( hwnd, ID_LIST, LM_INSERTITEM,
          MPFROMSHORT( 1 ), szBuff );

/* Window procedure                                                          */

MRESULT EXPENTRY ClientWndProc ( HWND hwnd, ULONG msg, MPARAM mp1, MPARAM mp2 )
   MRESULT mr = 0;

  switch (msg)
     case WM_CREATE:
        DoCreate( hwnd );

     case WM_SIZE:     /* keep list box same size as client */
        WinSetWindowPos( WinWindowFromID( hwnd, ID_LIST ),
               0, 0, 0, SHORT1FROMMP( mp2 ), SHORT2FROMMP( mp2 ), SWP_SIZE );

     case WM_USER:
        DoChar( hwnd, CHARMSG( &msg ) );

        mr = WinDefWindowProc (hwnd, msg, mp1, mp2) ;

  return mr;

/* M A I N   P R O C E D U R E                                               */

int main ( void )
  static CHAR  szClientClass [] = "ShowChar" ;
                              FCF_MINMAX  | FCF_SHELLPOSITION | FCF_TASKLIST;
  HWND         hwndFrame = NULLHANDLE, hwndClient = NULLHANDLE;
  QMSG         qmsg = { 0 };
  HAB hab = WinInitialize( 0 );
  HMQ hmq = WinCreateMsgQueue( hab, 0 );

  WinRegisterClass( hab, szClientClass, ClientWndProc, CS_SIZEREDRAW, 0 );

  hwndFrame = WinCreateStdWindow ( HWND_DESKTOP, WS_VISIBLE, &flFrameFlags,
                 szClientClass, "Show Characters", 0L, 0, 0, &hwndClient );

  if ( hwndFrame )
     JrnStart( hwndClient );

     while( WinGetMsg( hab, &qmsg, NULLHANDLE, 0, 0 ) )
          WinDispatchMsg( hab, &qmsg );

     JrnStop( hwndClient );

     WinDestroyWindow( hwndFrame );

  WinDestroyMsgQueue( hmq );
  WinTerminate( hab );

  return 0 ;

DESCRIPTION    'Show characters as typed'


Presentation Manager gives you a flexible and relatively simple way to add function to the various places where hooks are provided. This makes debugging, understanding and testing applications easier since the input data and messages can be captured and examined; and also make it possible to actually modify the behaviour of PM - for example using an 'input' hook.

The IBM compiler provides a large number of ways of writing and packaging dynamic link libraries - probably more than you want!

Once you have an example of an application and DLL demonstrating one such hook I hope you will be encouraged to experiment further.