Communication Between OS/2 and WIN-OS/2 - Part 3

From EDM2
Jump to: navigation, search

by David Kenner

The previous two volumes of The Developer Connection News reviewed how an OS/2 process and a WIN-OS/2 process can communicate. This volume concludes the project by describing how a WIN-OS/2 process can talk to the same VDD, thus completing the communication loop.

VDD Initialization

To complete this loop, you must add several items to the original VDD that it needs during initialization. You need to register an entry point into which the WIN-OS/2 process can call. Do this using VDHRegisterAPI. This API lets us register a real-mode and protect-mode entry point in the VDD. The first problem that we will encounter is that we cannot call VDHRegisterAPI during initialization. It must be called within the context of a DOS session creation. To handle this, we will create a couple of user hooks that are called during DOS session creation. Add to the original initialization code the following calls:

  • VDM_CREATE, which tells the system to notify us during DOS session initialization.
  • VDM_TERMINATE, which tells the system to notify us during DOS session termination.

The following sample code is the updated Init section, with the new calls:

 if(!VDHRegisterVDD(VDDNAME,OS2Service,NULL))     {
/* Register a VDM termination handler entry point.              */
    if ((VDHInstallUserHook(VDM_TERMINATE, (PUSERHOOK)
VddTerminate)) == 0)
        return FALSE;             /* Return FALSE if VDH call failed */
/* Register a VDM creation handler entry point.                  */
    if ((VDHInstallUserHook(VDM_CREATE, (PUSERHOOK) VddCreate)) == 0)
        return FALSE;             /* Return FALSE if VDH call failed */

Sample Code 1. Updated Init section

Note: We specified the VDM_Terminate simply for the sake of consistency. We actually do not do any work in this entry point. When the DOS session terminates, all entry points that have been registered for this session automatically get deregistered.

DOS Session Creation

Now that we have our mechanism to get notified when the DOS session is created, we can make our call to VDHRegisterAPI. The VDD name should be the name that other processes know (this is not necessarily the file name of the VDD). This entry point gets registered with the kernel, so that any real- or protect-mode process that must communicate with the VDD can access it. Essentially, we associate the VDD name that we specify with a real- and protect-mode entry point. If you don't require both entry points, you can pass a NULL to the one that you don't use. The call for our registration follows:

      *Register our entry point for protect mode
      rc  = VDHRegisterAPI(VDD_NAME,VddRealEntry,VddDpmiEntry);

Sample Code 2. Registration call

Let's look at our entry point. Basically, it's big switch statement. The first parameter specifies the message type being sent; the rest of the protocol is up to you. The following sample code illustrates how to set up the entry point.

BOOL VddDpmiEntry(ULONG ulMessageId,PVOID pvDataIn,PVOID pvDataOut)
        BOOL fError  =FALSE;
            case WINOS2_INIT:
                fError = VddDpmiInit(pvDataIn,pvDataOut);
            case  WINOS2_REGISTERCALLBACK:
                fError = VddDpmiRegisterCallBack(pvDataIn,pvDataOut);
            case  WINOS2_POSTMESSAGE:
                fError = VddDpmiPostMessage(pvDataIn,pvDataOut);
            case  WINOS2_TERMINATE
                fError =  VddDpmiTerm(pvDataIn,pvDataOut);
                fError = TRUE;

Sample Code 3. Setting up the entry point

Note: This sample code works equally well for both the real- and protect-mode entry point.

The following messages will be used for this project:

  • WINOS2_INIT - Performs processing specific for this session. This is done by the WIN-OS/2 process.
  • WINOS2_REGISTERCALLBACK - Lets the WIN-OS/2 process pass into the VDD an entry point within the OS/2 application. WINOS2_REGISTERCALLBACK lets the VDD call back into the OS/2 process. The calling protocol between the VDD and the WIN-OS/2 process is up to you. Our program uses this message to notify the WIN-OS/2 process that a event has happened. Typically, this triggers the WIN-OS/2 process to perform an action.
  • WINOS2_POSTMESSAGE - Posts a message to the OS/2 process.
  • WINOS2_TERMINATE - Lets the VDD terminate any process- or session-specific information and frees up any memory that was allocated for this particular session.

Now, we can export our interfaces within the VDD to the WIN-OS/2 session and also do a callback to the WIN-OS/2 process.

Getting Your WIN-OS/2 Process to Talk to the VDD

During the WIN-OS/2 initialization, we will do an INT 2F call. INT 2F is a generic service provider. We will use this service to 1) determine whether or not we are running on OS/2 and 2) obtain the real- and protect-mode entry points.

Use the following sample code to determine whether or not we are running on OS/2:

assume   CSsion of OS/2 we are running
     jne       OS2Running,BX
;    RestoreRegs
     pop       DS
     pop       SI
     pop       DI
     pop       ES
OS2Test    Endp

Sample Code 4. Determining your environment

After you have determined that you are on OS/2, you need to get the address of the entry point that we previously registered with VDHRegister API. The INT 2F call takes two parameters. The first is the service code 4011. Place this in register ax. The other parameter is the actual name of the VDD. Upon a successful call to our service routine, we will have the address of our entry point returned to us. We now have a function pointer that we can use to call into our VDD. The call for the service routine, in assembler, follows:

GetVddApiEntry     Proc Far
;    Save our initial register state
     push      DS
     push      ES
     push      DI
     push      SI
     push      AX
     mov       AX,4011h
;Load DS mov       VDDProcAdr,ES
;    Restore our register state
     pop       DS
     pop       SI
     pop       DI
     pop       ES
     pop       DS
GetVddApiEntry   Endp

Sample Code 5. Obtaining the address of the entry points

After we have an entry point into the VDD, we want to give the VDD a way to be able to call back into the WIN-OS/2 process. Do this by calling into the service routine within the VDD with the message type set to WINOS2_ REGISTERCALLBACK. Our second parameter will be the local address of our routine in the OS/2 process.

Upon a successful return, we now have a two-way mechanism for communication. The WIN-OS/2 process can call into the VDD and the VDD can call back to the WIN-OS/2 process.

A Word About Context Issues

Let's take a look at what we need to do when the VDD needs to call back into the WIN-OS/2 process. First, do the setup before the call back. Start with the VdhArmStiHook. This will arm the hook for the call back. You need to make sure this is done only once; otherwise, the kernel will generate a panic. After we set the call back, we need to save the VDM register state. This must be done before our call because the call back function will alter the client register state. On the return, ensure that the client frame points to the correct cs, eip, and ds registers. After we are on the return side of our call back, we will restore the register state.

Note: When you prototype the call-back function in the WIN-OS/2 process, you need to specify the LOADDS for the call-back specifier. This forces the data segment register to get reloaded with the right value for the call back.

The sample code forces a one-to-one correspondence between the WIN-OS/2 process and the OS/2 process. When the WIN-OS/2 process sends the message data the call into the VDD uses the WINOS2_POSTMESSAGE message accompanied by the message data to send. The VDD makes a copy of the message, puts into its queue, clears the semaphore for the OS/2 process, and returns. Recall from our first article (see The Developer Connection News, Volume 2), by copying in the message data from a local address of each process when we are at task time lets the correct Local Descriptor Table (LDT) to be active at the right time.


You now have a working model that lets the OS/2 and WIN-OS/2 process send messages to each other, as well as display them to the console. You can expand on this simple model, so the messages can be broadcast to all processes that are registered or to a designated process within a group of registered processes.

The sample code is included on The Developer Connection for OS/2 CD-ROM. Play around with it; it's there for your use.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation