WorkPlace Shell Programming In Assembler/2

Original Work by Micho Durdevich

Introduction

In this article we are going to highlight the most important steps in creating WorkPlace Shell objects using the machine language paradigm. The only tools that are necessary to build the corresponding dynamic link libraries are: the Watcom assembler (WASM), the linker program (WLINK) and the resource compiler (RC). The IBM Toolkit/2 should be installed, too.

At a first sight it might appear silly to try WorkPlace Shell in assembler: As is well known, the OS/2 graphical user interface is built on completely object-oriented grounds of SOM by IBM (System Object Model). The WPS design is truly unique, and establishes a "universe of objects" on its own. Traditionally, it is thought that such object-oriented programming can be formulated only within appropriate so-called high-level languages (like C or C++), which somehow contain the philosophy of objects implicitly or explicitly built in their syntax and semantic rules. In our opinion, this view is entirely wrong: The object orientation has not much to do with the programming language, but with the general vision of the programming model.

As we already mentioned in the introduction to this series, assembly language is really interpretable as the highest level programming language, if we adopt the viewpoint of the language expressive power. Therefore, in principle it should be possible to express any idea written in a "high-level" language (like C or C++) directly in terms of assembler. Of course, such a conversion might not be nice or easy at all, and in order to get a meaningful assembly-level result it might be necessary to rewrite the entire execution environment ensuring the existence of the objects in question. As an example of such a poorly designed system from the assembly viewpoint, we can mention a charming Qt toolkit by Trolltech.

Fortunately, in the case of WPS and SOM for OS/2, thanks to a high internal simplicity and elegance of the API set, we are able to proceed directly with assembly language programming without having to change the internals of the universe of objects.

The structure of a typical WPS class library

Initialization Structures

At first, let us mention that every WPS class DLL should possess the following entry procedure declared as public and starting point:

public _dllentry
_dllentry proc
  mov        eax, 1
  ret
_dllentry endp

Next, there are initialization routines for global (class-level) and local (instance) attributes. Here is the corresponding code, concretised to the case the object in question is derived from WPDataFile class (taken from our first sample, see below). During its startup, WorkPlace shell will execute the {xxx}NewClass (local attributes) routine for every registered class. This routine, in its turn, calls the global "sister" routine. Global symbols are almost always prefixed with "M_".

public myObjectNewClass
myObjectNewClass proc near
   push      ebp
   mov       ebp,esp
   push      ecx

   push      0x00000002
   push      0x00000001
   call      WPDataFileNewClass
   
   push      0x00000002
   push      0x00000001
   call      M_myObjectNewClass
   
   push      dword [ebp+0xC]
   push      dword [ebp+8]
   push      offset class_information_object
   push      0x00000001
   call      somBuildClass
   add       esp, 0x00000020
   
   mov       somclass_id_obj, eax
   mov       ecx, num_parentmethods_obj
   
@obj_numresolve_loop:

   mov       eax, obj_parentmethods_table[ecx*4]
   push      ecx
   
   push      dword WPObjectClassData[eax]
   push      0x00000001
   push      dword myObjectExtraData
   call      somParentNumResolve
   add       esp, 0x0000000C

   pop       ecx
   mov       obj_parentmethods_table[ecx*4], eax

   loop @obj_numresolve_loop

   mov       eax, somclass_id_obj
   
   pop       ecx
   pop       ebp
   ret
myObjectNewClass endp

The global sister routine is a little simpler... Let us also observe that in the loops involving somParentNumResolve, we had to push/pop the loop counting ecx register. This was necessary because the register is not preserved across the call to this particular API. In general, return codes of OS/2 APIs are stored in the eax register, but many of the OS/2 APIs do not care much about other registers like ebx, ecx and edx. This property should be carefully taken into account. In future versions of eComStation, we will be systematically replacing the APIs so that all non-return-type registers are preserved across the OS calls (it is worth mentioning here that FreeBSD kernel fully complies with this important property).

public M_myObjectNewClass
M_myObjectNewClass proc
   push      ebp
   mov       ebp, esp
   push      ecx

   push      0x00000002
   push      0x00000001
   call      M_WPDataFileNewClass
   add       esp, 0x8

   push      dword [ebp+0xC]
   push      dword [ebp+8]
   push      offset class_information_global
   push      0x00000001
   call      somBuildClass
   add       esp, 0x00000010
   
   mov       somclass_id_cls, eax
   mov       ecx, num_parentmethods_cls
   
@cls_numresolve_loop:

   mov       eax, cls_parentmethods_table[ecx*4]
   push      ecx

   push      dword M_WPObjectClassData[eax]
   push      0x00000001 
   push      dword M_myObjectExtraData
   call      somParentNumResolve
   add       esp, 0x0000000C 
   
   pop       ecx
   mov       cls_parentmethods_table[ecx*4], eax

   loop      @cls_numresolve_loop

   mov       eax, somclass_id_cls
   
   pop       ecx
   pop       ebp
   ret
M_myObjectNewClass endp

Object & Class Data

Let us have a look at important data structures figuring in the initialization calls of the previous section. At first, we have cls_parentmethods_table and obj_parentmethods_table. These data structures are linear lists of dwords of the form

parent_wpsmethod    dd token_wpsmethod

At the beginning of each list, we have a static variable num_parentmethods_cls and num_parentmethods_obj respectively. During the class initialization, original token_wpsmethod values (from the class in which these methods are originally introduced) are replaced by the flat addresses of the corresponding parent methods. In this way, the selected parent methods become available to our objects.

Next interesting data structure is 8 bytes long and suffixed by ExtraData string. In the listings above, we have 2 of them: myObjectExtraData and M_myObjectExtraData. The first dword is reserved for the corresponding parent method table list address. It is filled during the processing of somBuildClass. The second dword is filled out with the address of the object data retriever routine with the help of which we can access global and local variables the object is using. A typical call to this routine would be:

   push      somSelf
   call      WPSObjExtraData[4]
   add       esp, 0x00000004
   
   mov       somThis, eax

After this, variable somThis contains the flat offset of the allocated global or local data. Almost always, the somSelf pointer is simply given by somSelf = dword [ebp+0x00000008]

Finally, the only exported data structures are global and local ClassData-dwords. They are filled with the addresses of the associated token method tables (during somBuildClass processing). This allows other objects to gain access to the specific methods of our object.

Object & Class Tables

There are 2 critical complex data structures that "coordinate" all aspects of a given WPS class (corresponding to global and local aspects, as always). We shall call them class_information_object and class_information_global. They are passed as parameters to somBuildClass calls. Both are incarnations of a fundamental SOMClassInformation structure, which is defined below. Non-applicable parameters are usually left zero, when instantiating the structure.

SOMClassInformation struct
   somVersion                dd 4
   numStaticMethods          dd 0       ; Number of fixed internal methods
   numStaticOverrides        dd 0       ; Number of static overrides
   numNonInternalData        dd 0
   numProcMethods            dd 0
   numVarArgsFuncs           dd 0
   majorVersion              dd 0
   minorVersion              dd 0
   instanceDataSize          dd 0
   numMaxMethods             dd 0
   numParents                dd 0
   of2ClassName              dd 0        ; 2-fold pointer
   of2ClassMeta              dd 0        ; 2-fold pointer
   implicitParentMeta        dd 0
   of3ParentName             dd 0        ; 3-fold pointer
   
   offClassData              dd 0        ; Offset to ClassData structure
   offExtraData              dd 0        ; Offset to ExtraData structure
   tblStatic                 dd 0        ; Offset to static methods table
   tblMethodOverrides        dd 0        ; Offset to method overrides
   nitReferenceBase          dd 0         
   datatokensInstance        dd 0        ; Datatokens for instance data
   arbitraryMembersCD        dd 0        ; Arbitrary ClassData members
   stubsVarArgs              dd 0        ; Varargs stubs
   classInitFunction         dd 0        ; Class init function
   alignementByte            dd 0        ; Desired byte alignement
   numDirectInitClass    dd 0xFFFFFFFF
   tblDirectInitClass    dd 0
   numGeneralMethods         dd 0
   methodTokens              dd 0
   protectedDataOffset       dd 0
   somSciVersion             dd 0
   numInheritedMethods       dd 0
   impInheritedMethods       dd 0       ; Inherited methods implementations
   numClassDataEntries       dd 0       ; Number of method entries in ClassData
   tblClassDataEntryNames    dd 0
   numMigratedMethods        dd 0
   impMigratedMethods        dd 0       ; Migrated methods implementations
   numInitializers           dd 0
   tblInitializers           dd 0       ; Pointers to initializers, in release order.
   directToSOMClass          dd 0
   dynamicallyComputed       dd 0
SOMClassInformation ends

In the above structure, we can see pointers to several other important data objects. At first, we see of2ClassName and of2ClassMeta, they have the 2-fold pointer form

class_name_off        dd flat:class_name           class_meta_off dd flat:class_meta
class_name db "the-name-of-the-class", 0           class_meta db "meta-class-name", 0

In the case of class_information_global, the field for of2ClassMeta is always zero (no meta^2)! Another interesting entry corresponds to of3ParentName. It is a 3-fold pointer, realized as follows (global/local):

parent_name_@ff dd flat:parent_name_off            parent_meta_@ff dd flat:parent_meta_off
parent_name_off dd flat:parent_name                parent_meta_off dd flat:parent_meta
parent_name db "the-name-of-the-parent", 0         parent_meta db "the-meta-parent_name", 0

Perhaps the most important SOMClassInformation entry is the pointer to method overrides table. This table has the form of the linear list of the pairs

dd  flat:@ff_parent_method_name
dd  flat:new_implementation_proc

where we have again nice 3-fold pointers

@ff_parent_method_name     dd flat:off_parent_method_name
off_parent_method_name     dd flat:str_parent_method_name
str_parent_method_name     db "classname::standard-wps-name", 0

and new_implementation_proc is the procedure that overrides the method parent_method_name.

Furthermore, let us examine another very important entry: tblStatic. It is a pointer to the table of static methods introduced by the given class. The static method table is a linear listing of the following 6-fold entries, one for each method:

   dd   flat:myobjectClassData[method_index]
   dd   flat:@ff_methodNameBase
   dd   flat:@ff_methodNameFull
   dd   flat:method_procedure
   dd   flat:method_redispatch
   dd   flat:method_applystub

where method_procedure is the procedure that implements the method, method_index is offset to the method entry in the ClassData structure, while @ff_methodNameBase and @ff_methodNameFull are nice 3-fold pointers entangled in the following structure (recommended, to avoid duplications):

   dd   str_methodNameFull: db "::myObject::"
   dd   str_methodNameBase: db "method_name", 0
   dd   off_methodNameBase  dd flat:str_methodNameBase
   dd   @ff_methodNameBase  dd flat:off_methodNameBase
   dd   off_methodNameFull  dd flat:str_methodNameFull
   dd   @ff_methodNameFull  dd flat:off_methodNameFull

The last 2 dwords in the above 6-fold method table entry point to associated redispatch and apply stubs procedures. In the simplest scenario, these fields should be 0xFFFFFFF and 0x0000000 respectively (no redispatches/apply stubs).

Related to static methods are also entries {numStaticMethods, numClassDataEntries, numMaxMethods}. When defining ClassData structures, enough space should be left to accommodate all method tokens, and the class information (the first entry).

Finally, let us observe that the field instanceDataSize determines the amount of memory reserved for object data. It is exactly this memory area which is getting mapped by calling WPSObjExtraData[4] with somSelf as the unique argument. As we already mentioned, the result is the value of the somThis pointer.

Example A: Quantum Rectangles

Object Description

In the accompanying sample code, we are presenting a simple yet sufficiently illustrative WPS object, based on the random-rectangles PM program (discussed in detail within the PM-assembling article). We are constructing a child of WPDataFile object, displaying randomly fluctuating rectangles as the default view. The random number generator routine is the same as the one used in the PM example, based on a powerful multiply-with-carry algorithm. However here we control the rectangles via a special timer (WinCreateTimer, WinSetTimer) while in the PM example it was a simple cyclic thread created by DosCreateThread. Our object also features:

• A special settings page, controlling the state of the rectangles system: speed, stop/go. The mentioned settings page is introduced by overriding wpAddBecomePage method;
• A possibility to save the state via wpSaveDeferred method;
• Modification of the object pop-up menu, so that the quantum rectangles view properly appears.

All the samples are available at our download section.

How To Compile

The creation of the class DLL is very simple, in 3 steps: assembling, linking and resource-compiling. Explicitly,

wasm qr.asm
wlink @qr
rc qr qr.dll

Here, the linking info is stored in the file qr.lnk and the resource info is within qr.rc file. Don't forget to check the size of the DLL :)

In order to use the newly created class, it is necessary to register it:

/* Registering qRectangle class */

   call RxFuncAdd 'SysLoadFuncs', 'RexxUtil', SysLoadFuncs'
   call SysLoadFuncs

   if SysRegisterObjectClass(qRectangle, "QR") then
   say "Okidoki!"
   else say "Oops, a problem :("

Object instances are created in the standard way. For example,

/* Creating an instance of qRectangle class */

   call RxFuncAdd 'SysLoadFuncs', 'RexxUtil', SysLoadFuncs'
   call SysLoadFuncs
   
   if SysCreateObject("qRectangle", "Quantum Rectangles", "<WP_DESKTOP>", "", "R") then
   say "Object created successfully!"
   else say "Oops, a problem :("

Connecting PM and WPS stuff

An interesting problem appears when interconnecting PM and WPS code. Since PM is not inherently aware of WPS object-related entities like somSelf and somThis, how to keep this information available to PM windows? The solution is to reserve the memory during the window creation, and then save the pointer to this reserved memory during the window initialization procedure.

More precisely:

• When calling WinRegisterClass, specify the amount of reserved memory for the WPS-related stuff.
• Create frame and client windows using WinCreateWindow function;
• When creating the client window, pass the pointer to a reserved memory location containing the somSelf pointer, as an appropriate argument to WinCreateWindow;
• During the processing of WM_CREATE message, setup a pointer to the reserved memory location, with the help of WinSetWindowPtr.

Here are relevant pieces of code corresponding to the outlined methodology. At first, window creation. This is a fragment of the initialization procedure specified in the object-specific (overridden) version of wpOpen (the creation of the window in which a quantum-fluctuating rectangles will be displayed).

   push      HWND_DESKTOP
   call      WinQueryAnchorBlock
   add       esp, 0x00000004
   mov       hab, eax

   push      0x00000040
   push      (CS_SIZEREDRAW+CS_SYNCPAINT)
   push      offset rectangles_procedure
   push      offset rectangles_window_class
   push      hab
   call      WinRegisterClass
   add       esp, 0x00000014

   push      0x00000000                 ; At first, creating the frame window...
   push      offset frame_ct_data
   push      0x00000020
   push      HWND_TOP
   push      0x00000000
   push      0
   push      0
   push      0
   push      0
   push      0
   push      offset qwindow_title
   push      WC_FRAME
   push      HWND_DESKTOP
   call      WinCreateWindow
   add       esp, 0x00000034
   mov       hwnd_frame, eax

   push      0x00000000                 ; Let us allocate some memory, for the use
   push      0x40                       ; of various object parameters, that
   push      somSelf                    ; are related to windows rectangles stuff.
   call dword os2_wpAllocMem
   add       esp, 0x0000000C            ; {somSelf, UseItem, ViewItem}
 
   mov       ebx, somSelf               ; Store somSelf at the beginning...
   mov       [eax], ebx
 
   mov dword [eax][0x4], USAGE_OPENVIEW
   mov dword [eax][0x8], 0              ; Eight bytes for the UseItem structure.
 
   mov dword [eax][0xC], Q_OPEN         ; ViewItem.view
   mov       ebx, hwnd_frame            ; ViewItem.handle
   mov       [eax+0x10], ebx
   mov dword [eax][0x14], 0             ; ViewItem.ulViewState
   mov dword [eax][0x18], 0             ; ViewItem.hwndCnr
   mov dword [eax][0x1C], 0             ; ViewItem.pRecord

   mov       client_data, eax

                                        ; Now we are creating the client window...
   push      0x00000000
   push      eax
   push      FID_CLIENT
   push      HWND_TOP
   push      hwnd_frame
   push      0
   push      0
   push      0
   push      0
   push      0
   push      0x00000000
   push      offset rectangles_window_class
   push      hwnd_frame
   call      WinCreateWindow
   add       esp, 0x00000034
   mov       hwnd_client, eax

Now, the fragment of the window WM_CREATE processing procedure.

   push      [ebp+0x00000010]
   push      0x00000000
   push      hwnd
   call      WinSetWindowPtr
   add       esp, 0x0000000C

Here the pointer to the reserved memory is passed as the third argument to the window procedure with arguments {hwnd, ulmsgid, mp1, mp2}, hence mp1=[ebp+0x00000010]. From this point on, the data will be easily accessible from any window subroutine, by calling WinQueryWindowPtr.

Examble B: Very Simple Object

It is one of the simplest possible WPS objects. A child of WPDataFile, with only 4 simple overrides: {wpclsQueryIconData, wpclsQueryTitle, wpclsQueryInstanceType, wpclsQueryDefaultView}. The library creation involves only assembling and linking, as there are no any resources defined.

Example C: A Dangerous Folder

In this example we construct a derived class qHole from WPFolder class. We override wpDrop method, in order to introduce a couple of new options, besides the standard drop behavior: Based on the value of an instance variable, the drop operation will

• Call the parent method (standard folder behavior);
• Erase the dropped object (and its element objects, if the dropped object is a folder type); * Erase only subobjects that are not of folder-type (leaving the "skeleton" of an initial folder);
• Allow entry to objects of qHole only

The instance variable is controlled via a special settings page, introduced on top of other settings pages by overriding wpAddSettingsPages method.

Our qHole object features two simple instance methods, qholeSetState and qholeGetState, controlling the above-mentioned variable.

In constructing this sample (available at our download section) we were inspired by a well known Black Hole class [4].

Here is the main destroyer procedure. It checks first the dropped object type, and if the dropped object is a folder then it would enter a recursive loop to handle the subobjects. In case of WPFileSystem objects, the procedure would reset the file attributes, before deleting. It would also reset the object flags in general, before invoking the (sub)object-specific version of wpFree (calculated via somResolve).

actual_delete proc                      ; ebx contains the skeleton/full-destroy
   push      ebp                        ; choice! The unique argument is the
   mov       ebp, esp                   ; object we are applying the procedure to.
   sub       esp, 0x00000010

   mov       [ebp-0x00000008], ebx

   push      WPFolderClassData          ; Let us first check to see if we deal 
   push      [ebp+0x00000008]           ; with folder objects...
   
   call      SOMObjectClassData[tok_somIsA]
   add       esp, 0x00000008
   test      eax, eax

   jz @ad_test4filesystem
 
   push      0x00000000                 ; If yes, fully populate the folder
   push      0                          ; so that we can examine its contents.
   push      0
   push      [ebp+0x00000008]
   call      WPFolderClassData[tok_wpPopulate]
   add       esp, 0x00000010
 
   test      eax, eax
   jz @ad_test4filesystem

   push      QC_FIRST                   ; Let us see if there is at least one
   push      0                          ; object in the folder.
   push      [ebp+0x00000008]
   call      WPFolderClassData[tok_wpQueryContent]
   add       esp, 0x0000000C

   mov       [ebp-0x00000004], eax
   test      eax, eax
   jz @ad_folder_done                   ; Folder empty => proceed further.

@ad_folder_loop:

   push      QC_NEXT
   push      [ebp-0x00000004]           ; <= We are referring to the current
   push      [ebp+0x00000008]           ; object in the folder contents list

   call      WPFolderClassData[tok_wpQueryContent]
   add       esp, 0x0000000C 
   mov       [ebp-0x0000000C], eax      ; Save the next object before deleting!

   mov       ebx, [ebp-0x00000008]
   push      [ebp-0x00000004]
   call      actual_delete
   add       esp, 0x00000004

   mov       eax, [ebp-0x0000000C]
   mov       [ebp-0x00000004], eax
   test      eax, eax
   jnz       @ad_folder_loop

@ad_folder_done:                         ; Checking for the skeleton mode...
   cmp dword [ebp-0x00000008], 2
   jz  @ad_exit

   push      0x00000000                 ; Once again, fully populate the folder
   push      0                          ; so that we can examine new contents.
   push      0
   push      [ebp+0x00000008]
   call      WPFolderClassData[tok_wpPopulate]
   add       esp, 0x00000010

   push      QC_FIRST                   ; Let us double-check to see if the
   push      0                          ; folder is really empty, if not it
   push      [ebp+0x00000008]           ; means an error occurred, so we quit!

   call      WPFolderClassData[tok_wpQueryContent]
   add       esp, 0x0000000C

   test      eax, eax
   jnz @ad_exit

   jmp short @ad_filesystem_ok

@ad_test4filesystem:

   push      WPFileSystemClassData
   push      [ebp+0x00000008]
   call      SOMObjectClassData[tok_somIsA]
   add       esp, 0x00000008
   cmp       eax, 0
   jz  @ad_nofilesystem

@ad_filesystem_ok:                       ; We are resetting the attributes so
                                         ; that the fileobject can be deleted.
   push      [ebp+0x00000008]
   call      WPFileSystemClassData[tok_wpQueryAttr]
   and       eax, 0xFFFFFFFEh

   push      eax
   push      [ebp+0x00000008]
   call      WPFileSystemClassData[tok_wpSetAttr]
   add       esp, 0x0000000C

@ad_nofilesystem:                        ; General style modification, before
                                         ; calling wpFree.
   push      0
   push      OBJSTYLE_NODELETE
   push      [ebp+0x00000008]
   call      WPObjectClassData[tok_wpModifyStyle]
   add       esp, 0x0000000C

   push      WPObjectClassData[tok_wpFree]
   push      [ebp+0x00000008]
   call      somResolve
   add       esp, 0x00000008

   push      [ebp+0x00000008]
   call      eax
   add       esp, 0x00000004

@ad_exit:
   mov       esp, ebp
   pop       ebp
   ret
actual_delete endp

The compilation goes in a straightforward way:

wasm qhole.asm
wlink @qhole
rc qhole qhole.dll

To play with the library, we have to register the class qHole and create its objects, for example using the appropriate REXX scripts.

Concluding Remarks

There is a lot of fun in constructing WPS objects in assembler. In the above discussed examples, we tried to emphasize the simple internal structure of objects, and therefore we have not always optimized the code for maximum performance (for example, by holding certain variables in registers instead of using memory). We also used the ebp-frame format for the majority of procedures, and stack space for procedure arguments... All 3 examples feature custom icons (standard and animation, in case of qHole). These icons are fixed by overriding class methods wpclsQueryIconData and wpclsQueryIconDataN. We decided to specify icons as resources from the main WPS library PMWP.DLL.

And no doubts, it takes more efforts to code a WPS library in assembler, than using pre-defined macros linked with the Interface Definition Language. However all the difficulties are non-essential, and there are quite non-trivial advantages in using our programming model:

• Complete control of objects behavior;
• The best possible optimization;
• Enhanced creativity;
• Removal of junk code;
• Deeper understanding of the WPS internals, and OS/2 in general.

In forthcoming articles, we shall discuss more complex situations, including sophisticated requester WPS objects from our UAME2 package for diskless remote-booting.

References

1. The Art of Assembly Language Programming and HLA. By Randall Hyde. An extensive and beautiful assembly language tutorial + related topics.
2. SOM Programming Reference and Guide. By IBM. Part of OS/2 Programming Toolkit {somguide.inf + somref.inf}.
3. WPS Programming Reference. By IBM. Files {wps1.inf + wps2.inf + wps3.inf} of OS/2 Programming Toolkit.
4. Black Hole WorkPlace Shell Class. By Gregory Czaja. Available at Black_Hole_0-3.zip.