View source for Calling C functions from Rexx

''By [[Roger Orr]]''

==Introduction==
Most OS/2 programmers are by now aware of the existence of Rexx - a powerful, yet fairly simple, interpreted language which is shipped with OS/2. If you have never used Rexx I recommend having a go even if you only use it for writing smarter command files.

However, there are some restrictions with Rexx. Firstly, as it is interpreted, there are some things it does too slowly. Secondly there are OS/2 native APIs which are not available using the standard Rexx API; for example getting and setting thread priority.

One way to continue to use Rexx for overall program structure (where its ease of use and quick development turn around time are great benefits) is to make use of the 'external function' provision Rexx provides. This lets you write some code in a language such as C and call it directly from Rexx.

In addition if you use a visual Rexx tool you can write the user interface in Rexx (simple drag-n-drop development environment) and the underlying functionality in C (perhaps using company wide libraries implementing specific algorithms).

OS/2 is shipped with one example of such a library - the RexxUtil library. This contains functions such as 'SysFileTree' which lets a Rexx program obtain a list of files matching a selection of conditions. Many of you may have experimented with these functions and found that you can quite easily create relatively powerful programs to manipulate files and data. Once again, if you haven't done so may I encourage you to look at the on-line Rexx help and try the functions out.

It is however quite easy to write external functions yourself, and this technical tip describes how you might implement a simple 'sort' function.

==Overview of the method==
The Rexx interpreter uses 'stem' variables to provide a method to create a collection of related variables. A 'stem' variable consists of a basic name followed by a period and then the reference for the specific variable requested. The stem and the period alone refer to the whole item and allow the entire collection to be initialised to a common value. By convention simple arrays of variables are referred to using the index (1..n) and index 0 is used for the number of items in the array.

For example:
 capital. = ''     /* initialise array to empty strings */
 capital.0 = 3
 capital.1 = 'London'
 capital.2 = 'New York'
 capital.3 = 'Moscow'

We will use this common convention for our sort function and expect to be passed the name of the stem of the variables to sort - in this case 'capital'.

However we must do a little work in the Rexx command file before we can use an external function - we must tell the interpreter the name of the function and where to go to find it.

For this example I have called the function RexxSort, and placed it in a DLL (also called RexxSort) and exported it using the name RexxSort. So the Rexx command to load this external function is
 call RxFuncAdd 'RexxSort', 'RexxSort', 'RexxSort'

which saves you having to worry about which parameter is which (!) Once loaded the function is just called in exactly the same way as any of the intrinsic Rexx functions, for example:
 retval = RexxSort( "Capital" )
Note we put the stem name in quotes just in case there is a Rexx variable called capital.

At this point the Rexx interpreter will try to load the DLL RexxSort, locate the entry point RexxSort and call it. So it is time to look at the C code you need to write!

==External function interface==
Rexx uses a data type 'RXSTRING' to pass data to and from external functions. An RXSTRING contains a length and a string pointer - usually the string pointer refers to memory obtained with DosAllocMem() and freed with DosFreeMem(). A NULL pointer means an invalid string, a non-NULL pointer but a zero length means a zero length string and both non-zero means a valid string. The RexxSAA.h header file (supplied with the OS/2 toolkit) defines the data type and also provides some useful macros for accessing it.

Note that for convenience of C programmers RXSTRING variables passed to an external function will be NUL terminated - but you must bear in mind that Rexx strings can contain embedded NUL characters and so the string length is the definitive measure. (For simplicity the example code uses the C convention.)

So an external function called by Rexx is defined like this:
 RexxFunctionHandler ExtFunc; /* Declare ExtFunc as a Rexx function */
 
 ULONG ExtFunc( PUCHAR Name, ULONG Argc, PRXSTRING Argv,
                PSZ QueueName, PRXSTRING Retstr )
Beware - if you miss out the 'RexxFunctionHandler' line the compiler won't complain...but your function will fail because the compiler will not generate the correct linkage for the arguments you are passed.

The Name tells you what name the function was given by Rexx (can be useful for multiple functions using the same entry point).

Argc and Argv are Rexx-like equivalents to the standard C argument vector - except there is no entry for the program name. Rexx functions in general support variable numbers of parameters with sensible defaults for the absent parameters, so try and write your own functions the same way.

QueueName tells you the name of the current Rexx queue - which we shall ignore for now as the use of Rexx queues is outside the scope of this article.

Finally Retstr points to a Rexx string for the return code.

If the external function succeeds it returns 0. It should also fill in the return code by setting up the 'Retstr' RXSTRING pointer to a suitable string - in the example we return the number of valid strings which were sorted. If the return string is larger than the size of the Retstr supplied (256 bytes) you can allocate a larger string buffer with DosAllocMem.

If the external function returns non-zero the Rexx interpreter treats it as syntax error 40 ("Incorrect call to routine"); note that unless 'signal on syntax' has been specified such a failure of an external function will abort the command file!

==Some comments on the code==
The RexxSort function is defined as described above, and firstly checks that it has been called with 1 and only 1 parameter - the 'stem' name. It then calls the Rexx runtime function RexxVariablePool to obtain the value of the variable 'stem'.0 which we expect to contain the number of items to be sorted.

The API call used (RexxVariablePool) takes a linked list of 'SHVBLOCK' structures so we could fetch or set many variables at once. However by initialising our structure to zero we set the 'next' pointer to null and so do only one operation per call for clarity.

If this call fails or the 0th item is invalid we fail the function.

Memory (pSortBlock) is allocated and each of the items to be sorted is fetched. We specify a NULL string when fetching to force Rexx to allocate memory for the variable value since we don't know how long the value might be. For simplicity we retain this allocated memory until after the sort, but please note that given the fact that OS/2 uses PAGED memory each item to be sorted is using up as a minimum 4K bytes since each one is a separate memory object.

The qsort() C function is called to sort the strings which are sorted by the 'compfn' function using the standard C stricmp.

A complete solution might want to some clever country specific case matching and handle embedded NUL characters, but this is only a taster!

Once sorted we loop though again, resetting each variable if it has changed (this will slightly speed up sorting an already sorted array!)

Note we MUST remember to free the memory Rexx allocated for us or the caller (probably the command processor) will gradually accumulate memory.

==Creating the DLL==
A linker definition file is required, as shown. I simply used the ICC command line:
 icc /Ge- /wall+ppt- RexxSort.c RexxSort.def Rexx.lib
where:
 /Ge-       says we want to create a DLL
 /wall+ppt- warns about almost any 'dodgy' construct.
            REMEMBER to CHECK any warnings especially if your code
            does not work!!
We must also explicitly mention 'rexx.lib' in order to pick up the calls to the Rexx runtime library.

This should generate a file RexxSort.DLL.

Note: users of other OS/2 compilers will need to find out what the equivalent options are for their own environment.

==Test harness==
The simple Rexx command file 'testit.cmd' can be used to test that everything works. Firstly it sets up the external function and a simple array 'capital.' and then calls RexxSort on this.

If the call to RexxSort fails because the DLL cannot be found (Eg it is not on the LIBPATH) then the Rexx interpreter will generate error 43 (Routine not found).

Secondly the SysFileTree function (which is included in RexxUtil) is called to fill in a stem variable 'file.' with a list of filenames, which are then sorted before being printed. [Of course they are already sorted if you have an HPFS disk].

Finally we drop the function name from the interpreter's vocabulary.

NOTE however that Rexx will keep the DLL in use until the command shell in which you ran 'testit' has ended. When you are developing Rexx external functions this can be a pain!

==Conclusion==
Rexx is a powerful language for OS/2 command files (among other uses). It has also been designed to be extendable by writing external functions in C, and this is easy to do. The resulting combination is a very powerful way to program under OS/2 by linking together standard libraries of functions with a flexible interpreted language. I hope this article encourages some of you to take the first step in writing your own external functions for Rexx, or by adding a Rexx callable interface to DLLs which you may already have written.
 ------------------------------- RexxSort.c -------------------------------
 
 /*
 Simple RexxSort() function in C
 */
 
 #define INCL_RXFUNC               /* Defines for Rexx external functions     */
 #define INCL_RXSHV                /* Defines for Rexx variable pool i/f      */
 #include <rexxsaa.h> 
 
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 
 typedef struct _SORT              /* Private structure used for qsort()      */
   {
   LONG index;                    /* Original index in stem variable */
   RXSTRING value;                /* Actual value of variable */
   } SORT, *PSORT;
 
 int compfn( const void *p1, const void *p2 ) /* Compare function for qsort() */
   {
   if ( !RXVALIDSTRING( ((PSORT)p1)->value) )
      return 1;
   else if ( !RXVALIDSTRING( ((PSORT)p2)->value) )
      return -1;
   else
      return stricmp( RXSTRPTR( ((PSORT)p1)->value ),
                      RXSTRPTR( ((PSORT)p2)->value ) );
   }
 
 
 RexxFunctionHandler RexxSort;   /* Declare RexxSort as a Rexx function */ 
 
 ULONG RexxSort( PUCHAR Name, ULONG argc, PRXSTRING argv,
                PSZ QueueName, PRXSTRING Retstr )
   {
   LONG i = 0;                    /* Loop counter                            */
   SHVBLOCK shvblock = {0};       /* variable pool data structure            */
   char buff[ 256]  = "";         /* Local buffer for current stem var.      */
   PSZ pszStem = NULL;            /* name of Rexx stem variable              */
   PSORT pSortBlock = NULL;       /* pointer to sort area                    */
   LONG numVars = 0;              /* number of variables to sort             */
   LONG numValid = 0;             /* number of valid variables found         */
 
   Name = Name;                   /* This example doesn't use these two */
   QueueName = QueueName;
 
   /* Get stem name */
   if ( ( argc != 1 ) || !RXVALIDSTRING( argv[0] ) )
      return 1;
 
   pszStem = RXSTRPTR(argv[0]);
 
   /* Get # of items in string variable from 'stem.0' */
   MAKERXSTRING( shvblock.shvname, buff, sprintf( buff, "%s.%i", pszStem, i ) );
   MAKERXSTRING( shvblock.shvvalue, 0, 0 );
   shvblock.shvcode = RXSHV_SYFET;
 
   if ( RexxVariablePool(&shvblock) != 0 )
      return 1;
 
   if ( !RXVALIDSTRING( shvblock.shvvalue ) )
      return 1;
 
   numVars = atoi( shvblock.shvvalue.strptr );
   DosFreeMem( shvblock.shvvalue.strptr ); /* MUST free mem returned by REXX */
 
   if ( numVars )
      {
      pSortBlock = calloc( (size_t)numVars, sizeof( pSortBlock[0] ) );
      if ( pSortBlock == NULL )
         return 40;
 
      /* Loop to get value of every stem variable */
      for ( i = 0; i < numVars; i++ )
          {
          MAKERXSTRING( shvblock.shvname, buff, sprintf( buff, "%s.%i",
                        pszStem, i+1 ) );
          MAKERXSTRING( shvblock.shvvalue, 0, 0 );
          shvblock.shvcode = RXSHV_SYFET;
 
          RexxVariablePool(&shvblock);
          pSortBlock[i].index = i+1;
          pSortBlock[i].value = shvblock.shvvalue;
          }
 
      qsort( pSortBlock, (size_t)numVars, sizeof( pSortBlock[0] ), compfn );
 
      /* Rewrite changed variables */
      for ( i = 0; i < numVars; i++ )
          {
          if ( i+1 != pSortBlock[i].index )
             {
             MAKERXSTRING( shvblock.shvname, buff, sprintf( buff, "%s.%i",
                           pszStem, i+1 ) );
             shvblock.shvvalue = pSortBlock[i].value;
             shvblock.shvcode = RXSHV_SYSET;
 
             RexxVariablePool(&shvblock);
             }
          if ( RXVALIDSTRING( pSortBlock[i].value ) )
             numValid++;
          DosFreeMem( pSortBlock[i].value.strptr ); /* MUST free memory */
          }
 
      free( pSortBlock );
      }
 
   /* Return numValid to the caller */
   MAKERXSTRING( *Retstr, RXSTRPTR( *Retstr ),
                 sprintf( RXSTRPTR( *Retstr ), "%u", numValid ) );
   return 0;
   }

 ------------------------------- rexxSort.def -------------------------------
 
 LIBRARY INITINSTANCE
 
 DATA NONSHARED
 
 EXPORTS
 RexxSort

 ------------------------------- testit.cmd -------------------------------
 
 /* Simple test script for 'RexxSort' */ 
 
 call RxFuncAdd 'RexxSort', 'rexxsort', 'RexxSort'
 
 capital. = ''           /* initialise array to empty strings */
 capital.0 = 4
 capital.1 = 'London'
 capital.2 = 'New York'
 capital.3 = ''          /* Leave a gap to check invalid string handling */
 capital.4 = 'Moscow'
 
 call RexxSort "capital"
 
 do i = 1 to result
   say capital.i
 end
 
 call RxFuncAdd 'SysFileTree', 'RexxUtil', 'SysFileTree'
 call SysFileTree "*.*", "file", 'O'     /* only filenames are required */
 call RexxSort "file"
 do i = 1 to file.0
   say i || ':' file.i
 end
 
 call RxFuncDrop 'RexxSort'
 exit

Roger Orr 16 Jul 1994 

[[Category:Scripting_Articles]]