Feedback Search Top Backward Forward
EDM/2

Dos Development Tools Under OS/2

Written by Charles R. Oldham

 

Introduction

In this new age of interoperability, OS/2 has been billed by the marketing types as the Integrating Platform. "Run your Windows, Dos, and OS/2 applications on one machine! Preserve your existing investment in Dos and Windows software--upgrade only when you are ready!" they cry. Well, for a change, the marketing people are on the right track. OS/2 is indeed the Integrating Platform, but there are far more advantages for the developer than there are for the user.

For the first time developers have the ability to write software for two different operating systems and three different operating environments: Dos, Windows, and OS/2 PM (four if you count OS/2 text mode programs). And we can do it all from one machine that hosts a multitasking, multithreaded, protected-mode operating system. Unlike systems that support cross-compilers, we are also able to test these applications to a great extent, using the Dos and WinOS /2 subsystems of OS/2 2.x.

Developers are faced with a difficult question when a transition to a new operating system is possible:

  • How can I get the work done that I did before, but still utilize the full potential of the Operating System?

Frustration is a real possibility in a situation like this. The developer may discover after a few hours or days of programming that running a Borland IDE in a Dos fullscreen session is still a lot like running Dos!

Though we seem to have all of OS/2's power at our fingertips, having to use Dos tools to develop Dos programs limits us somewhat. Dos tools are not multitasking or multithreaded; they do not understand filenames that are not in the 8.3 format; interprocess communication is almost nonexistent, or at best limited due to the fact that Dos understands only the single process.

It is reasonable to conclude, therefore, that we would like to minimize our contact with Dos tools, and maximize the number of OS/2 based tools that we would use to get our work done. Unfortunately, some tools, like compilers, in tegrated development environments, and especially debuggers, are irreplaceable.

The question then becomes

  • How can I minimize my contact with Dos while still utilizing my irreplaceable tools?

DPipeLn

Origins, Usage, and Internal Structure

DPipeLn Origins

The utility called DPipeLn was created because when developing for Dos, I tended to find myself using a Dos box all day. This frustrated me because I felt like I wasn't using the abilities of OS/2. Also, though I like Borland's Integrated Development Environments, I find them somewhat cumbersome for large projects, and I wanted to be able to use the more advanced utilities like NMake, Gnu Make, and the other Gnu programming utilities that are rapidly being ported to OS/2. I thought about my requirements and realized that it was probably possible to write an OS/2 utility that would start a Dos session transparently, run the appropriate command, and pipe the results back into the OS/2 session.

This I did, and DPipeLn is the result. 


dpipeln [-v] [-f] [-sdos_settings_filename]
           [-ttemp_dir_name] dos_command
        -v: Start Dos session visible
        -s: Specify file to get Dos Settings from
        -f: Start Dos session in the foreground
        -t: Specify location of temporary files

Now it is possible to call any Dos command-line based tool from an OS/2 prompt, and to see the results as if it was executed as a native OS/2 command. Dos programs can become part of pipelines (though the program must be at the head of the command line because input is not fed to the Dos program), they can be called from OS/2 based Make utilities, even from IBM's WorkFrame/2!

DPipeLn even passes return codes back from a terminated Dos program. This is essential to allow make utilities to operate correctly.

DPipeLn Internals

DPipeLn was developed with the excellent EMX/GCC development system by Eberhard Mattes. It is a standalone program; it contains none of the library calls that require emx.dll or emxlibc.dll.

The executable itself and the kit that contains the files needed to build DPipeLn can be found in the Zips/ directory that accompanies this issue of EDM/2.

DPipeLn Program Flow

Here is what happens when DPipeLn is executed:

  1. Stdout is forced into binary mode.

  2. The command line is processed. Several variables that control program operation are set.

  3. The named pipe used for transferring the stdout (and stderr when 4Dos is in use) is created.

  4. The named pipe used for transferring the Dos program return code is created.

  5. The StartData structure, which is used by DosStartSession() OS/2 API function, is populated with the proper data.

  6. The auxiliary batch file is created. DosStartSession actually calls the Dos command processor and instructs it to execute this batch file. The contents will be explained when this function is reviewed.

  7. DPipeLn executes the DosStartSession call. The Dos program runs asynchronously.

  8. A separate thread is initiated that waits for the return code from the Dos program.

  9. DPipeLn connects to the Dos stdout pipe and prints out data that is fed by the Dos program through the pipe.

  10. We wait for the return code thread to complete. When it is complete, we know the Dos program is complete and we set DPipeLn's return code to be the same as the Dos return code.

  11. Malloc'ed memory is freed and the named pipes are disconnected.

  12. DPipeLn exits with the return code previously set.

DPipeLn.c

This portion of the article is intended to be an introduction to the relevant portions of the OS/2 API. If you are already familiar with the Named Pipe API, the Session API, and the Thread API, you will probably find it more worth your while to examine DPipeLn's source code directly to learn about the internals.

#defines, #includes, and Global Variables

DPipeLn.c 


#define INCL_DOSSESMGR    /* Session Manager values */
#define INCL_DOS          /* DOS Calls (not MS-DOS) */
#define INCL_DOSNMPIPES   /* Named Pipe Support */
#define INCL_DOSPROCESS   /* Process IDs */
#define INCL_DOSQUEUES    /* Queue support */

Since the OS/2 header files are so big, there are #ifdefs throughout them that allow you to tell the compiler only which sections you would like included. DPipeLn needs to use these sections. 


#define BUFSIZE 80     /* Pipe buffer size */
#define SETSIZE 4096   /* Dos settings maximum size */

Like the comments say, make symbolic constants for the named pipe buffer size, and the size of the Dos settings structure. 


#define new_string(x) ((char *)malloc((x) + 1))
#define freemem(x) if ((x) != NULL) free((x))

Macros to make string creation and memory release a little easier. 


#define USING_4DOS

When compiling the software, #define this if you are using the 4Dos command processor replacement. The reason for this will be discussed later in the article. 


#include <os2.h>
#include <stdio.h>
#include <string.h>
#include <ctype.h>
#include <stdlib.h>

#include the header files that we need.

The following are the function prototypes for the program. I'm including them here with a description of what they all do because I won't show and describe the internals of each function--only the interesting ones. 


PBYTE get_dos_settings(char *file);

Read in the Dos settings file. The Dos settings file lets the user describe the settings for the Dos session that the indicated program will be run in. The format of the settings file is documented later in this article. 


char *get_basename();

Creates a base name for the named pipes and auxiliary batch file out of the PID of the DPipeLn process. The function allocates a string which is freed at the termination of the program. 


char *make_npipename(char *PQbasename);

Takes the passed string and makes an appropriate name for the named pipe through which the Dos session will communicate with the OS/2 session. 


char *make_batname(char *base, char *tempdir);

Makes the auxiliary batch file name. The function returns a fully-qualified path. 


int build_batchfile(char *Batname,
                    char *Errpname,
                    char *Command);

Builds the auxiliary batch file. Returns FALSE if the build failed. 


void process_cmdline(int argc, char *argv[]);

Processes all the command line switches and sets appropriate global variables that control the program flow. 


void usage();

Prints a usage message, calls cleanup(), then exits. 


void cleanup();

Frees allocated memory and disconnects named pipes. 


char *get_exename(char *arg);

Strips the path off of argv[0] to make an executable name out of it. Returns a character string containing just the name by which DPipeLn was executed. The string is used when reporting errors. 


void setup_startdata();

DPipeLn calls the OS/2 API function DosStartSession() to create the Dos session in which the Dos command is run. DosStartSession() takes a struct called STARTDATA. This function puts the proper data in the struct. 


void transfer_data(HPIPE *piphand);

This function reads the data from the named pipe that is connected to the Dos programs stdout (and stderr if 4Dos is the command processor). 


void read_errpipe();

This function reads the data from the named pipe that is connected to the "echo" command in the auxiliary batch file. It runs in a separate thread and collects the return code from the Dos program. 


char *make_rcdname(char *base);

We need a name for the named pipe that transfers the return code from the Dos session. make_rcdname creates a character string that contains the name. 


char *get_tempdir();

Scans the environment looking for a suitable temporary directory name. This directory will hold the auxiliary batch file. get_tempdir() returns a character string with the dirname in it.

Now, global variables: 


PSZ    PgmTitle = NULL;    /* Title of window in which
                           Dos program will execute */

PSZ    PgmCmdLine = NULL;  /* Command line for Dos
                           program */
PSZ    PgmSettingsFile = NULL;  /* File containing DOS
                                settings */
PSZ    Exename = NULL;     /* DPipeLn Executable
                           name */
UCHAR  ObjBuf[100];        /* Object buffer.  Used
                           by DosStartSession */
PSZ    Command;            /* The actual DOS command
                           to be executed */

PSZ    Errpname;           /* Name of the pipe that
                           contains the return code */

HPIPE  Piphand, Rcdpipe;   /* Named Pipe handles */
int    Opt_visible;        /* Run DOS session
                           invisibly */
int    Opt_settings;       /* DOS session has a
                              settings file */
int    Opt_foreground;     /* Run session in
                           foreground */
char   *PQbasename;        /* Unique name for the
                              named pipe */
char   *Pname;             /* Fully qualified path
                              for named pipe */
char   *Batname;           /* Unique name of the
                              batch file */
ULONG  OwningPID;          /* PID of dpipeln.exe */

char   RCstring[BUFSIZE];  /* String to temporarily
                              hold ret code from DOS
                              program */

char   *Tempdir = NULL;    /* temporary directory
                              in which to put batch
                              file */

main()

On to the main program. I'll leave out the local variables and only mention them when pertinent. Also, I won't cover each and every statement. Most are documented thoroughly within the code if you are interested. 


USHORT main(int argc, char *argv[]) {

Standard stuff. 


if (_fsetmode(stdout, "b") == -1) {
  fprintf(stderr,
          "Could not set stdout to binary mode.\n");
  exit(-1);
}

Set stdout to binary mode. This is necessary because when stdout is in text mode, EMX converts \n to \r\n on output. This helps when porting applications from Unix to OS/2, but is not desired behavior here. 


PQbasename = get_basename();
Pname = make_npipename(PQbasename);
Errpname = make_rcdname(PQbasename);
Batname = make_batname(PQbasename, Tempdir);

Now we create unique names for the pipe through which the Dos program will send it's output, the pipe that will transfer the return code from the Dos program, and the auxiliary batch file. All the variables will contain fully qualified paths to the referenced resources. 


rc = DosCreateNPipe(Pname,
      &Piphand,
      NP_ACCESS_DUPLEX,
       NP_WAIT ||
       NP_TYPE_MESSAGE ||
       NP_READMODE_BYTE ||
       NP_UNLIMITED_INSTANCES,
      outbuffer, inbuffer, timeout);
rc = DosCreateNPipe(Errpname,
      &Rcdpipe,
      NP_ACCESS_DUPLEX,
       NP_WAIT ||
       NP_TYPE_MESSAGE ||
       NP_READMODE_BYTE ||
       NP_UNLIMITED_INSTANCES,
      outbuffer, inbuffer, timeout);

Important part--create the named pipes. The variables refer to the following: 


Pname, Errpname
  The names of the pipes.

Piphand, Rcdpipe

The OS/2 programming reference calls these "Pipe Handles". These were defined in the global variable section, and the API takes addresses to them. The API function then populates them with the proper data so they will refer to the named pipe. 


NP_ACCESS_DUPLEX

These will be duplex (two-way) pipes. For some reason, I was unable to get them to function as one-way pipes. 


NP_WAIT

Open the pipe in blocking mode, i.e. DosRead and DosWrite block if there is no data available. 


NP_TYPE_MESSAGE

Data is written to the pipe as a stream of messages, as opposed to a stream of bytes (NP_TYPE_BYTE). 


NP_READMODE_BYTE

Data is read from the pipe as a stream of bytes. 


NP_UNLIMITED_INSTANCES

There is no limit on the number of instances of this pipe. This parameter doesn't really matter, as we know there will always be no more than one of each of these pipes (since the names are unique). 


outbuffer, inbuffer, timeout

These local variables set the size of the output and input buffers for the pipes, and set the default amount of time DosWaitNPipe will wait for a pipe to become available. 


setup_startdata(&sd);

DosStartSession takes a special parameter--a pointer to what is called the StartData structure. It contains members that define the different characteristics of the new session.

setup_startdata()

Here is the definition of the StartData structure, taken from the EMX header files: 


typedef struct {
  USHORT Length;
  USHORT Related;
  USHORT FgBg;
  USHORT TraceOpt;
  PSZ    PgmTitle;
  PSZ    PgmName;
  PBYTE  PgmInputs;
  PBYTE  TermQ;
  PBYTE  Environment;
  USHORT InheritOpt;
  USHORT SessionType;
  PSZ    IconFile;
  ULONG  PgmHandle;
  USHORT PgmControl;
  USHORT InitXPos;
  USHORT InitYPos;
  USHORT InitXSize;
  USHORT InitYSize;
  USHORT Reserved;
  PSZ    ObjectBuffer;
  ULONG  ObjectBuffLen;
} STARTDATA;

void setup_startdata(STARTDATA *SD) {

setup_startdata() takes one parameter--a pointer to a STARTDATA struct. 


SD->Length = sizeof(STARTDATA);

First we set the length of the structure. Different values enable different properties--specifically a length of 30 or 32 will cause DosStartSession to leave the rest of the struct blank and allow the installation file to fill in appropriate fields. We need all the functionality, so we will initialize SD->Length to the actual size of the struct. 


SD->Related = SSF_RELATED_INDEPENDENT;

We want an independent session. Originally I had this set as SSF_RELATED_CHILD, because it seem to make sense that I would want DPipeLn to spawn a child session to run the Dos program in. Also, child sessions cause a Termination Queue to be created, which, for Dos sessions, contains the Dos program's exit code. Obtaining the exit code is then merely a matter of invoking the necessary functions from the queue API. However, the Toolkit reference says this about SSF_RELATED_CHILD:

Once a process has issued DosStartSession specifying a value of 1 for Related, no other process within that session can issue related DosStartSession functions until all the dependent sessions have ended.

And I found that that was true. When running IBM's WorkFrame/2 from the 11/93 PDK CD-Rom and running a make that involved a Dos compiler and DPipeLn, I could not start any other tool until the make was finished. WorkFrame would complain with error 452 "ERROR_SMG_NOT_PARENT". Unfortunately, SSF_RELATED_INDEPENDENT type sessions do not pass exit codes back to the parent. To circumvent that, DPipeLn creates a Dos batch file each time it is run. The batch file contains batch commands to run the Dos program, and then send the return code back through the return code pipe to the OS/2 session. This one of two reasons is why DPipeLn works better with 4Dos than the stock command.com--4Dos includes primitives for retrieving the return code from the previously run program. The batch file for command.com is several hundred lines of if statements. The second reason is that command.com makes no provision for redirecting stderr. Thus, you get stderr from the Dos process only if you are running 4Dos. 


SD->FgBg = Opt_foreground ? SSF_FGBG_FORE : SSF_FGBG_BACK;

Opt_foreground is a global variable set while processing the command line. It is true if the user specified on the command line that the Dos session should appear in the foreground. FgBg is the element of STARTDATA that controls that characteristic. 


SD->TraceOpt = SSF_TRACEOPT_NONE;

It is possible to start a session specifically for debugging--if so, TraceOpt is set to SSF_TRACEOPT_TRACE or SSF_TRACEOPT_NOTIFY. 


SD->PgmTitle = PgmTitle;

PgmTitle contains the title of the session window. 


SD->TermQ = 0;

If the Dos session was a child session, we would set TermQ equal to the queue handle. 0 indicates that there is no termination queue. TermQ is ignored for independent sessions anyway. 


SD->InheritOpt = SSF_INHERTOPT_PARENT;

This determines if the new session will inherit the parents environment or not. In this case, we want to. 


SD->SessionType = SSF_TYPE_WINDOWEDVDM;

Though the default is for the session to be invisible, we still want to set its type to a windowed VDM in case the user shows the window. 


SD->IconFile = 0;

There is no icon associated with this session. 


SD->PgmHandle = 0;

Do not use the installation file. I'm not sure what this means or how it works. The CP reference is vague on this parameter. 


SD->PgmControl = Opt_visible ? SSF_CONTROL_VISIBLE :
                               SSF_CONTROL_INVISIBLE;

Start the program as visible, or invisible depending on the command line argument. 


SD->InitXPos = 30;
SD->InitYPos = 40;
SD->InitXSize = 200;

Here the initial window coordinates and size are set. 


SD->Reserved = 0;

This is a reserved parameter. The CP reference says it must be 0. 


SD->ObjectBuffer = ObjBuf;

DosStartSession calls DosExecProgram to actually start the program. If DosExecPgm fails, the name of the object responsible is placed in this buffer. 


SD->ObjectBuffLen = 100;

This is just the size of the object buffer. 


SD->Environment = Opt_settings ?
                  get_dos_settings(PgmSettingsFile)
                  : 0;

DosStartSession allows Dos sessions to get their settings from a string of null-terminated strings. If the user specified a settings file on the command line, then get_dos_settings() loads that file into memory and Environment is assigned a pointer to the beginning of the set. The settings cannot be any larger than 4096 bytes. This is not documented in the CP Reference, I learned how to do it by examining the code from Norm Ross's StartD program. Thus, DPipeLn and StartD can use the same settings files. 


SD->PgmName = 0;

PgmName is the name of the executable that DosStartSession is running. If this is 0 (or NULL) the Session will invoke the a shell--either the one specified in CONFIG.SYS or the one in the settings file. 


PgmCmdLine = new_string(strlen(Batname) + strlen("/c "));
strcpy(PgmCmdLine, "/c ");
strcat(PgmCmdLine, Batname);
SD->PgmInputs = PgmCmdLine;

PgmInputs is the last of the STARTDATA elements. It points to the command line for the program being invoked. Since we are actually invoking a batch file that runs the desired executable, we need to run it through the shell, thus the /c.

StartD Reference

StartD is a small OS/2 utility written by Norm P. Ross ( npross@undergrad. uwaterloo.ca) that starts a Dos session from an OS/2 command line. It is a lot like the built-in start /dos , except it allows a settings file to be specified. The settings file uses the same strings found in a standard Dos settings box. Here's the settings file I use when spawning Borland C/C++ for Dos. 


EMS_MEMORY_LIMIT=128
XMS_MEMORY_LIMIT=8192
DPMI_DOS_API=ENABLED
DPMI_MEMORY_LIMIT=8
IDLE_SENSITIVITY=50

StartD can be found at ftp-os2.nmsu.edu in /pub/os2/2.x/sysutils/startd.zip.

main() continued

Now that the STARTDATA struct is set up, we can create the batch file, and start the new session. The session is started asynchronously, so program flow continues after the call. 


rc = build_batchfile(Batname, Errpname, Command);
rc = DosStartSession(&sd, &sess_id, &s_pid);

DosStartSession() will return the new session's ID in sess_id, and the appropriate process ID in s_pid.

Because DPipeLn might still be busy transferring stdout from the Dos program when the Dos program ends and sends its return code over the pipe, we need to wait for that return code in a separate thread. Starting new threads in OS/2 is easy--almost easier than under Unix: 


rc = DosCreateThread(&threadid, read_errpipe,
                     targ, tflags, tstack);

DosCreateThread() needs the following

read_errpipe
The name of the function to execute in the thread.

targ
The address of a parameter block to pass to the function. This allows any information to be passed into the thread in the form of a pointer to some storage.

tflags
If true, the thread starts right away. If false, the program must call DosResumeThread() to start the thread.

tstack
Specifies the amount of stack to give to the thread. The main program sets this at 4096 bytes.

DosCreateThread() returns the thread ID in threadid. 


rc = DosConnectNPipe(Piphand);

After creating the pipe, we need to connect to it. 


transfer_data(&Piphand);

And now we transfer data from it. transfer_data() loops until the Dos program closes the pipe (by exiting). The data is gathered out of the pipe by a DosRead() call and printed to the stdout of the OS/2 session. 


rc = DosWaitThread(&threadid, DCWW_WAIT);

This waits for the read_errpipe() thread to complete. When it is complete we know that the Dos program (actually the batch file) has reported the exit code. This is the easy way out--normally in a multithreaded program semaphores are used to insure that global variables are accessed only when the data in them is valid. 


sscanf(RCstring, "%d", &dosreturn);

Put the return code into the local dosreturn. The batch file transmits the code as ASCII text, so the sscanf is necessary to convert it to an integer. 


DosWaitChild(DCWA_PROCESS,
      DCWW_WAIT,
      &wp_results,
      &the_pid,
      s_pid);

This call may not be absolutely necessary, since the Dos program is started as an independent session. It blocks until the Dos session is complete. 


cleanup();

cleanup() frees allocated storage and closes the named pipes. 


return (USHORT)dosreturn;

Lastly, return the Dos exit code to the calling process.

DPipeLn Examples

Now that you understand how DPipeLn works, let's look at a couple of ways to use it.

Make and DPipeLn

Here is a makefile I use that utilizes DPipeLn. Note that GnuMake supports parallel makes, and since OS/2 never loads code more than once, parallel makes can be a real win even with Dos compilers. 


SHELL=c:/4os2/4os2.exe

# Makefile for the School Improvement Program.

OBJS=fileio.obj filewins.obj prutils.obj \
     si_chk0.obj si_chk1.obj
si_chk2.obj \

si_chk3.obj si_chk4.obj si_chk5.obj si_chk6.obj \
si_chk7.obj si_chk8.obj si_chk9.obj si_chka.obj \
si_chkb.obj si_chkc.obj si_chkd.obj si_chke.obj \
si_chkf.obj si_chkg.obj si_chkh.obj si_pg0.obj \
si_pg1.obj si_pg2.obj \

OBJS2=d:\dosapps\borlandc\lib\c0l.obj

CFLAGS=-ml -c -If:\tcxl55\inc

DEBUGLINK= /m /v

DEBUG=-v
%.obj : %.c
    echos $< ^ >> sources.rsp

sip.exe: preclean objects $(OBJS) sip.rsp

    dpipeln.exe -sbcc.ini tlink.exe $(DEBUGLINK) /c
@sip.rsp,sip,sip,tcxlbcl cl

objects: $(OBJS)
    dpipeln.exe -sbcc.ini bcc.exe $(DEBUG) $(CFLAGS)
                @sources.rsp

sip.rsp: makefile
    echo $(OBJS2) + > sip.rsp
    echo $(OBJS) >> sip.rsp

preclean:
    copy nul sources.rsp

clean:
    erase /q sip.exe sources.rsp sip.rsp $(OBJS)

The makefile builds a list of targets in the file sources.rsp, and then passes that to BCC as a response file. BCC only loads itself once, but you could speed up processing by splitting the object files up into two targets and compiling them in parallel.

WF/2 and DPipeLn

If you have IBM's WorkFrame/2 and have read its README file, you may remember that it says that you can use Dos hosted tools, but you cannot see their output. With DPipeLn, you can! Dos command-line tools become just like OS/2 tools under WF/2.

Under Language Profile Management, simply specify x:\path\dpipeln.exe as the compiler executable name. Then, under Compiler Options, type the acual name of the compiler, along with any command-line options, in the options dialog. Here's an example using Turbo Pascal 6.0 for Dos.

And in the Compiler Options dialog box: 


...

Conclusion

DPipeLn has been a very useful tool for helping to seamlessly integrate my Dos development utilities with OS/2 hosted utilities such as GnuMake, IBM WF/2, and others. Source and executable is available, as mentioned at the beginning of this article.

Comments, questions, and suggestions are encouraged. I may be reached at the address found at the top of this document.

Happy Hacking!