View source for The Shell Game - Alternative Shells for OS/2

''by [[Monte Copeland]]''

During OS/2's six-year history, IBM has shipped three different shells for OS/2. OS/2 Version 1.0 had a character shell called the Program Selector. The shell for OS/2 versions 1.1, 1.2, and 1.3 was actually a suite of Presentation Manager (PM) applications for program launching, spooler management, file management, and system control. OS/2 2.0 premiered the Workplace Shell, the first implementation of Common User Access (CUA) '91, and the shell for current and future versions of OS/2.

==Why Use a Different Shell?==
With an easy-to-use GUI such as the Workplace Shell, why would you choose to use a different shell? Admittedly, using a different OS/2 shell is not for everybody. But some applications might benefit from a different shell: bulletin board systems, specialized servers, embedded systems, UNIX alternatives. The designer of such systems will forego an easy-to-use interface to regain system resources such as memory and thread slots. On the other hand, secure systems might require more shell, not less.

This article provides some shell alternatives for OS/2. One alternative is to subclass Workplace Shell objects to provide access control. Another removes Workplace Shell, but leaves PM. The last two shells completely remove PM, providing a low-resource, text-only shell.

==Shell Access Control==
''"I want to stop users from opening a command prompt."'' This quote is typical of MIS managers who wish to restrict users' access. Because the Workplace Shell is based on object-oriented programming, you can invoke existing methods or subclass Workplace Shell objects and change their behavior.

A logical first step is to password-protect folders. Subclass the <tt>WPFileSystem</tt> class, override the <tt>_wpOpen</tt> method, and present a password dialog box before invoking <tt>_wpOpen</tt> in the parent. The International Technical Support Center (ITSC) Red Books ''OS/2 Workplace Shell Exploitation'' (GG24-4201) and ''OS/2 Version 2.0 Volume 4: Application Development'' (GG24-3774) contain some sample code on how to do this.

DeskMan/2 from Development Technologies, Inc. (803-790-9230) already does this and more. The PSP Application Assistance Center in Basingstoke, UK (+44-256 -344530) has a restricted-shell product available in most European countries.

==The PROTSHELL Statement in CONFIG.SYS==
All OS/2 shells have one thing in common: The name of the shell program appears on the PROTSHELL line in the ''config.sys'' file. This name must be an OS/2 protected-mode executable program (''.exe''). The ''.exe'' named on PROTSHELL is the first process spawned after system initialization (SYSINIT).

If the PROTSHELL executable is not a PM application and does not call <tt>WinCreateMsgQueue</tt>, then it will operate in a single-session, single-console, protect-mode-only, text-only environment. It can spawn other protected-mode, non-PM processes in the same session, but the operating system will not manage access to the console. A process can detach any process that does not require a console.

If the PROTSHELL executable is a PM application (it calls <tt>WinCreateMsgQueue</tt>), the session manager is initialized. That process has the distinction of being the session manager process. You can replace the PM shell this way, but the RUNWORKPLACE architecture is preferred.

==The RUNWORKPLACE Architecture==
In ''config.sys'', the environment variable RUNWORKPLACE names an executable. When ''pmshell.exe'' is the PROTSHELL (as it usually is), it will spawn the program named by RUNWORKPLACE. Usually, the RUNWORKPLACE program is also ''pmshell.exe'', and this is how Workplace Shell starts. By naming a different executable on the RUNWORKPLACE setting, you can start a different shell program.

This architecture is perfect for turnkey applications. You write a program, name it on the RUNWORKPLACE setting, and you have turn-it-on-and-use-it dedicated machine.

Some things work as expected: Presentation Manager is initialized, '''Ctrl-Esc''' presents the Window List, '''Alt-Esc''' switches sessions, REXX works, and ''startup.cmd'' starts. But the RUNWORKPLACE process is responsible for the rest: Starting and managing the spooler, installing PM printer drivers, presenting and starting programs, and providing a safe shutdown.

In this environment, OS/2 text-only applications, OS/2 Presentation Manager applications, DOS applications, and Windows applications run fine. Because there is no Workplace Shell process, you must test Workplace Shell-aware applications to see if they work; chances are they will. However, DLL-only Workplace Shell applications will not run because they expect ''pmshell.exe'', running as the RUNWORKPLACE process, to load them.

==MShell: A Mini PM Shell==
Volume 3 of The Developer Connection for OS/2 CD-ROM contains a sample program called MShell that performs the minimum requirements for a RUNWORKPLACE shell. MShell is a mini PM shell that basically is a program launcher. See ''Figure 1''.

[[Image:Shell-Fig-1.gif]]

''Figure 1. MShell: A Mini PM Shell for OS/2''

MShell calls an Original Equipment Manufacturer Interface (OEMI) API SplQmInitialize to start the spooler. Because the function prototype is missing from The Developer's Toolkit for OS/2 2.1, ''Code 1'' shows you how to provide your own.
 VOID APIENTRY SplQmInitialize(PULONG);
''Code 1. Prototype for function to start the spooler.''

To resolve the reference to <tt>SplQmInitialize</tt> at link time, import it from ''pmspl.dll'' in your ''.def'' file. See ''Figure 3''. At runtime, call <tt>SplQmInitialize</tt> with a pointer to a ULONG. On return from a successful spooler start, expect the ULONG to contain zero. MShell uses the normal spooler APIs to manage the spooler including <tt>SplEnumQueue, SplHoldQueue, SplReleaseQueue, SplHoldJob, SplReleaseJob,</tt> and <tt>SplDeleteJob</tt>.
 IMPORTS
  SPLQMINITIALIZE = PMSPL.SPL32QMINITIALIZE
''Code 2. Module definition file statements to resolve the external SplQmInitialize at link time.''

Listing and starting programs is easy to do. MShell accepts program start information from a plain-text ''.ini'' file and presents the programs in a listbox. When the user selects a program, MShell spawns a background copy of ''cmd.exe'' to do the start. Starting DOS sessions with settings is tougher; the utility ''startdos.exe'' does that. To shut down, call <tt>WinShutdownSystem</tt>.

Many MShell users have 4MB machines with slow hard files. They perceive increased performance, but at the cost of usability. The reasons that a RUNWORKPLACE shell like MShell gets better performance involve the hard file because 1) MShell allocates 1.5MB less swappable memory than the Workplace Shell, so there is less paging to disk and 2) MShell does not use file extended attributes. Interestingly, MShell on unattended servers has little effect on performance; the system pages unused user-interface code to disk no matter what shell is running.

==CMD.EXE on the PROTSHELL==
One interesting OS/2 configuration involves naming ''cmd.exe'' as the PROTSHELL process. Because ''cmd.exe'' does not initialize the session manager, the machine will have a single, protected-mode, full-screen session running ''cmd.exe''. Attempts to start more sessions will fail.

The best reason to name ''cmd.exe'' on the PROTSHELL is to save time booting OS/2 from floppies. And, the best reason to boot from floppies is to run ''chkdsk.com''.

(To make bootable OS/2 2.1 floppies, make copies of the OS/2 Installation Diskette and Diskette 1. From Diskette 1, delete the following files: ''bundle'', ''sysinst2.exe'' and ''fdisk.com''. From Diskette 2, copy ''uhpfs.dll'' and ''chkdsk.com'' to Diskette 1. Finally, edit ''config.sys'' on Diskette 1 and name ''cmd.exe'' on the PROTSHELL. For a single-floppy boot diskette, see ''bootos2.exe'' on your accompanying Developer Connection for OS/2 CD-ROM. On CompuServe, type '''Go OS2USER''' at the ! prompt, and see ''boot2x.zip'' in Library 17.)

For embedded OS/2, name your ''.exe'' on PROTSHELL. Inexpensive Intel 80386 planar boards running OS/2 can make a multiprocess, multithreaded control system. Granted, OS/2 is not ROM-able in the traditional sense; it requires a solid-state, ROM-based floppy device from which to boot. Such a system would operate without paging (MEMMAN=NOSWAP) and without the latency of paging. For example, a 4MB system has roughly 2MB free in this configuration.

Using ''cmd.exe'' on the PROTSHELL gives you a text-only OS/2, but without multiple sessions. 

==TShell: A Text Shell For OS/2==
Some OS/2 users have expressed interest in a text-only, non-GUI PROTSHELL for OS/2. TShell (''T for Text'') is on the Developer Connection for OS/2 CD-ROM, Volume 3. TShell provides multiple OS/2 sessions; it can start DOS and WIN-OS/2 sessions if the machine is so configured.

TShell is similar to the OS/2 1.0 shell, Program Selector. On the left, there is a list of programs to start. On the right, there is a list of programs that are currently running. See ''Figure 2''. Press Tab to alternate between the two lists. TShell implements traditional OS/2 hot keys: '''Alt-Esc''' visits each session in a round-robin manner, '''Ctrl-Esc''' goes directly to TShell.

[[Image:Shell-Fig-4.gif]]

''Figure 4. TShell: A Text-Only OS/2 Shell''

OS/2 with TShell will not run OS/2 PM applications because the PMWIN window manager is not initialized. See the TShell readme file for a list of restrictions and known bugs.

With its restrictions, TShell might serve a purpose in low-resource machines, embedded systems, or test beds.

;Notes on TShell:
# Memory Use: TShell and the 2.1 debug kernel uses 2.9 MB of RAM. The retail kernel will use slightly less memory. TShell can boot in a 4 MB machine with or without swapping, although swapping generally is preferred.
# Doesn't Spool: There is no spooler with TShell.
# CID Server: TShell works on CID servers using SRVIFS. By using a low-resource shell, CID installers devote more memory to file system cache.
# Boots Fast: TShell appeals to ring 0 device driver developers who frequently trash test machines and want the fastest possible reboot.
# Programmable Start List: See the TShell readme for notes about using ''pgmshell.exe'' to customize TShell.

==Future Considerations==
All of these alternatives work well on OS/2 2.1. However, the suitability of these shells on future versions of OS/2 must be understood. In the order of least risk to most risk:
* Subclassing Workplace Shell objects is the most conventional and well-documented approach to alternative shells for OS/2.
* MShell, a RUNWORKPLACE shell, uses one OEMI API to start the spooler. This API may change, or perhaps the responsibility for starting the spooler will move to the PROTSHELL process. Either way, spooling is a must, and there will always be a way to start the spooler. MShell lacks a PM print driver installation process. MShell is on The Developer Connection for OS/2 CD-ROM with source code.
* Using ''cmd.exe'' as PROTSHELL for a single-session boot will work as long as ''cmd.exe'' does not require PM. To date, ''cmd.exe'' has no dependency on PM.
* TShell uses undocumented session manager APIs for initializing, starting, switching, and enumerating sessions. These have never been documented - not even as OEM APIs. TShell should run on future Intel versions of OS/2. TShell is on The Developer Connection for OS/2 CD-ROM without source code.
Finally, these shells are written and tested by the author. They are not subject to official testing by IBM. IBM makes no warranties about correctness or suitability of these shells for any purpose on any version of OS/2.

{{IBM-Reprint}}
[[Category:Developer Connection News Volume 3]]