A flat model device driver for OS/2:Chapter 4. - Analysis of the Problem

How We Tested
For our first conversion from the 16-bit model to the new flat model, we decided to start with a simple device driver that is used to access the parallel port called MMAP.C (see Appendix A – Listings). Programmers have frequently used this public domain device driver as a basis for learning how to write OS/2 device drivers (Most OS/2 base device drivers are written in C, which made the conversion process mush easier than if the driver had been written in assembly language).

The device driver contains the two main sections of an OS/2 device driver, the Initialization section, and the Strategy section. In order to keep the device driver simple; the driver does not contain an interrupt handler or timer handler (See Appendix – Listings). We began by identifying the variables that needed to be converted to accommodate our new model. Before modifying the driver variables, we decided to start with the local header files (see Appendix A - Listings) that are included by the device driver.

While operating in the 16-bit mode, the instruction set of the Intel processor allows two distinct types of addresses. To access data or instructions within a particular 64KB segment, a program can use near addressing. Near addressing assumes that the code or data being accessed resides in the same segment that is currently selected. Since the same segment is used, the only portion of the address that needs to change is the 16-bit offset.

Far addressing allows access to code or data anywhere within the 16MB memory area. In far addressing, the pointer contains a segment as well as an offset. Like near addressing, the offset portion specifies an address within a 64KB segment.

A 32-bit flat model program makes no distinction in addressing modes. All pointers and addresses are 32 bits long, and are referred to as linear addresses. In fact, the 32-bit C and C++ compilers do not allow the use of the common 16-bit modifiers near and far, so we began by deleting all of the near and far modifiers in the header files. For purposes of readability, we also deleted any references to the data type FARPOINTER, including any instances where FARPOINTER was used in function prototypes (13 Addressing in OS/2 device drivers has been one of the most difficult tasks to understand and implement because of the various modes of addressing.).

Header File Conversion
To avoid unnecessary duplication of 32-bit data types, we began by changing all of the unsigned short variables in the structure definitions to standard 32-bit data types defined in the 32-bit compiler’s header files. Variables that contain pointer data types are automatically expanded to 32-bits by the compiler when the compiler-defined pointer data type is used in the declaration. We also made sure that any variables that were not defined as pointers but that might hold a pointer temporarily were converted to 32-bit variables (see Appendix A - Listings).

We then converted any variables defined as BYTE or unsigned character to 16-bit. The reason we did this is that the compiler will attempt to optimize the storage of data structures by packing bytes together in adjacent memory locations. The data in the variables is sometimes passed by reference to another device driver or application. That is, the driver may send a pointer to the data structure to another device driver or application. The receiving driver or application may attempt to de-reference the pointer using a 16 or 32-bit pointer type, so the data must be word-aligned. Although we knew that memory requirements would be increased because of the larger data types, we felt that it was better to increase the memory size than to spend time debugging misaligned data structures. Unfortunately, we had to reverse this operation because we could not be sure that the system-level variables defined by these fields would be changed.

We then changed the system library function prototypes from NEAR PASCAL to PASCAL. The PASCAL parameter passing method is much easier to implement because a fixed number of parameters must be passed and the caller does not have to worry about cleaning up the stack. This makes even more sense because the device driver does not own the stack; therefore the device driver cannot manipulate the stack or stack pointer.

Driver Code Conversion
When we finished converting all of the header files, we began converting the variables in the device driver. OS/2 device drivers usually contain two types of variables, global and local. Global variables exist outside the C source code, and can be accessed by any function or subroutine in the device driver. Local variables are variables created within a particular function. Local variables are stored on the stack. We converted all of the pointers to the correct type, and then began changing some of the driver’s internal structures. Whenever we encountered a structure with a built-in pointer, we changed that pointer to a standard 32-bit pointer type. We also changed an important part of the device driver, the device header. This structure defines the name of the driver, the type of device the driver supports, its capabilities, and pointers to major sections of internal device driver code. This has been a source of problems with the 16-bit device driver model. In the 16-bit design, the device header contains 16-bit offset pointers to the Initialization and Strategy sections of the device driver. The Initialization entry point is called when the device driver is loaded and the Strategy section is called for every other command sent to the driver. Because those pointers were only 16-bit pointers, the largest driver that could be written had to fit within a 64K-code segment. There was a workaround, though, by having the header point to an entry point in the first 64K segment. The first segment would then jump to a second code segment. Implementing this was quite tricky and resulted in some ugly code. Making these pointers 32-bit allows the driver to install an interrupt or timer handler anywhere in physical memory (14 16-bit drivers may continue to be loaded in low memory for backward compatibility).

This is important because OS/2 loads all 16-bit device drivers in low memory (less than 1MB). This memory is a precious commodity, and it’s easy to run out of space with several large multi-segment device drivers. Allowing the driver to be located anywhere in memory releases it from many historical limitations and helps free up lower memory. This means a major change to the system memory allocation routines, however, since the areas where device drivers are loaded must be permanently locked in memory and must be arranged as to prevent fragmentation. As part of our rewrite, we have specified that the OS/2 kernel must perform a post-boot compaction of device drivers as to avoid fragmentation. We also specified that the driver contains a special bit to override compaction.

Another limitation that we addressed was the entry points for timer and interrupt handler routines. When a device driver needs to install an interrupt or timer handler, it calls a system function, passing the entry point of the interrupt or timer handler code. Because these pointers were 16-bit, the timer and interrupt handlers had to reside in the first code segment so the handler could be reached with a 16-bit pointer. Once in the interrupt handler, the handler code could call or jump into another segment, but this was also difficult to implement. Crossing segment boundaries is time consuming because the cache becomes invalidated and has to be reloaded each time a 64K boundary is crossed.

Driver Loading
For this rewrite, we have specified that OS/2 device drivers can be loaded after the system is booted and running. This requires a large change in the system loader and support code, but we feel this will be made easier by allowing the device drivers to be loaded anywhere in memory. When loading device drivers, the system keeps a list of installed device drivers in a chain of pointers. The chain pointer is initialized to the address of the first device driver’s header. Each subsequent device driver header contains the address of the next device driver in the chain. The 16-bit device driver header uses an unsigned long (ULONG) for this pointer, so the system should be able to chain 16-bit and 32-bit device drivers without any extra logic in the chaining algorithm.

When the driver is opened, the system will check a specific bit in the device driver’s header that identifies the driver as a new 32-bit driver.

Once the determination has been made, the system can selectively send request packets that are formatted correctly for the model. A READ request packet, for example, contains the address of a data buffer in program space where data is to be stored. For the 16-bit model, those pointers will be passed as 16:16, for the 32-bit model, the pointers will be passed as linear addresses. The system will handle thunking 32-bit pointers to 16-bit, and 16-bit to 32-bit. By using this transparent thunking, current 16-bit drivers will continue to work normally while allowing the new 32-bit model to be supported. This is also important because OS/2 supports both 16-bit applications and 32-bit applications at the same time. It is possible, for example, for a 16-bit application to call a 32-bit driver, or a 32-bit application to call a 16-bit driver. Existing applications should continue to run unchanged, and to be able to utilize the new device driver model without a recompile (Thunking is time-consuming and should be avoided whenever possible).

Resource Allocation
Device drivers allocate and use system hardware and software resources to perform operations. If the device uses DMA to transfer data, the device driver must reserve (or have available) a DMA channel it can use when it needs to transfer data. If the device uses interrupts, the driver must reserve that interrupt so it can handle the interrupt from the device. If the driver waited until the last possible moment to see if the resource was available, it would fail if a program tried to use the device and the resource was not available. This might be satisfactory for a printer, but not for a cardiac monitor or bank teller machine.

Driver Information File
The device driver determines the acceptable hardware configuration by referring to a driver information file, or DIF (see Figure 4.1. - Sample DIF File. Figure 4.1. Sample DIF File

Using the information in the DIF file and the current configuration of the hardware, the driver registers for the resources it requires by calling the system’s Resource Manager. The resource manager keeps track of the allocation and use of system resources. When a driver requests a resource, it calls the Resource Manager to request the use of that resource. The Resource Manager returns the status of that resource as AVAILABLE, NOT_AVAILABLE, or AVAILABLE_SHARED. Based on this information, the driver can determine what action to take. If the resource is available and the driver must absolutely have the resource, the driver calls the Resource Manager to reserve that resource exclusively. In some cases, the driver might allow sharing of that resource, but this is usually only safely done if the other driver or drivers sharing the resource are also aware that it will be shared. The driver uses the DIF file for a list of valid resources that the hardware supports. The DIF file also contains a list of substitute resources should the desired resource be unavailable. For instance, a particular device might request interrupt 5, but might also be able to use IRQ 4, IRQ 3, or IRQ 11. The DIF file contains a list of these resources in a hierarchy of desired resource allocation (see Figure 4.2). [Interrupts] 5 4 3 11 ''Figure 4.2. DIF File Interrupt Entry''

If the first resource in the category is unavailable, the driver will query the resource manager using the next supported resource until the request has been satisfied or all of the supported resources have been tried. The resources are organized in a hierarchical fashion, specifying the most optimal settings first followed by the least optimal. Using this method, the driver can determine the best configuration based on its intimate knowledge of the device. For instance, while configuring an interrupt, the driver could decide that based on the priority of the interrupt level it is granted, that the DMA channel should be an 8 bit DMA channel instead of a 16-bit DMA channel. The driver can make that decision based on its detailed knowledge of the device.

After the driver has been loaded, the user can change the values of these parameters using the Configuration Manager application. This application displays the current configuration and settings for all devices, and wherever possible, allows these settings to be changed. Settings that cannot be changed are displayed in a lighter colour to indicate that they cannot be adjusted manually.

To make this concept work, existing 16-bit drivers should also include a DIF file and the associated code to request resources. It is not an absolute requirement, however.

Recovery
Each time the Configuration Manager is run, or a driver installed or changed, the current configuration is backed up to a file in case the current configuration causes the system to become unbootable or unstable. Each backup file is created with a different name using the current time and date. For example, a configuration file saved on October 23, 1996 would be called something like C102396.DAT and stored in the Configuration Manager’s default directory.

When the system is rebooted, a menu will appear for 10 seconds that will allow the user to reboot using a different configuration than the current configuration. If the user selects this option, a menu of the current saved configurations is listed. The user highlights the desired configuration and continues the boot progress. If the boot process succeeds, the system will ask the user if the new configuration should replace the existing configuration. This will allow the user to boot the system with several experimental configurations without losing the last saved good configuration. The system always keeps a copy of the original configuration used when the operating system was installed to allow a default configuration to be used in the event that the system is rendered unbootable.

Device Locator
Each device driver must supply a device locator program that verifies the presence of the particular device and verifies its configuration. The configuration manager uses this program to detect if the device is present, and to locate the device driver in the event that it needs to install the device driver. The locator program name and path are located in the driver’s DIF file.

Configuration Manager
The OS/2 configuration manager should run every ten to fifteen seconds to detect the installation of new hardware devices. It queries the current configuration and compares it against the last stored configuration to see if any hardware has been added. To determine the current configuration, the configuration manager runs a list of locators. A locator is a program written by the device driver writer that queries the particular device to see if it is present. In addition to running every 10 seconds, the configuration manager will also be run if it detects a PCMCIA card insertion event. During system boot, one of the first drivers that are loaded is the configuration manager itself. At that time, the configuration manager driver signs up to receive power management and PCMCIA events. If a PCMCIA card is inserted or removed, the configuration manager is called to determine the new configuration.

The configuration manager will allow the user to enter settings for device drivers and hardware that do not participate in the new configuration architecture. Existing 16-bit device drivers may elect to supply a DIF file and the code to arbitrate resources with the configuration manager. For those drivers that can’t be modified, the configuration manager program will supply a manual input to allow resources for legacy devices and drivers to be reserved. If, for example, a certain device has a non-movable port address of 0x300, the user can reserve that port address using the resource manager.

Dynamic Loading
If the configuration manager detects that a new device has been installed, it looks in the \OS2\DRIVERS directory for a DIF file that matches the device. If it finds the DIF file, it calls the system device driver loader to load the device driver. If the device driver is loaded successfully, the locator associated with that device driver is added to the list of installed locators. The next time the configuration manager runs, the locator associated with that device will find the device already present.

If the correct DIF file cannot be found, the system will ask the user to enter a floppy disk that contains the DIF file and driver associated with the device. The system copies that DIF file and driver to the system’s hard disk, and restarts the configuration process. In some cases, such as drive letter reassignment, rebooting the system may be required.

Dynamic Driver Binding
In current versions of OS/2, the device driver code uses static linking. Run time libraries, supplied by IBM and other vendors are linked with the device driver at build time and become part of the driver executable code. Because these driver libraries were created by different companies, the names of the functions are different, as well as the number and name of the function parameters. We plan to solve this problem by supplying a 32-bit standard driver run time library that is dynamically linked at driver load time. The driver links with the import library of the run time DLL just like a normal application links with a DLL’s import library. The difference is that the driver gets its own copy of the run time library which becomes statically linked at driver load time, not at build time, resulting in much smaller driver executables. The driver library has the look and feel of a standard DLL, but a separate instance of the DLL is created for each driver. Unlike traditional DLLs, this DLL-like entity can be loaded at boot time and while the system is operating at ring 0. Updated library DLLs can be shipped via the normal update channels. This DLL could have been shipped as a standard library (LIB) file, but by using a DLL format, we can include debug symbol information and allow the library to be run and tested at ring 3. We can also insure that the current shipping library DLL is compatible with the current build of the operating system, and that the library is MP safe.

Debugger Modifications
One of the problems in developing OS/2 device drivers is the lack of good debugging tools. The current debugger is actually a replacement OS/2 kernel. It is loaded in place of the standard OS/2 kernel file, OS2KRNL. It provides support for debug information to be displayed and input from a standard terminal on a serial COM port. The current debugger is barely adequate, and does not support the display and modification of data structures. Commands and functions are cryptic and difficult to understand. We don’t feel, however, that the base command set should be changed. In spite of its weaknesses, developers have learned to use the debugger successfully, and we don’t want them to learn yet another command set. Since the debugger can already display and modify 32-bit values, it should be able to be used “as is” with few actual modifications.