[Here is a file to accompany the article. Ed.]
The purpose of this article is to assist developers of OS/2 device
drivers to ensure that their device drivers perform as expected in a
Symmetric Multi-Processing (SMP) environment. Some of this material is
also covered in the smp.inf online documentation. This document is
available on the Warp Server with SMP install CDROM. I will also make sure
that EDM/2 has a copy.
Although most device drivers will work fine without modification, there
are four main areas of concern to bear in mind. Specifically, these are:
- Local Infoseg access
- Port I/O operations and IRQ masking
- Serialization concerns
- High Memory access
- If you use the Local Infoseg (LIS) to get information about the
currently-running thread, you can get in trouble if you're also using a
32-bit device driver. Specifically, the selector returned by
DevHelp_GetDOSVars is remapped for each CPU to point to the correct
information. However, the linear addr underneath is not (at least for now)
remapped to the correct physical page. That means that, when you get this
selector, if you then call DevHelp VirtToLin to get the linear address,
the pointer you get back will only be valid on the CPU you happened to be
on when you called VirtToLin. The moral of this is that, even if you are
using a 32 bit DD, you HAVE to use a 16:16 access method to get to the
- Port I/O and IRQ masking: there are two different concerns here.
First, many device drivers, particularly those modeled after some old DDK
samples, may issue an End-of-Interrupt (EOI) to the 8259 chip by directly
doing an "out" to port 20h. DON'T DO IT! This will fail catastrophically
on systems running in "advanced interrupt" mode (see the Intel
"Multiprocessor Specification", Intel order number 242016-004). Use
ifndef SMP ; old way
else ; new way
mov dl,DevHlp_EOI ; 76h
call DWORD PTR [device_help]
; as a (somewhat useless) example, here is the old EOI method rewritten
; to use the new port_io devhelp. Taken from smp.inf.
port_io_port DD ?
port_io_data DD ?
port_io_flags DD ?
IO_READ_BYTE EQU 0000H
IO_READ_WORD EQU 0001H
IO_READ_DWORD EQU 0002H
IO_WRITE_BYTE EQU 0003H
IO_WRITE_WORD EQU 0004H
IO_WRITE_DWORD EQU 0005H
IO_FLAGMASK EQU 0007H
PORT_IO port_io_s <20h,20H,IO_WRITE_BYTE>
; EXIT: port_io_struc.data filled in if I/O read
A related function, VDHPortIO(), is provided for virtual device drivers.
Similarly, getting and setting IRQ masks may need to be mediated by the
platform-specific driver (PSD). I will omit the code sample here -- see
the INF file.
- serialization concerns: with only one CPU, it suffices for a device
driver to use CLI/STI (or push/CLI/popf) to ensure that interrupt
processing will not disrupt a critical code region. With more than one
CPU, this can fail as follows:
You are already processing an interrupt, but the system running your
device driver is in "advanced interrupt mode". What do you do? Basically,
you need to use a mutual exclusion ("mutex") semaphore, just as you would
if you were writing ring 3 apps. Warp SMP provdes a suitable package, and
I will also provide another one below. Warp provides a package called
"SpinLocks" that is accessible from your device driver via the following
calls to [DevHelp]:
name DL value ax:bx value
DevHlp_CreateSpinLock 79h &hlock
DevHlp_FreeSpinLock 7Ah hlock
DevHlp_AcquireSpinLock 71h hlock
DevHlp_ReleaseSpinLock 72h hlock
Interestingly, and currently undocumented, you can use the same
interface at ring 3 via the following DOSCALL1 exports:
The parameters are pretty much the same as above (though passed on the
stack, of course). The 32 bit calls are simply thunked to their 16 bit
counterparts for you. The nice thing here is that if you have some data
that is manipulated by a ring 3 daemon for one reason or another, you can
use a single lock handle to serialize access. DO NOT BLOCK WHILE HOLDING A
SPINLOCK! This also implies that you should not call any APIs of any sort
while holding a spinlock. You have been warned...
If you want, you can always "roll your own" locks. Supplied is one possible way of doing this (written
with the Visual Age optlink parameter-passing convention in mind, wher
parm1 is EAX and parm2 is EDX). This may be a better choice where the
resource in contention might be tied up for long periods of time. In this
case, the requesting thread will get put to sleep until the resource is
available. Also note that I store the pid/tid of the owner in the ULONG.
This helps to detect deadlocks and also will make your driver more easily
There is *probably* no danger of being interrupted by an external
interrupt on CPU 0 if you do a CLI on cpu1. In theory, what will happen is
this: Before calling your DD, os2krnl will have acquired the
R0SubsysSpinlock. Before the interrupt manager calls your Interrupt
handler, it will try to save and grab that spinlock. If you've done a CLI,
your CPU will not respond to the interprocessor communication request, and
so everything should work (though not on 2.11 SMP). This does not affect
synchronization of shared data, however, between ring 2/3 and ring 0 or
other issues if you *don't* do the CLI.
- High Memory Area (HMA) considerations: in Warp Server SMP (and the
forthcoming Warp Server for E-Business), apps can now allocate private or
shared memory above the 512mb virtual address line. Because of its
address, this is not "thunkable" using the usual shift and add algorithm.
Be aware that ring 3 apps might pass pointers like this (i.e. addr >
0x1fffffff). In this case, you can either access the pointer directly if
you have a 32 bit DD or create a GDT alias.
There are some other new features introduced in Warp Server SMP that
are not specifically SMP issues, such as Raw File System access and access
to PerfSysTrace data. Perhaps I'll cover these another time.