OS/2 Debugging in Style

From EDM2
Jump to: navigation, search

by Chris Matthews and Dave Evans


There's great news for all of you OS/2 kernel debugger (KDB) users! We are proud to introduce a new environment for you to debug source-level code for your OS/2 Warp device drivers, installable file system (IFS) binaries, and applications - all to increase your productivity during debug time. If you are worried about KDB, don't be! We have not taken it away or hindered it in any way.

The new debugging environment is the Remote Interactive Code Analysis Tool (Remote ICAT) for OS/2, which you can install from the Toolkit category of the Developer Connection catalog. Remote ICAT works on these versions of the target machine OS/2 Warp Version 4, Warp Server SMP Version 4, and OS/2 Warp Version 3.0 at FixPak 23 or later. Remote ICAT lets you debug a target OS/2 system from a remote host with the familiar ICAT debugger interface. This makes it easier to debug applications, IFS binaries, and device drivers, especially drivers using the new adapter device driver (ADD) and graphics ADD (GRADD) modules written in C. Here are some of the features of the new debugger.

Source-Code Support

Remote ICAT supports 16- and 32-bit C, C++, and assembler language for debugging source. It supports both the High-Level Language (HLL) debug format used by IBM VisualAge C++ and the CodeView (CV) debug format used by Microsoft's assembler, C, and C++ environments.

Stack Unwind

Remote ICAT provides the ability to unwind stacks across 16- and 32-bit C, C++, and assembler routines It also provides the ability to unwind the stacks through ring transitions (for example, from ring 0 to ring 3).

I/O Breakpoints

This is a new feature added to OS/2 Warp Version 4 in order to support breakpoints on I/O ports. It is a feature supported on the Pentium processor and some versions of the 486 processor. This feature has also been added to KDB.


The Remote ICAT's Passthru feature lets you access KDB while working in the source-code debugger. This is handy for retrieving information that might be easier to obtain through KDB (for example, by using the DG command).

Note The Passthru option can be dangerous if you modify the memory or register (state) on the target machine. Remote ICAT assumes that state is not changed while it has control. Therefore when you do make changes to the target machine state with the Passthru window, ensure that you select the Resync push button before Remote ICAT's next run or step directive. Even with the Resync option, it is safest to request information with the Passthru window as opposed to altering state.

Higher Baud Rates

With this new version of OS/2 Warp, the source-code debugger and KDB can operate at speeds up to 57,600 bps (115,200 bps if you use the special com.sys drivers of OS/2 Warp Version 4 on the host system) The default for the debuggers is 9,600 bps, but this rate can be changed to the higher rates in the setup file for Remote ICAT or by using the KDB b command

Operation of the Debugger


In the initial setup of the remote debugger, the target machine contains any of your IFS, Physical Device Driver (PDD), Virtual Device Driver (VDD), or application code to be debugged These must be compiled and linked using debug flags (the specific flags depend on your your compiler and linker) If file size is a problem, you can strip debug information from the binaries before putting them on the target machine, but they must be generated with the debug information initially.

The real debugging work is done on the host machine connected to the remote target machine through the COMx port (the same setup that KDB already uses). The host machine maintains the unstripped version of the debug binary files and the source used for building the binaries. The Remote ICAT debugger depends on the host machine s environment variables to access the directories that contain the debug binaries and source files. This allows Remote ICAT to debug device drivers, IFS binaries, or applications at the source level.


Once the initial setup of the target and host system is completed, the debugger environment is created to establish communication information, source and binary paths, module list selection, and other Remote ICAT options. Then, Remote ICAT can be started It takes a few seconds for Remote ICAT to initialize communication with the target system, but once the communication is established, the following initialization dialog is displayed Select Attach to complete the connection.

Note If you experience a communication problem, check to make sure that there is not a terminal emulator program running that competes for the COMx port. Also check to see if you can bring up KDB under a terminal emulator program (without Remote ICAT running) and communicate successfully. Don't forget to shut this terminal emulator down before retrying the Remote ICAT connection.

When the connection is complete, the debugger opens a Debugger Session Control (DSC) window. This window displays a list of Slots (process and thread IDs) along with the modules that are being debugged.

Note If you do not see your module in the list, it may mean that the module was not compiled with the debug flags, the module was not found in Remote ICAT s binary search path, or that the target machine was excluded due to an environment variable.

Notice that all the processes running in the system are displayed in the process list. This is because the source-code debugger piggybacks on KDB so that it comes in at the supervisor level just as KDB does. This gives the source-code debugger the flexibility to do things like debug an application and device driver in the same debug session at the source-code level with the same tool. You can watch your application call into your device driver and debug the device driver as the call enters the strategy routine.

Once the kernel attachment is complete, you can select a module from the module list in the DSC window and then select the source code for the function you want to debug. Then you can set a breakpoint and let Remote ICAT run When the breakpoint is hit, you can unwind the stack, look at storage, monitor properties of a variable, and so forth. At any time during the debug session, you can use the Passthru window to look at things like the global descriptor table (GDT), local descriptor table (LDT), and so forth. OS2DebugginginStyle-Fig-3.gif

Helpful Remote ICAT Debugging Tips

The following are some Remote ICAT debugging tips that can be helpful for both novices and more advanced users.

Debugging Init Routines

Here are some ways of debugging an init routine for your device driver, DLL, or other modules using the source-code debugger:

  • Place a breakpoint interrupt (Int 3) in the beginning of your module s init routine When the init routine is hit, the interrupt causes KDB to stop. When this happens, attach to the kernel with the Remote ICAT debugger The program counter (PC) register should now point to the source for the init routine. From here, you can continue to step through your code, set breakpoints, or run. This guarantees that your module s page has been loaded.

Note Don't forget to release the KDB terminal emulator before attaching with the Remote ICAT debugger Otherwise, a COMx port conflict might occur

  • Attach to the kernel when the system is coming up and KDB has already been loaded on the target machine. At this point, you can set your breakpoint in the appropriate location and run This method is risky because the init routine might not have been loaded or paged in yet Hence, the breakpoint might not be valid for the routine selected. It is possible to use the Passthru into the kernel in order to prefault the page (using the i command) if the module has already been loaded, but this is not certain You will get more consistent results with the pre-embedded Int 3 technique.

Disassembling a Raw Storage Location

At times you might find yourself debugging code placed in a raw storage location. This can happen if your application uses run-time generated instructions that are built into memory. To debug this code with the Remote ICAT debugger, you can modify the EIP register to point to the raw storage offset and let the Remote ICAT disassembler display the generated assembler information in the disassembly window.

Viewing Registers

Remote ICAT maintains a PC register, which is the linear address of the current execution point for the slot in question; that is PC == %(CS EIP) in KDB debugger parlance. If you want to see the EIP and EFLAGS registers, select Options-> Display style->Status Flags in the Registers window.

Ring-3 Debugging

You can debug applications (EXE and DLL files) at ring 3 with Remote ICAT. The application's modules will be displayed in the DSC window, and you can click on these modules to show their constituent parts. Note that you need to be in the context of your application when you bring up its source under Remote ICAT. By context, we mean the active thread must be one of your application's threads (slots). If not, then you are mapping source for (say) app1.exe on top of (say) app2.exe's address space. You don't run into this problem with device drivers or IFS binaries because they are mapped into the global address space.

The simplest way to establish this correct context is to open the Passthru window and emit the KDB p command. Find the application of interest from the KDB p response and note its process ID (PID). Now look at Remote ICAT's DSC threads in the Threads pane Double-click on any thread whose slot has the same PID you found in the p response. This forces Remote ICAT to establish that PID as the current context for ring 3.


You should now be on your way to successful remote source-code debugging of system-level binaries. We will continue to provide quality enhancements and features to the debugger, and your feedback is always welcome!

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation