Revision as of 19:24, 18 May 2025

Kernel Debugger

The Kernel Debugger is essentially a replacement OS/2 Kernel module that contains a built-in debugger component. The kernel debugger can be used to halt system execution, inspect and alter memory and registers, and display system control blocks. The kernel debugger is described in detail in The OS/2 Debugging Handbook - Volume II, IBM publication number SG24-4641.

The kernel debugger architecture is such that only one thread can be in the debugger at any given time. The kernel debugger in OS/2 Warp Server for SMP serializes its access using a spinlock.

When entered, the debugger informs the user of the state of all the processors, even though the other processors are still executing code. This is accomplished by sending a spin command via an interprocessor interrupt, or IPI, to all the other processors. When a processor receives the spin command, it saves its current state information, acknowledges receiving the command, and then spins. This allows the debugger to switch to a slot which is currently executing on another processor and determine what it is doing.

New and Changed Commands

All kernel debugger commands work as before, but a few have been modified to display or use multiprocessor-specific information. New commands have also been added.

The new and changed commands are:

.DP (processor status) The .DP command has been added to dump out a processor control block verbosely, that is, to display a processor's status. The command takes as an argument:

 * * Displays status based on the real current slot
 * # Displays status based on the currently selected slot
 * 0-based processor number Displays status for a specific processor. .DP 3 would display the status of processor 3.
 * blank Displays the status of all processors.

.DL (display processor spinlocks) The .DL command has been added to display the spinlocks owned by a particular processor. Valid arguments are:

 * * Displays spinlocks based on the real current slot
 * # Displays spinlocks based on the currently selected slot
 * 0-based processor number Displays spinlocks for a specific processor. .DL 1 would display the spinlocks held on processor 1.
 * Address of a spinlock Displays a specific spinlock.
 * blank Displays all spinlocks.

.DS Dumps the Inter-Processor Interrupt (IPI) statistics for each processor, including the IPI type and IPI delivery method.
.DSH Dumps the IPI history, which is the last 256 IPIs generated in the system. The source, destination, and type of the IPI are displayed as well as the IPI delivery method and the address and symbol name where the IPI was generated.
.PQ Dumps the scheduler's priority queues based on the debugger's slot number.
.R and R (register commands) The register commands have been enhanced to indicate which processor the currently selected slot is running on. This information is displayed at the end of the third register line in the form p=xxyy where xx is the 1-based processor number, and yy are the flags. A processor number of 00 means that the currently selected slot is not running on any processor, or is blocked. The flags are:

 * s processor is currently spinning.
 * r processor is attempting to grab the ring 0 suspend lock.

.SS (change current slot) The .SS command has been modified to reference the proper PSA (processor save area) when displaying a variable on a different processor.
.S The .S command is now identical to the .SS command. The PLMA is displayed properly for each processor.

KDB Communications Protocol

Generally, the kernel debugger communicates over a serial communications link with a terminal emulator program running on another machine. This allows a user to debug a problem by issuing kernel debugger commands in the emulator program and seeing the results displayed on the debug console.

To automate the debugging process, or to provide a high-level language debugging environment, this interaction with the kernel debugger could instead be handled by a program running on the other machine. This debug engine would interact with the user, convert the user's debugging request to a series of kernel debugger commands, issue them, and then present the response from the kernel debugger back to the user in a user-friendly format.

Communications between the debug engine and the kernel debugger can proceed in one of two modes:

raw (dumb TTY) mode ASCII characters are sent to the kernel debugger one at a time. The kernel debugger echoes each character. A carriage return (^M or 0x0d) ends a line. The kernel debugger returns data to the debug engine one ASCII character at a time.
packet mode Packets are sent to the kernel debugger. A packet consists of a fixed sized header followed by zero or more bytes of data. The kernel debugger returns data to the debug engine in packets.

Raw Mode

Any kernel debugger command may be sent in raw mode. Debug engines that communicate in packet mode may wish to use raw mode to issue a .B command to set the communication rate for the serial connection.

To enter raw mode from packet mode, or to get the kernel debugger's attention while the system is running and enter raw mode, the debug engine should send a break character (^C or 0x03) and wait for the kernel debugger to issue a prompt.

Packet Mode

To enter packet mode from raw mode, or to get the kernel debugger's attention while the system is running and enter packet mode, the debug engine should send the KDP break character (0x1f). If the system was running, the kernel debugger will respond with an event packet containing a CVK_RET_ASYNC event. If the system was quiesced, the kernel debugger will not respond at all.

Packet Format

The format of a packet is as follows:

+-------------------------------+---------+
| 0x1d | 10-byte packet header | packet body (optional) | 0x1e |
+-------------------------------+---------+

The packet header and the packet body, if present, are bitstuffed. The data is treated as a stream of bits and is broken into seven-bit chunks. Each chunk is put into the seven low-order bits of a byte and the high order bit of the byte is set. The bitstuffed data is padded at the end with zero bits to the next byte boundary. For example, the header 00009540 0000ac57 when bitstuffed becomes 808092d4 808081ac abc0.

Packet Header Format

The header, prior to bitstuffing, contains a 4-byte logical ID field, a 2-byte length field, and a 2-byte checksum. The checksum of n bytes of data is computed as follows:

unsigned char  data[];
unsigned short checksum = 0xa1e8;
for (i = 0; i < n; i++)
{
  checksum += data[i];
  checksum = (checksum << 3) + (checksum >> 13);
}

Packet Data Format

All multibyte items are presumed to appear in little-endian order. Thus, the checksum computed for the header 00009540 0000 is 0x57ac; when stored in the header the low order byte (0xac) appears first.

The header length field contains the number of bytes of data in the packet body before the packet body is bitstuffed. If the header length field is zero, there is no packet body. If a packet body is present, it includes a 2-byte checksum that is not accounted for in the header length field. For example, if the header length field is 0x12, there are actually 20 bytes in the unbitstuffed packet body.

The logical ID field takes one of the following forms (sequence numbers are one byte long):

CVK_HDR_DATA - 0x8000                          |
               0x4000 if 'flast' flag is set   |
              ((index number & 0x3f) << 8) |
               sequence number

CVK_HDR_ACK - (0x4000 | ((index number & 0x3f) << 8) | sequence number) << 16

CVK_HDR_NACK- (0xc000 | ((index number & 0x3f) << 8) | sequence number) << 16

Maximum Packet Size

The maximum packet size for a bitstuffed packet is defined by CVK_PACKET_MAXSIZE as 623. This is derived by:

Start byte      1
Header         10
Data          611
End Byte        1

General Considerations

General Considerations

Kernel Debugger Packet Responses

Kernel Debugger Packet Responses
Event	Code	Description
CVK_RET_SUC	0x0000	Success
CVK_BAD_COMMAND	0x0002	Unrecognized command
CVK_RET_PAGEIN	0xffef	Discarded page reloaded
CVK_RET_TEND	0xfff0	Task died
CVK_RET_TNEW	0xfff1	Task created
CVK_RET_ASYNC	0xfff5	Asynchronous halt (break)
CVK_RET_LIB	0xfff8	Module loaded
CVK_RET_GPF	0xfff9	General protection fault
CVK_RET_KIL	0xfffa	Module unloaded
CVK_RET_NMI	0xfffb	Non-maskable interrupt
CVK_RET_BPT	0xfffc	Software breakpoint (INT3)
CVK_RET_TBT	0xfffd	Single step
CVK_RET_ERR	0xffff	Failure

Events Reported by the Kernel Debugger

The following is a summary of the data reported when the kernel debugger sends an event in packet mode:

Events Reported by the Kernel Debugger

Kernel Debugger Packet Commands

Kernel Debugger Packet Commands

@@ Line 111: / Line 111: @@
 === General Considerations ===
+* [[CPGuide_-_Kernel_Debugger_Communications_Protocol#General_Considerations|General Considerations]]
-The debug engine initiates all transactions with the kernel debugger, normally by sending a command while the kernel debugger is waiting to receive one. In this case, the kernel debugger expects to receive a CVK_HDR_DATA packet and will discard any CVK_HDR_ACK or CVK_HDR_NACK packets it receives. (The kernel debugger also discards a packet if its header cannot be unbitstuffed or has a bad checksum. The kernel issues a CVK_HDR_NACK packet containing the sequence number and index number from the original CVK_HDR_DATA packet if the packet's body cannot be unbitstuffed or has a bad checksum.)
-Once the kernel receives a valid CVK_HDR_DATA packet, it extracts the sequence number and index number and uses them to construct a CVK_HDR_ACK packet, which it returns to the debug engine. (The ''flast'' flag in the CVK_HDR_DATA packet is ignored.) The kernel debugger then performs the action requested by the command and returns the result.
-In general, the result is returned in a single CVK_HDR_DATA packet whose sequence number matches the sequence number contained in the debug engine's original command packet, whose index number is zero, and whose ''flast'' flag is FALSE. After the result is transmitted, the kernel debugger waits for a response from the debug engine. The kernel is expecting either a CVK_HDR_ACK packet whose sequence number and index number match those sent in the result _or_ a CVK_HDR_DATA packet (containing the next command). If the kernel debugger receives any other response, it resends the CVK_HDR_DATA packet containing the result.
-If the debug engine sends a KDP break character while the victim machine is running, either to initiate a transaction or to regain control after the victim machine has resumed execution, the kernel debugger responds with a CVK_HDR_DATA packet whose sequence number matches the sequence number from the CVK_HDR_DATA packet that caused the system to resume. There is no such packet when the kernel debugger responds to the first KDP break sent by the debug engine. The sequence number in that case contains garbage.
-The kernel debugger does not generate replies for some commands, such as reboot, and the replies to commands that cause the victim machine to resume execution, such as resume or step, are not sent until an event such as a breakpoint, module load, or break signal from the debug engine, has caused the victim machine to quiesce.
-The kernel debugger responds somewhat differently to the CVK_CMD_RAW command, which is used to issue arbitrary kernel debugger commands while in packet mode. Each line in the response is returned in a separate CVK_HDR_DATA packet whose sequence number matches the sequence number in the CVK_CMD_RAW command's header. The index number in the first reply packet is 0; the index number increases by 1 in each successive reply packet (and wraps from 63 to 0). The debug engine should return a CVK_HDR_ACK packet with the appropriate sequence number and index number after each reply packet is received.
-The kernel debugger does not manipulate or increment sequence numbers and uses them only to generate ACKs and NACKs and to match ACKs with replies. A debug engine could use the same sequence number for every request, but this is not recommended.
 === Kernel Debugger Packet Responses ===