kdb(1M)
kdb --
kernel debugger
Synopsis
kdb
Description
KDB
is a kernel debugger that works like a Reverse
Polish Notation (RPN) calculator.
KDB can set breakpoints, display kernel
stack traces and various kernel structures, and modify
the contents of memory, I/O, and registers.
The debugger supports basic arithmetic operations, conditional
execution, variables, and macros.
KDB does conversions from a kernel symbol name to its
virtual address, from a virtual address to the value
at that address, and
from a virtual address to the name of the nearest kernel symbol.
You have a choice of different numeric bases,
address spaces, and operand sizes.
This is an advanced tool, only for those who are thoroughly
familiar with the UNIX kernel.
Because UNIX systems differ, you could damage your
system by following some of the examples in this discussion.
You can invoke the debugger by using the kdb command
or the sysi86(SI86TODEMON) system call
on all systems,
<Ctrl><Alt>D (from the
console only) on an AT-bus system, or the interrupt
character (from the console only) on a Multibus system.
In addition,
KDB is entered automatically under various conditions,
such as panics and breakpoint traps.
Any time the kdb>>
prompt
appears, you are in the debugger.
I/O is done via the console (kd), or a serial terminal.
To exit the debugger, type q or <Ctrl>D.
When you exit and re-enter the debugger, its
state is preserved, including the contents of the value stack.
Using kdb as a calculator
KDB operates as an RPN calculator, similar to
dc(1).
This calculator has a 32-level value stack for storing results and
intermediate values.
Commands and values you enter operate on the value stack,
which is an internal data structure in KDB.
It has no connection with the kernel stack or any
other stack in the system.
To use KDB, at the kdb>>
prompt type one or more items (values or commands) on a line.
Separate items with spaces or tabs.
Press <Enter> to end a line and send its contents to KDB
for processing.
Each item is processed separately,
from left to right.
The values can be:
- Numbers
-
Use positive or negative integers.
Numbers must begin with a digit, or a minus sign for
negative numbers.
Begin octal numbers with ``0o'' and hex numbers with ``0x''.
Otherwise, numbers are assumed to be in the default
base -- the default is hex, unless you change it.
(See
``Resetting the numeric base''
for instructions.)
- Character constants
-
You can have KDB convert characters to a number by
entering one to four characters inside single quotes.
C-style escapes are supported in character constants.
- Strings
-
Use C-style strings, enclosed in double quotes.
- Kernel symbol names
-
When you type a kernel symbol name, its address is pushed
onto the value stack.
When you enter a number or a string, it is pushed
onto the value stack, becoming the new Top Of Stack (TOS).
Values remain on the value stack until they are popped
off as a result of a command.
In the descriptions below, [TOS] means the value on the
top of the stack and
[TOS-1] means the value immediately below it (pushed previously).
Stack operations
KDB provides these commands for examining or changing the value stack:
- stk
-
Print all values on the stack
- p
-
Print [TOS]
- dup
-
Push [TOS]
- pop
-
Pop 1 value
- clrstk
-
Pop all values
Stack operation examples
- stk
-
Displays the entire stack.
For example, starting with an empty
value stack, the input:
5 "xyzzy" 7 stk
produces the output:
5
"xyzzy"
7
- p
-
Displays the top value on the stack.
In the example, this is:
7
The next example uses the p command to display
the address of a kernel symbol.
The input:
lbolt p
produces an address, for example:
D01821BC
- dup
-
Uses a value twice
in a calculation.
For example:
5 3 * dup 2 + * p
would produce the output:
FF
which is the value of (((5 * 3) + 2) * (5 * 3)).
- pop
-
Removes the top
value from the value stack.
For example, if this is the stack:
5
"xyzzy"
7
the input:
pop stk
removes the top value from the stack and
displays the resulting stack:
5
"xyzzy"
- clrstk
-
Clears the value stack.
Remember that the contents of the stack are saved when
you exit and re-enter KDB.
Arithmetic operations
You can perform arithmetic operations on the top two values on the stack:
- +
-
compute [TOS-1] + [TOS]; pop 2; push result
- -
-
compute [TOS-1] - [TOS]; pop 2; push result
- *
-
compute [TOS-1] * [TOS]; pop 2; push result
- /
-
compute [TOS-1] / [TOS]; pop 2; push result
- %
-
compute [TOS-1] % [TOS]; pop 2; push result
- >>
-
compute [TOS-1] >> [TOS]; pop 2; push result
- <<
-
compute [TOS-1] << [TOS]; pop 2; push result
- <
-
compute [TOS-1] < [TOS]; pop 2; push result
- >
-
compute [TOS-1] > [TOS]; pop 2; push result
- ==
-
compute [TOS-1] == [TOS]; pop 2; push result
- !=
-
compute [TOS-1] != [TOS]; pop 2; push result
- &
-
compute [TOS-1] & [TOS]; pop 2; push result
- |
-
compute [TOS-1] | [TOS]; pop 2; push result
- ^
-
compute [TOS-1] ^ [TOS]; pop 2; push result
- &&
-
compute [TOS-1] && [TOS]; pop 2; push result
- ||
-
compute [TOS-1] || [TOS]; pop 2; push result
- !
-
replace [TOS] with ![TOS]
- ++
-
replace [TOS] with [TOS] + 1
- --
-
replace [TOS] with [TOS] - 1
For example, this input (subtracting 5 from 7):
7 5 - p
would produce this output:
2
The power of KDB's calculator feature lies in its ability
to evaluate expressions like this:
callout 16
This pushes the address of the callout table on
the stack and adds 16 to it.
If the size of a callout table entry is 16 bytes,
the result of the calculation is the address of the second entry
in the callout table.
(Use the size command of
crash(1M)
to find
the sizes of common system tables.)
CAUTION:
Make sure the divide operator (slash character)
is both preceded and followed by spaces.
If any other character
appears next to the slash, it indicates a suffix instead of division.
Reading and writing to memory
These commands operate like an RPN calculator, but they perform
specific debugging operations instead of calculations.
To examine and set the contents of memory (and I/O) use the commands:
- r
-
Replace [TOS] with the value at virtual address [TOS].
- w
-
Write [TOS-1] into virtual address [TOS]; pop 2.
- dump
-
Show [TOS] bytes starting at virtual address [TOS-1]; pop 2.
- fdump
-
Show [TOS-1] formatted items at [TOS-2] with format [TOS]; pop 3.
Examples
- r
-
You can find, for example, the value
of the (long) kernel variable, lbolt, by typing:
lbolt r p
This puts the virtual address of lbolt on the
stack, replaces it with the value found
at that address, and prints the result.
- w
-
To change the value of lbolt to 2000, type:
2000 lbolt w
This writes 2000 at lbolt's virtual address.
You could increment lbolt by typing:
lbolt r ++ lbolt w
This puts the virtual address of lbolt on
the stack, replaces it with the value found
at that address, adds 1 to the value, and writes the
result at lbolt's virtual address.
- dump
-
This command displays a range
of memory, both in hex and ASCII.
For example, if you typed:
putbuf 10 dump
This shows 10 bytes, starting at the virtual
address of putbuf,
you would see something like:
........ ........ ........ 61746F74 D0108C50 ............tota
6572206C 6D206C61 726F6D65 ........ D0108C60 l real memor....
In each line, the block of four values on the left shows the values of
16 bytes, displayed as four 4-byte longwords in hex.
The dots represent values outside of the requested range.
(dump may also display question marks here:
that means the address is invalid.)
The next column is the address of the first of the 16 bytes.
The last column is the same 16 bytes displayed in
ASCII.
Dots represent values outside the requested range, or
unprintable characters.
- fdump
-
This command displays an arbitrary memory structure in
a formatted fashion according to a format string.
The syntax for fdump is:
address count format_string fdump
This displays count items starting at address
address.
fdump keeps track of a memory address
called dot while it is
processing the format string; some format commands
change the value of
dot; others use dot to access memory;
dot is initially
set to address.
format_string is a (double-quoted) string
which is a sequence of
any of the command characters listed below.
Any digits (0-9) in the format string are interpreted as the
``current number'' (num), which can modify
the following command;
most commands reset num after they have finished.
Any other character in the format string is printed literally.
- Number commands
-
- x
-
Print value at dot in hex (base 16), unsigned.
- o
-
Print value at dot in octal (base 8), unsigned.
- d
-
Print value at dot in decimal (base 10), signed.
- u
-
Print value at dot in decimal (base 10), unsigned.
- Size modifiers (prefix to number commands)
-
- B
-
byte (char)
- H
-
half (short)
- I
-
int
- L
-
long
- Other print commands
-
- c
-
Print value at dot as a character.
- s
-
Print value at dot as a (null-terminated) string.
- i
-
Print value at dot as a machine instruction.
- p
-
Print value at dot symbolically.
- a
-
Print dot symbolically (hex if num non-zero).
- n
-
Print a newline character.
- t
-
Print a tab character.
- Register modification (256 registers; first 200 are general purpose)
-
- m
-
Move dot to register num.
- M
-
Move register num to dot.
- r
-
Move num2 to register num.
- R
-
Move register num to num.
- .
-
Move num to dot.
- /
-
Move num to num2.
- Indirection
-
- *
-
Push dot; move value at dot to dot.
- ^
-
Pop dot from indirection stack.
- Quoting
-
- \
-
Print the next character literally.
- `` ''
-
Print the characters between the quotes literally.
- Other commands
-
- +
-
Increment dot (by num if non-zero).
- -
-
Decrement dot (by num if non-zero).
- ;
-
Repeat next command num times.
- ( )
-
Group commands together.
All print commands advance dot by the size of the object
printed.
Print commands are modified by non-zero num
and/or num2 as follows.
For number and instruction commands, num specifies the
output field width.
For character and string commands, num specifies the
style of
character printing: default is that all non-printable
characters show as a dot (.);
1 means to use ``control-character'' style which
prints control
characters as a caret (^) followed by
the control-character letter.
For string commands, num2 specifies the maximum number
of characters to be printed.
Note that the stack used for pushes and pops in the indirection commands
is not the KDB value stack; it is a special stack just for those commands.
As an example of using fdump, consider the following
kernel data structure:
struct cdevsw {
int (*d_open)();
int (*d_close)();
int (*d_read)();
int (*d_write)();
int (*d_ioctl)();
int (*d_mmap)();
int (*d_segmap)();
int (*d_poll)();
int (*d_msgio)();
struct streamtab *d_str;
char *d_name;
int *d_flag;
int d_cpu;
} cdevsw[];
The following fdump command could be used to
print out three elements from this cdevsw array:
cdevsw 3 "7;(p, )nt3;(p, )'*s', ^4+*x, ^4+d" fdump
The output from this command would be something like this:
wdopen, wdclose, wdread, wdwrite, nodev, nodev, nodev,
nodev, nodev, 0x00000000, 'wd', 112, 0
nulldev, nulldev, nodev, nodev, nodev, nodev, nodev,
nodev, nodev, scinfo, 'sc', 100, 0
mmopen, mmclose, mmread, mmwrite, mmioctl, mmmap, nodev,
nodev, nodev, 0x00000000, 'mm', 100, 0
Suffixes
Suffixes can be appended to many KDB commands.
They always begin with the slash character (/).
CAUTION:
Do not leave spaces before or after
the ``/''.
When the
``/'' is preceded and followed by a space, it
indicates division instead of a suffix.
- Operand-size suffixes
-
The r, w and dump commands can also work
with units of bytes and words, as well as the default longs.
To do this, append one of these suffixes to the command:
- /b
-
byte
- /w
-
word (2 bytes)
- /l
-
long (4 bytes) -- this is the default
- /L
-
long long (8 bytes)
For example, to display the value of a short (2-byte) variable at
address 0xD0008120, type:
0xD0008120 r/w p
Entering the dump command with /b displays
16 1-byte values per line, with /w displays
eight 2-byte values per line, and with /l (or nothing)
displays four 4-byte values per line.
- Address-space suffixes
-
The r, w and dump commands, by default, work
with kernel virtual addresses.
You can change to physical addresses, I/O addresses, or user
process virtual addresses by appending one of these suffixes
to the command:
- /k
-
kernel virtual -- this is the default
- /p
-
physical
- /io
-
I/O port
- /un
-
user process number n virtual (n is a process slot number in hex)
- /cpun
-
CPU number n (n is in hex)
- /cn
-
same as /cpun
Examples of using suffixes
- /p
-
For example, to dump 40 (hex) bytes
in longword format from physical address 2000,
type:
2000 40 dump/p
The default address is kernel virtual, so the /p suffix
is required for the physical address.
Note that an operand-size suffix is not required, because
long is the default.
- /io
-
For example, to read from port
300 (in bytes) and display the result, type:
300 r/io/b p
- /un
-
For example, to dump 20 longwords from
process 16's u-area at an offset of 1000, type:
1000 u + 20 dump/u16
- /cpun
-
For example, to print a stack-trace of CPU 5's stack, type:
stack/cpu5
or
stack/c5
- Suffix formats
-
Address-space suffixes can be combined with operand-size suffixes;
only the first slash is required.
For example, to do the read from I/O port 300
shown above, any of these command lines is acceptable:
300 r/io/b
300 r/b/io
300 r/iob
300 r/bio
Suffixes can also be attached directly to an address
as shorthand for ``read and print''.
Thus, 2000 r/p p can be shortened to 2000/p.
Since the default address-space is kernel virtual,
the common operation of ``read and print from kernel virtual''
can be even further shortened.
Type lbolt/ to read and print the value of the
(long) kernel variable, lbolt.
Displaying and writing to registers
You can examine the CPU's general registers
(and some pseudo-registers) with these commands:
- %register
-
This pushes the contents of the 32-, 16- or 8-bit register.
register can be any of the following:
- 32-bit registers
-
eax, ebx, ecx, edx,
esi, edi,
ebp, esp, eip,
efl
- 16-bit registers
-
cs, ds, es, fs, gs,
ax, bx, cx, dx,
si, di,
bp, sp, ip,
fl
- 8-bit registers
-
al, ah,
bl, bh,
cl, ch,
dl, dh
- %trap
-
push the trap number
- %ipl
-
push the interrupt priority level
You can modify the values of general-purpose
registers with these commands:
- w%register
-
write [TOS] into register; pop 1
- w%trap
-
write [TOS] into the trap number pseudo-register; pop 1
Register sets
The commands listed above can also be used
to access specific register sets.
Multiple sets of general registers may have been saved
on the kernel stack (one for each interrupt, trap, and so on).
For more information see
``Printing kernel stack traces''.
Register sets are numbered from 0 to 19, with 0 being the current
(most recent) set.
By default, the general-register commands use register set 0,
but you can override this with a ``register-set suffix'':
- /rsn
-
register set number n
Note that by combining suffixes, you can access
any register of any process.
For example, you can get the eax register from process 5's
register set 1 by typing:
%eax/u5rs1
This command will
push the contents of that register (%eax) in register set 1
(/rs1) of user process 5 (/u5).
CPU control registers
In addition to the general registers,
you can examine the values of CPU control registers with these commands:
- cr0
-
push the contents of register cr0
- cr2
-
push the contents of register cr2
- cr3
-
push the contents of register cr3
- cr4
-
push the contents of register cr4
Creating debugger variables
KDB allows you to create named variables that are stored in the
debugger and hold debugger values (numbers or strings).
Two KDB commands
apply to variables:
- = variable
-
store [TOS] in [variable]; pop 1
- vars
-
show values of debugger variables
Examples
- = variable
-
This command assigns a value to a debugger variable.
For example:
5 = abc
creates the variable abc if it does not exist,
and sets the variable equal to 5.
Now whenever you use the variable name, its
value is pushed onto the stack.
For example:
abc abc + 2 - p
will yield 8 (5 + 5 - 2).
Note that variable names share the same namespace
as debugger macros and kernel global symbols.
- vars
-
Look at all the existing variables.
Variables are shown in the following format:
name =
value
The vars command also lists macros, in this format:
name ::
value
Setting breakpoints
Set and modify breakpoints with these commands:
- B
-
Either of the following:
-
set breakpoint number [TOS] at address [TOS-1]; pop 2
-
set breakpoint number [TOS] at address [TOS-2] with command string [TOS-1]; pop 3
- b
-
Either of the following:
-
set first free breakpoint address [TOS]; pop 1
-
set first free breakpoint at address [TOS-1] with command string [TOS]; pop 2
- bn
-
set breakpoint (like b) and push breakpoint number
- brkoff
-
disable breakpoint number [TOS]; pop 1
- brkon
-
re-enable breakpoint number [TOS]; pop 1
- brksoff
-
disable all breakpoints
- brkson
-
re-enable all (disabled) breakpoints
- trace
-
set breakpoint number [TOS] trace count to [TOS-1]; pop 2
- clrbrk
-
clear breakpoint number [TOS]; pop 1
- clrbrks
-
clear all breakpoints
- clraddrbrks
-
clear all breakpoints for address [TOS]; pop 1
- curbrk
-
push the current breakpoint number, or -1 if
not entered from a breakpoint
- ?brk
-
show current breakpoint settings
You can have up to 20 breakpoints, numbered 0 through 19,
set at one time.
- B, b, and bn
-
The B command lets
you set specific breakpoints,
while the b command automatically picks the first un-set
breakpoint.
This example sets breakpoint 3 at a specific address:
0xD0125098 3 B
Normally, you will just set a breakpoint at a certain address.
For example:
read b
This sets an instruction breakpoint at the beginning of the
kernel read routine, using the next available breakpoint number.
When the specified address is executed (after
exiting from the debugger),
you enter the debugger again, with a message indicating which breakpoint
was triggered.
Debugger command strings can be added to the breakpoint commands.
Enter a quoted string of commands after the address:
read "stack" b
which is used as a series of debugger commands that
are executed when the breakpoint is triggered.
If there are several items in the string, separate them with spaces:
ie6unitdata_req "300 r/bio p" b
After these commands are executed, you are prompted for debugger
commands, as usual, unless the q (quit) command is executed
in the command string.
The bn command works like b except that it leaves the
breakpoint number on the value stack.
This is useful for macros.
You can append breakpoint-type suffixes to the
breakpoint commands (B, b, and bn).
By default, breakpoints are ``instruction'' breakpoints,
which trigger when
the specified address is executed.
The suffixes cause breakpoints to trigger on data accesses instead.
The breakpoint-type suffixes are:
- /a
-
data access breakpoint
- /m
-
data modify breakpoint
- /i
-
instruction execution breakpoint -- this is the default
- /io
-
I/O port access
With access and modify breakpoints, you can also
use operand-size suffixes to control the size of the
address range that will trigger the breakpoint.
The default is /l (4 bytes); you can also
use /w (word) and /b (byte).
(See the discussion of suffixes
in
``Reading and writing to memory''
for more information.)
- brkoff and brkon
-
These commands let you temporarily
disable and re-enable a breakpoint, instead of clearing
it with clrbrk and then re-entering it later.
This is especially handy for breakpoints with command strings.
- trace
-
This command sets a trace count for a breakpoint.
This causes the debugger to just print a message
and decrement the count
when the breakpoint is triggered,
instead of entering the debugger, until the count reaches zero.
Commands attached to the breakpoint are not executed.
- ?brk
-
Use this command to determine the current breakpoint settings.
Each set breakpoint is displayed with: the breakpoint number,
the address in hex, the symbolic address,
the current state, and the type:
0: 0xD003907C(read) ON /i
The possible states are:
- ON
-
set and enabled
- DISABLED
-
set, but currently disabled
- OFF
-
un-set (these breakpoints are not displayed by ?brk)
The possible types (in this example /i) are the same as
the breakpoint-type suffixes described earlier.
If a breakpoint has a non-zero trace count, that is displayed
after the breakpoint state.
If a breakpoint has a command string, it is displayed
at the end of the line.
For example, with a count of 5 and a stack command,
the above breakpoint would display as:
0: 0xD003907C(read) ON 0x5 /i "stack"
Single-stepping through instructions
You can use these commands for single-stepping:
- s
-
single step 1 instruction
- ss
-
single step [TOS] instructions; pop 1
- S
-
single step 1 instruction (passing calls)
- SS
-
single step [TOS] instructions (passing calls); pop 1
s and ss single-step all instructions.
S and SS single-step all instructions except
call instructions.
They do not step down into the called routine,
but instead skip ahead to the return from the call,
treating the whole subroutine sequence as a single instruction.
Branch-stepping through instructions
Four commands are available for branch-stepping:
- bs
-
branch step: execute until a branch is taken
- bss
-
branch step through [TOS] branches; pop 1
- lbr
-
show from-and to- address for last branch taken
- lcall
-
as call but using long long (8 bytes)
- lint
-
show from- and to- address for last interrupt or exception
Some advanced CPUs support a branch-stepping feature that
allows instruction execution to proceed until the next jump or call
instruction which causes a branch in program flow.
These commands allow use of this feature when available.
On CPUs that do not support branch-stepping,
these commands will not be active.
Examining kernel data structures
The ps command shows information about each
active process in the system.
This information includes process IDs, flags,
and command names, on one line,
with an additional line per LightWeight Process (LWP) within the process
with LWP IDs, flags,
states, and scheduling information.
The CPU field of an LWP line is blank for non-running LWPs;
otherwise it is the CPU number on which that
LWP is currently running.
Printing kernel stack traces
KDB provides the following commands to look at kernel stack traces:
- stack
-
kernel stack trace for the current process
- lstack
-
kernel stack trace for LWP [TOS]; pop 1
- tstack
-
``try'' kernel stack trace from [TOS]; pop 1
- stackargs
-
set maximum number of arguments in stack trace to [TOS]; pop 1
NOTE:
The argument to lstack can be specified
either as the address of the LWP structure
or -1 for the current LWP for the given CPU.
(-1 lstack is equivalent to the stack command.)
tstack is sometimes useful if lstack or stack
fail to give
a full stack trace for some reason.
(For example, another CPU might be hung and fails to
respond to the stack-trace
request.)
It takes a single argument, which is a stack pointer
value, and attempts
to find a potentially valid trace starting from some
stack address greater
than or equal to that value.
The output of stack, lstack and tstack
have the same format.
A typical stack trace
(for the current process, entered using <Ctrl><Alt>D)
looks like this:
(current) idle stack:
DEBUGGER ENTERED FROM USER REQUEST
kdcksysrq(D101FD40 D00DE624 81 20)........esp:FFFE9C94 ret:D008F592
*kdintr+0x186(1)...........................esp:FFFE9CD8 ret:D0011A3A
INTERRUPT TO devint0+0x78 from 8:D001218A (r0ptr:FFFE9CEC)
eax: 8 ebx: ------- ecx:FFFFFFFF edx: 8 efl: 246
esi: ------- edi: ------- esp:FFFE9D04 ebp: ------- regset: 0
idle(0 D00EDDD0 D106BC00).................esp:FFFE9D0C ret:D006F11F
*swtch(0 0 0)..............................esp:FFFE9D40 ret:D002464C
>use_private+0xAB()
The stack trace shows a history of which routine called which other
routine, up until the point the debugger was entered
(or in the case of a non-current process, until the process was
context-switched out).
The most-recently-entered routine is shown on the first line.
In the example, the debugger was entered from kdcksysrq,
which, in turn, was called by kdintr;
idle was called from swtch, and so on.
The stack trace ends at the point the kernel was entered from user mode.
In the case of a system process or an idle stack (as shown here)
where there is no user mode, the stack trace ends at the top-level
routine (use_private in this case).
(An idle stack is a per-CPU private stack which is used when the CPU is idle
or is otherwise not running an LWP.)
Routine trace format
The trace for each routine has four parts:
its address, the arguments passed to it,
the value of the esp register on entry to the routine,
and its return address.
For example:
kdcksysrq(D101FD40 D00DE624 81 20)........esp:FFFE9C94 ret:D008F592
- Address
-
The address that was called usually appears in symbolic form.
A routine name may also include:
- An asterisk (``*''): *kdintr+0x186
-
This means the routine was called indirectly.
There is insufficient information in the stack format
to be 100% sure of the correctness of indirect call traces.
Whenever you see an asterisk in a stack trace,
there is a small chance that some part of the stack
trace from that point on is incorrect.
- An offset (a plus sign (``+'') and a hex number): *kdintr+0x186
-
The offset may mean that the actual address called was
somewhere past the start of the indicated routine.
This will most likely happen if a subroutine was declared
static
.
Since the debugger only has access to global symbols,
it finds the nearest preceding global symbol.
The offset may also mean that the exact address
called cannot be determined.
The address displayed in this case is the return address into this
routine from the routine it called.
This will most likely happen if this routine was called indirectly
via a function pointer.
- Arguments
-
The arguments passed to the routine appear as a list of hex numbers,
enclosed in parentheses.
Since the actual number of arguments passed cannot be
determined, KDB assumes that each routine has no more than
a certain maximum number of arguments. The default
is three, but you can change it with the stackargs command.
If a routine actually has:
- Fewer arguments than displayed
-
Only the first ones are real.
In rare cases when the debugger can deduce that a
routine could not have been called with
the maximum number of arguments (because there is not
enough room on the stack),
it displays only the maximum possible number of arguments.
In the above stack trace, the call to kdintr is
shown with only one argument ``(1)''.
- More arguments than displayed
-
Increase the number with stackargs and
then display the stack trace again,
or dump out a portion of the stack directly
in order to see all the arguments (continue to
``Trap frames''
for details).
- esp register
-
The value of the esp register on entry to the routine
is shown as a hex number following
esp:
.
This value can be used as a ``frame pointer'' to access
arguments and local variables for the routine.
The following diagram illustrates the stack layout:
| . . . |
+-------------------------+
[ESP] + 8 | argument 2 |
+-------------------------+
[ESP] + 4 | argument 1 |
+-------------------------+
[ESP] ---> | return address |
+-------------------------+
[ESP] - 4 | local or saved register |
+-------------------------+
[ESP] - 8 | local or saved register |
+-------------------------+
| . . . |
For example, if you want to see all the arguments to a routine that
takes five arguments, find its esp value from the
stack trace -- say
0xD2473CD4 -- and enter these commands:
0xD2473CD4 4 + 5 4 * dump
or, more succinctly:
0xD2473CD8 14 dump
- Return address
-
This is the address within its caller to which the routine returns.
It is shown as a hex number following
ret:
.
Trap frames
In addition to lines for each routine, stack traces
will often include
``trap frames'' created when an event causes
suspension of current processing,
saving all register values on the stack.
Typical events are interrupts, hardware exceptions,
and system calls.
Trap frames are three lines each, starting with
an uppercase, non-indented keyword (like INTERRUPT
in the following example). The next two lines contain
the values of the registers at the time the event occurred.
The first line of a trap frame is in one of these formats:
INTERRUPT TO devint0+0x78 from 8:D001218A (r0ptr:E0000D84)
TRAP 0x1 from 8:D001218A (r0ptr:E0000D94)
TRAP EVENT from 17:830676D (r0ptr: E0000D94, ss:esp: 1F:80468E8)
SYSTEM CALL from 17:830676D (r0ptr: E0000D94, ss:esp: 1F:80468E8)
SIGNAL <Return> from 17:830676D (r0ptr: E0000D94, ss:esp: 1F:80468E8)
These represent interrupts, hardware exception traps, trap event processing,
system calls, and returns from old-style signal handlers, respectively.
The number after TRAP is the hardware exception number;
the most common are 0x1 for breakpoint traps
and 0xE for page faults.
See /usr/include/sys/trap.h for a full list of trap numbers.
The colon-separated numbers after the word from
are the segment
and offset (cs and eip) at the time the event occurred.
The values in parentheses show the r0ptr value for the beginning of
the trap frame (for example, the address of the saved eax register;
see /usr/include/sys/reg.h for the layout of saved registers),
and the user stack pointer segment and offset at the time
the event occurred.
The user stack information is only displayed if the trap frame is
for an entry into the kernel from user mode.
Resetting the numeric base
If you do not start numbers with ``0o'' (for octal)
or ``0x'' (for hex), KDB
assumes they are in the default numeric base.
Initially, the defaults for both input and output
are set to 16 (hex), but you can use these
commands to change them:
- ibase
-
set default input base to [TOS]; pop 1
- ibinary
-
set default input base to 2
- ioctal
-
set default input base to 8
- idecimal
-
set default input base to 10
- ihex
-
set default input base to 16
- obase
-
set output base to [TOS]; pop 1
- ooctal
-
set output base to 8
- odecimal
-
set output base to 10
- ohex
-
set output base to 16
Converting address spaces
Use these commands to convert a virtual address to a physical address:
- kvtop
-
convert kernel virtual address [TOS] to physical
- uvtop
-
convert user proc number [TOS] address [TOS-1] to physical; pop 1
Performing conditional execution
KDB provides two commands for conditional execution:
- then
-
if [TOS] = 0, skip to endif; pop 1
- endif
-
end scope of then command
In other words, a sequence like:
condition then commands endif
executes commands if and only if the condition
is true (non-zero).
These are mostly useful for macros and breakpoint command strings.
For example, imagine you wish to set a
breakpoint for when the function
inb is called with 2E as its first argument.
Use the following command:
inb "%esp 4 + r 2E != then q" b
This sets a breakpoint at inb,
but enters the debugger only if the content of %esp+4
is equal to 2E.
This works because esp points to the
return address on the stack,
and the longword after that is the first argument.
For the second argument, you would add 8 instead of 4
(see
``Printing kernel stack traces''
for details of the stack layout).
If you do a ?brk command, the display for
that breakpoint includes the string of debugger
commands:
0: 0xD003907C(inb) ON /i "%esp 4 + r 2E != then q"
Calling a kernel function
Use these commands to call an arbitrary kernel function:
- call
-
call the function at address [TOS-1] with [TOS] arguments,
given by:
[TOS-([TOS]+1)],...[TOS-2]; pop [TOS]+2
- vcall
-
call the function at address [TOS-1] with [TOS] arguments,
given by:
[TOS-([TOS]+1)],...[TOS-2]; pop [TOS]+2
- lcall
-
as call, but using long long arguments
Use the call command to call a function which returns a value;
this value will be the [TOS] value on the value stack
after the call command.
Use the vcall (``void call'') command to
call a function with no return value.
To call psignal with two arguments,
the current process and 9, type:
curproc r 9 psignal 2 call
curproc r gives the value of the current process,
the first argument, and 9 is the second
argument.
psignal is converted into the address at which
that function can be called, and 2 specifies
the number of arguments to pass to psignal.
Performing a system dump
Some systems support dumping all of memory to disk with a
sysdump kernel function.
For these systems, you can take a dump by calling
this function with the command:
sysdump 0 vcall
All of memory and the current state is dumped to the
dump partition on the disk, so
you can use
crash(1M)
to do a
postmortem.
Miscellaneous commands
Some miscellaneous KDB commands are:
- findsym
-
Print kernel symbol with address closest to [TOS]; pop 1
- dis
-
Disassemble [TOS] instructions starting at address [TOS-1]; pop 2
- nonverbose
-
Turn verbose mode off
- verbose
-
Turn verbose mode on
- newdebug
-
Switch to another debugger on next debugger entry
- help
-
Print a help message
- ?
-
Print a help message (same as help)
- cmds
-
Print a list of all debugger commands
Writing macros
KDB provides the ability to assign a string of
commands to a single new command name, called a macro.
When a debugging task involves repeating the
same set of commands many times (possibly doing other
things in between),
it is easier to define a macro and use it in place
of the whole set of commands.
These commands are used for macros:
- :: macro
-
define macro as command string [TOS]; pop 1
- P
-
print [TOS] in raw form; pop 1
- PP
-
print [TOS] values in raw form, from [TOS-[TOS]],...[TOS-1]; pop [TOS]+1
- vars
-
show values of debugger macros and variables
Examples
- :: macro
-
Use this command to define macros.
For example:
"curproc r 16 - p" :: newaddr
Note that macro names share the same namespace
as debugger variables and kernel global symbols.
- P and PP
-
These commands are provided to aid in writing macros.
P and PP print values in raw form,
without the embellishments provided by the
p command, such as quotes around strings
and automatic newlines after each value.
This allows complete control over formatting.
For example, the input:
"The value of curproc is " curproc r ".\n" 3 PP
might produce the output:
The value of curproc is 0xD1011E80
To put something like this into a macro means putting
strings inside strings,
so you will have to escape the inner quotes:
"\"The value of curproc is \" curproc r \".\n\" 3 PP" :: pcurproc
- vars
-
Use this command to show the macro definitions.
Macros are shown in this format:
name :: value
Note that the vars command also shows
the values of variables, in this format:
name = value
Output control
Some KDB commands send a great deal of output to the screen.
There are various ways you can slow the output down if necessary:
-
XON/XOFF flow control is supported,
using the <Ctrl>Q/<Ctrl>S keys.
-
Output may be interrupted (and command execution aborted)
with either the <Del> or <Ctrl>C key.
-
Output is automatically paginated.
That is, after each screenful of output,
a
[MORE]---
prompt is
output, and KDB waits for an input key before continuing.
The number of lines per screen is controlled by the db_nlines
global variable.
You can change the number of lines to, for example, 60, with:
60 db_nlines w
Executing debugger commands at boot time
KDB allows you to specify an arbitrary command sequence to be executed
at boot time, when the system is coming up.
You can do this by writing the commands into the file
$ROOT/$MACH/etc/conf/cf.d/kdb.rc, then rebuilding the kernel
with
idbuild(1M).
At boot time, after the (possibly blank) string is executed,
the system enters KDB at the kdb>>
prompt,
unless a q command was executed as part of the string -- just
like conditional breakpoints.
(A non-existent or zero-length kdb.rc file acts as a single
q command, so KDB is not entered.)
Using a serial terminal
KDB can be used from a serial terminal as well as the console.
This is particularly useful if you are trying to debug a scenario that
involves graphics or multiple virtual terminals on the console.
Before you attempt to use the debugger from a serial terminal,
make sure there is a
getty(1M)
or
ttymon(1M)
running on it.
It may be either logged in or waiting at the login prompt.
This ensures that the baud rate and other parameters are properly set.
You can switch from the console to a terminal,
and vice-versa, with the newterm command.
This immediately switches you to the new terminal.
The debugger continues to use this terminal until you give it
the newterm command again, even if you exit and re-enter KDB.
The newterm command takes two arguments.
The first argument is a string which is the device
name of a console-capable
device driver (for example, iasy).
The second argument is the unit number of the actual device
(usually the minor device number) you wish to use.
Once you exit from KDB, you can invoke it again from either
the console or a serial terminal.
Use the kdb command to invoke the debugger
from a terminal; <Ctrl><Alt>D only works from the
console.
Regardless of where you invoke KDB, its I/O appears
where you directed it during the last KDB session.
Entering the debugger from a driver
If you are debugging a device driver or another part
of the kernel,
you can directly invoke the kernel debugger by including
this code in your driver:
#include <sys/xdebug.h>
(*cdebugger) (DR_OTHER, NO_FRAME);
DR_OTHER
tells the debugger that the reason for entering is other.
See sys/xdebug.h for a list of other reason codes.
Note that this mechanism cannot be used for debugging early kernel
startup code or driver init routines, since the debugger cannot
be used until its init routine (kdb_init) has been
called.
Configuring KDB
The Kernel Debugger (KDB) can be configured two ways: it can be
enabled at all times, or it can be enabled only on demand. If
demand mode is chosen, KDB must be manually enabled before it can
be invoked.
Enabling KDB
KDB can be invoked by a console key sequence, by a system crash,
or by the kdb command. However, it can only be invoked if it
is enabled. KDB is enabled when it is loaded into the kernel.
If you choose to enable it always, KDB will be loaded into the
kernel at all times and can be invoked at any time.
If you choose to enable it on demand, KDB must first be loaded
with the modadmin -l kdb command before it can be invoked.
KDB can be unloaded with the modadmin -U kdb command.
The advantage to having KDB always enabled is that it will
be ready even for unanticipated use. The disadvantage is that
whenever it is enabled, KDB consumes about 250K of main memory.
Most of this memory is used for the kernel symbol table, which
is locked in memory whenever KDB is loaded, unless the PAGESYMTAB
tunable is set to 2.
This will affect system performance on systems with small memories.
Security checks
KDB also provides optional security checks which prohibit an
unauthorized user from invoking the kernel debugger. If you do not
turn on these security checks, the kernel debugger can be entered
from a key sequence on the console, presenting a potential security
breach if your console is not physically secure.
Disabling the <Ctrl><Alt>D sequence
If the security check is not enabled, a user can type a key
sequence at the console without having to log into the computer
and enter the kernel debugger. Unless the console is in a room
that is locked or accessible to only a controlled group of people,
this security check should be enabled to prevent a security breach.
KDB can only be called from the
console using <Ctrl><Alt>D
if the kdb_security flag was set to 0 when
the kernel was built.
To disable the <Ctrl><Alt>D key sequence, reset
the kdb_security flag by using
/etc/conf/bin/idtune to change
the KDBSECURITY tunable to 1.
Note that the flag setting does not affect the kdb command.
In order to enter the debugger after disabling this
key sequence, the user must log into the
computer using a privileged account and then type the kdb command.
Command summary
- +
-
compute [TOS-1] + [TOS]; pop 2; push result
- -
-
compute [TOS-1] - [TOS]; pop 2; push result
- *
-
compute [TOS-1] * [TOS]; pop 2; push result
- /
-
compute [TOS-1] / [TOS]; pop 2; push result
- %
-
compute [TOS-1] % [TOS]; pop 2; push result
- >>
-
compute [TOS-1] >> [TOS]; pop 2; push result
- <<
-
compute [TOS-1] << [TOS]; pop 2; push result
- <
-
compute [TOS-1] < [TOS]; pop 2; push result
- >
-
compute [TOS-1] > [TOS]; pop 2; push result
- ==
-
compute [TOS-1] == [TOS]; pop 2; push result
- !=
-
compute [TOS-1] != [TOS]; pop 2; push result
- &
-
compute [TOS-1] & [TOS]; pop 2; push result
- |
-
compute [TOS-1] | [TOS]; pop 2; push result
- ^
-
compute [TOS-1] ^ [TOS]; pop 2; push result
- &&
-
compute [TOS-1] && [TOS]; pop 2; push result
- ||
-
compute [TOS-1] || [TOS]; pop 2; push result
- !
-
replace [TOS] with ![TOS]
- ++
-
replace [TOS] with [TOS] + 1
- --
-
replace [TOS] with [TOS] - 1
- %register
-
push the contents of the 32-, 16- or 8-bit register.
register can be any of the following:
- 32-bit registers
-
eax, ebx, ecx, edx,
esi, edi,
ebp, esp, eip,
efl
- 16-bit registers
-
cs, ds, es, fs, gs,
ax, bx, cx, dx,
si, di,
bp, sp, ip,
fl
- 8-bit registers
-
al, ah,
bl, bh,
cl, ch,
dl, dh
- %trap
-
push the trap number
- %ipl
-
push the interrupt priority level
- = variable
-
store [TOS] in [variable]; pop 1
- :: macro
-
define [macro] as command string [TOS]; pop 1
- ?
-
print a help message (same as help)
- ?brk
-
show current breakpoint settings
- B
-
Either of the following:
-
set breakpoint number [TOS] at address [TOS-1]; pop 2
-
set breakpoint number [TOS] at address [TOS-2] with command string [TOS-1]; pop 3
- b
-
Either of the following:
-
set first free breakpoint address [TOS]; pop 1
-
set first free breakpoint at address [TOS-1] with command string [TOS]; pop 2
- bn
-
set breakpoint (like b) and push breakpoint number
- brkoff
-
disable breakpoint number [TOS]; pop 1
- brkon
-
re-enable breakpoint number [TOS]; pop 1
- brksoff
-
disable all breakpoints
- brkson
-
re-enable all (disabled) breakpoints
- bs
-
branch step: execute until a branch is taken
- bss
-
branch step through [TOS] branches; pop 1
- call
-
call the function at address [TOS-1] with [TOS] arguments,
given by [TOS-([TOS]+1)],...[TOS-2]; pop [TOS]+2;
push function return value
- clraddrbrks
-
clear all breakpoints for address [TOS]; pop 1
- clrbrk
-
clear breakpoint number [TOS]; pop 1
- clrbrks
-
clear all breakpoints
- clrstk
-
pop all values
- cmds
-
print a list of all debugger commands
- cr0
-
push the contents of register cr0
- cr2
-
push the contents of register cr2
- cr3
-
push the contents of register cr3
- cr4
-
push the contents of register cr4
- curbrk
-
push the current breakpoint number, or -1
if not entered from a breakpoint
- dis
-
disassemble [TOS] instructions starting at address [TOS-1]; pop 2
- dump
-
show [TOS] bytes starting at virtual address [TOS-1]; pop 2
- dup
-
push [TOS]
- endif
-
end scope of then command
- findsym
-
print kernel symbol with address closest to [TOS]; pop 1
- fdump
-
show [TOS-1] formatted items at [TOS-2] with format [TOS]; pop 3
- help
-
print a help message
- ibase
-
set default input base to [TOS]; pop 1
- ibinary
-
set default input base to 2
- idecimal
-
set default input base to 10
- ihex
-
set default input base to 16
- ioctal
-
set default input base to 8
- kvtop
-
convert kernel virtual addr [TOS] to physical
- lbr
-
show from- and to- address for last branch taken
- lint
-
show from- and to- address for last interrupt or exception
- lstack
-
kernel stack trace for LWP [TOS]; pop 1
- newterm
-
switch KDB console I/O to device [TOS-1] unit number [TOS]; pop 2
- newdebug
-
switch to another debugger on next debugger entry
- nonverbose
-
turn verbose mode off
- obase
-
set output base to [TOS]; pop 1
- odecimal
-
set output base to 10
- ohex
-
set output base to 16
- ooctal
-
set output base to 8
- P
-
print [TOS] in raw form; pop 1
- p
-
print [TOS]
- PP
-
print [TOS] values in raw form,
from [TOS-[TOS]],...[TOS-1]; pop [TOS]+1
- pop
-
pop 1 value
- ps
-
show process information
- q
-
exit from the debugger
- r
-
replace [TOS] with the value at virtual address [TOS]
- S
-
single step 1 instruction (passing calls)
- s
-
single step 1 instruction
- SS
-
single step [TOS] instructions (passing calls); pop 1
- ss
-
single step [TOS] instructions; pop 1
- stack
-
kernel stack trace for the current process
- stackargs
-
set the maximum number of arguments in the stack trace to [TOS]; pop 1
- stk
-
print all values on the stack
- then
-
if [TOS] = 0, skip to endif; pop 1
- trace
-
set breakpoint number [TOS] trace count to [TOS-1]; pop 2
- tstack
-
``try'' kernel stack trace from [TOS]; pop 1
- uvtop
-
convert user process number [TOS] address [TOS-1] to physical; pop 1
- vars
-
show values of debugger variables
- vcall
-
call the function at address [TOS-1] with [TOS] arguments,
given by [TOS-([TOS]+1)],...[TOS-2]; pop [TOS]+2
- verbose
-
turn verbose mode on
- w
-
write [TOS-1] into virtual address [TOS]; pop 2
- w%register
-
write [TOS] into register; pop 1
- w%trap
-
write [TOS] into the trap number pseudo-register; pop 1
Command suffixes
- Operand size
-
- /b
-
byte
- /w
-
word (2 bytes)
- /l
-
long (4 bytes) -- this is the default
- /L
-
long long (8 bytes)
- Address space
-
- /k
-
kernel virtual -- this is the default
- /p
-
physical
- /io
-
I/O port
- /un
-
user process number n virtual
- /cpun
-
CPU number n
- /cn
-
CPU number n (same as cpun)
- Register set
-
- /rsn
-
register set number n
- Breakpoint type
-
- /a
-
data access breakpoint
- /m
-
data modify breakpoint
- /i
-
instruction execution breakpoint -- this is the default
References
crash(1M),
dc(1)
30 January 1998
© 1998 The Santa Cruz Operation, Inc. All rights reserved.