160 likes | 355 Views
System Crash Dump. Version C.00 H4264S Module 9 Slides. System Panic. An HP-UX system will panic when it encounters a situation in which it does not know how to respond. In these situations, the system halts (it panics). There are three types of system panics: - HPMC - ToC - PANIC.
E N D
System Crash Dump Version C.00 H4264S Module 9 Slides
System Panic • An HP-UX system will panic when it encounters a situation in which it does not know how to respond. In these situations, the system halts (it panics). • There are three types of system panics:- HPMC - ToC - PANIC
High Priority Machine Checks Hardware initiated failures Usually indicate a hardware problem BUT Can also be caused by software.
Transfer of Control • Can be initiated by the user, usually because the system has hung • Can be caused by MC/ServiceGuard, while attempting to ensure data integrity
Panics • Panics are software failures in kernel mode. • The HP-UX kernel has detected an internal problem. • The system is “stopping,” because continuing to process could cause more problems. Stopping is safer.
System Crash Dump • A crash dump is a core image stored to the dump area upon system panic. The dump area is usually primary swap, but it is configurable. • The /sbin/savecrash command causes the dump area contents to be saved to dump files. It must be executable. • A crash dump consists of the following files in the following directories: /var/adm/crash crash.1 crash.0 INDEX image.0.1 image.0.2 vmunix.0
1 2 Big Picture Kernel OS Tables CPU Dump/Swap #2 Dump/Swap /var Memory Disks /var/adm/crash/crash.0 INDEX image.0.1 image.0.2 vmunix.0
Memory Classes As of HP-UX 11.00, physical memory is divided into 8 classes: Kernel text Kernel data (static and dynamic tables) F/S Data • kcode • ksdata • kddata • fsdata • bcache • userpg • ustack • unused BC Text Data Text Data Text Data Use the crashconf -v command to view the configured memory classes. Unused Pages Memory
Configuring Memory Classes and Dump Devices The crashconf command allows configuration of dynamic memory classes and dump devices while the system is up and running: # crashconf -i userpg # crashconf -e fsdata # crashconf /dev/vg01/dumplvol # crashconf -v # crashconf -v CLASS PAGES INCLUDED IN DUMP DESCRIPTION -------- ---------- ---------------- ------------------------------------- UNUSED 36007 no, by default unused pages USERPG 7911 no, by default user process pages BCACHE 11559 no, by default buffer cache pages KCODE 1507 no, by default kernel code pages USTACK 191 yes, by default user process stacks FSDATA 89 yes, by default file system metadata KDDATA 4093 yes, by default kernel dynamic data KSDATA 4179 yes, by default kernel static data Total pages on system: 65536 Total pages included in dump: 8552 DEVICE OFFSET(kB) SIZE (kB) LOGICAL VOL. NAME ------------ ---------- ---------- ------------ ------------------------- 31:0x005000 72544 245760 64:0x000002 /dev/vg00/lvol2 ---------- 245760
Text Data How Much Dump Space Is Needed? Selective items are dumped! Kernel text Swap (1.5 GB) Kernel data (OS tables) Buffer Cache Text data Text Data Free Pages Disk Memory (2 GB)
Dump Configuration Startup Files Pool of startup/shutdown scripts Data Configuration Files rc script /sbin/rc1.d /sbin/init.d /etc/rc.config.d start S080crashconf S440savecore crashconf savecrash crashconf savecrash . . . . . .
Obtaining the Crash Dump Message Procedure for Extracting the Dump Message from the Crash Files # crashutil -v COREFILE /var/adm/crash/crash.0 \ /tmp/corefile /tmp/kernfile # cd /tmp # adb -k kernfile corefile msgbuf+8/s . . . Panic: Data page fault . . . $q # Note: Enter $q to quit/terminate /var/adm/crash/crash.0 /tmp INDEX image.0.1 vmunix.0 kernfile corefile
The Interactive Editor — ied • Provides command rerun, recall, search, and editing in any interactive utility • Uses the EDITOR shell variable • Creates utility command history in -hfilename • export EDITOR=vi • ied -h ~root/.q4_history interactive utility or command
Another Debugger — q4 • Displays kernel, process, and thread symbols, structures, and unions • Uses Perl scripts • Includes commands in /usr/contrib/bin • Includes documentation in /usr/contrib/doc • Installs from compressed tarball in /usr/contrib/lib • cd /usr/contrib/lib • uncompress Q4Lib.tar.Z • tar xvf Q4Lib.tar Prepare the kernel for debugging with q4: • q4pxdb/stand/vmunix# Live Kernel • q4pxdb/var/adm/crash/crash.0/vmunix# Crash dump Kernel
q4 Documentation, Commands, and Syntax • q4 help lists, commands, topics, and syntax q4>help q4>help topic q4>help examine • Commands to retrieve basic system information and status q4>load struct utsname from &utsname; print -tSystem versions q4>examine panicstr using s Panic string q4> examine &msgbuf+8 using s Console message buffer q4> examine &time using Y System time at the crash q4> examine &boot_string using s Bootstring (device path and kernel name) q4> examine &nproc using DMaximum # of processes (full word decimal) q4> examine &maxfiles_lim using DMaximum # of file descriptors/proc q4> examine &maxdsiz using DMaximum amount of private data q4> examine &maxvgs using DMaximum # of volume groups • exit or quit or bye to exit the debugger