The SDA CLUE command invokes the Crash Log Utility Extractor, which captures specific crash dump information and, upon system reboot, preserves it in a file with the following naming scheme:
You enter CLUE extension commands at the SDA prompt. For example:
SDA> CLUE CONFIG
You can get full help on CLUE by entering HELP CLUE at the SDA>
5.1 Overview of SDA CLUE Extension
SDA CLUE (Crash Log Utility Extractor) commands automate the analysis of crash dumps and maintain a history of all fatal bugchecks on either a standalone or cluster system. You can use SDA CLUE commands in conjunction with SDA to collect and decode additional dump file information not readily accessible through standard SDA commands. SDA CLUE extension commands can summarize information provided by certain standard SDA commands and provide additional detail for some SDA commands. For example, SDA CLUE extension commands can quickly provide detailed extended QIO processor (XQP) summaries. You can also use SDA CLUE commands interactively on a running system to help identify performance problems.
You can use all CLUE commands when analyzing crash dumps; the only CLUE commands that are not allowed when analyzing a running system are CLUE CRASH, CLUE ERRLOG, CLUE HISTORY, and CLUE STACK.
When you reboot the system after a system failure, you automatically invoke SDA by default. To facilitate better crash dump analysis, SDA CLUE commands automatically capture and archive summary dump file information in a CLUE listing file.
A startup command procedure initiates commands that do the following:
The CLUE HISTORY command adds a one-line summary entry to a history file and saves the following output from SDA CLUE commands in the listing file:
The contents of this CLUE list file can help you analyze a system failure. If these files accumulate more space than the threshold allows (default is 5000 blocks), the oldest files are deleted until the threshold limit is reached. You can also customize this threshold using the CLUE$MAX_BLOCKS logical name.
For additional information on the contents of the CLUE listing file, see the reference section on CLUE HISTORY.
It is important to remember that CLUE$nodename_ddmmyy_hhmm.LIS contains only an overview of the crash dump and does not always contain enough information to determine the cause of the crash. The dump itself should always be saved using the procedures described in Section 2.2.2 and Section 2.2.4.
To inhibit the running of CLUE at system startup, define the logical
CLUE$INHIBIT in the SYLOGICALS.COM file as /SYS TRUE.
5.2 Displaying Data with CLUE
To invoke a CLUE command, enter the command at the SDA prompt. For example:
SDA> CLUE CONFIG
DOSD (Dump Off System Disk) allows you to write the system dump file to a device other than the system disk. For SDA CLUE to be able to correctly find the dump file to be analyzed after a system crash, you need to perform the following steps:
In the following example, the dump file has been placed on device $3$DUA25, which has the label DMP$DEV. You need to add the following commands to SYS$MANAGER:SYCONFIG.COM:
$ MOUNT/SYSTEM/NOASSIST $3$DUA25: DMP$DEV DMP$DEV $ DEFINE/SYSTEM CLUE$DOSD_DEVICE DMP$DEV
The following pages describe the SDA CLUE extension commands.
Displays key information, such as the PC of the caller, from the active call frames at the time of the crash.
CLUE CALL_FRAME [/CPU [cpu-id|ALL]
ALLWhen used with /CPU, it requests information about all CPUs in the system. When used with /PROCESS, it requests information about all processes that exist in the system.
cpu-idWhen used with /CPU, it gives the number of the CPU for which information is to be displayed. Use of the cpu-id parameter causes the CLUE CALL_FRAME command to perform an implicit SET CPU command, making the indicated CPU the current CPU for subsequent SDA commands.
process-nameWhen used with /PROCESS, it gives the name of the process for which information is to be displayed. Use of the process-name parameter, the /ADDRESS qualifier, the /INDEX qualifier, or the /IDENTIFICATION qualifier causes the CLUE CALL_FRAME command to perform an implicit SET PROCESS command, making the indicated process the current process for subsequent SDA commands. You can determine the names of the processes in the system by issuing a SHOW SUMMARY command.
The process-name can contain up to 15 letters and numerals, including the underscore (_) and dollar sign ($). If it contains any other characters, you must enclose the process-name in quotation marks (" ").
/ADDRESS=nSpecifies the PCB address of the desired process when used with CLUE CALL_FRAME/PROCESS.
/CPU [cpu-id|ALL]Indicates that the call frame for a CPU is required. Specify the CPU by its number or use ALL to indicate all CPUs.
/IDENTIFICATION=nSpecifies the identification of the desired process when used with CLUE CALL_FRAME/PROCESS.
/INDEX=nSpecifies the index of the desired process when used with CLUE CALL_FRAME/PROCESS.
/PROCESS [process-name|ALL]Indicates that the call frame for a process is required. The process should be specified with either one of the qualifiers /ADDRESS, /IDENTIFICATION, or /INDEX, or by its name, or by using ALL to indicate all processes.
The CLUE CALL_FRAME command displays call chain information for a process or a CPU. The process context calls work on both the running system and dump file; the CPU context calls only on dump files.
If neither /CPU nor /PROCESS is specified, the parameter (CPU-id or process-name) is ignored and the call frame for the SDA current process is displayed.
SDA>CLUE CALL/PROCESS IPCACP Call Chain: Process index: 000B Process name: IPCACP PCB: 8136EF00 ----------------------------------------------------------------------- Procedure Frame Procedure Entry Return Address --------------- ---------------------------------- --------------------------- 7FFA1CA0 Null 800C8C90 SCH$WAIT_PROC_C 7FFA1D00 Stack 800D9250 SYS$HIBER_C 0003045C IPCACP+0003045C 7FFA1D50 Stack 00030050 IPCACP+00030050 800D11C8 EXE$CMKRNL_C+000D8 7FFA1E60 Null 800B6120 EXE$BLDPKTSWPR_C 7FFA1E78 Null 800B6120 EXE$BLDPKTSWPR_C 7FFA1EC0 Null 80248120 NSA$CHECK_PRIVILEGE_C 7FFA1F00 Null 80084640 EXE$CMODEXECX_C 7FFA1F70 Stack 800D10F0 EXE$CMKRNL_C 80084CC8 EXE$CMODKRNL_C+00198 7B01FAB0 Stack 00030010 IPCACP+00030010 83EA3454 SYS$IMGSTA_C+00154 7B01FB10 Stack 83EA3300 SYS$IMGSTA_C 83D99CC4 EXE$PROC_IMGACT_C+00384 7B01FBA0 Stack 83D99BA0 EXE$PROC_IMGACT_C+00260 83D99B9C EXE$PROC_IMGACT_C+0025C
In this example, the CLUE CALL_FRAME command displays the call frame from the process IPCACP.
SDA>CLUE CALL/CPU ALL Call Chain: Process index: 0000 Process name: NULL PCB: 827377C0 (CPU 0) --------------------------------------------------------------------------------- Procedure Frame Procedure Entry Return Address --------------- ---------------------------------- --------------------------- 8F629D28 Null 80205E00 SYS$SCS+05E00 8F629D68 Null 8020A850 SCS$REC_MSGREC_C 8F629D98 Null 914A5340 SYS$PBDRIVER+07340 8F629DB8 Null 914A4FD0 SYS$PBDRIVER+06FD0 8F629DE0 Stack 914AACF0 SYS$PBDRIVER+0CCF0 914AE5CC SYS$PBDRIVER+105CC 8F629E50 Stack 914AE418 SYS$PBDRIVER+10418 800503B0 EXE_STD$QUEUE_FORK_C+00350 8F629F88 Null 800E95F4 SCH$WAIT_ANY_MODE_C 8F629FD0 Stack 800D0F80 SCH$IDLE_C 800E92D0 SCH$INTERRUPT+00BB0 Call Chain: Process index: 0000 Process name: NULL PCB: 827377C0 (CPU 2) --------------------------------------------------------------------------------- Procedure Frame Procedure Entry Return Address --------------- -------------------------------- --------------------------- 90FCBF88 Null 800E95F4 SCH$WAIT_ANY_MODE_C 90FCBFC8 Null 800E95F4 SCH$WAIT_ANY_MODE_C 90FCBFD0 Stack 800D0F80 SCH$IDLE_C 800E92D0 SCH$INTERRUPT+00BB0 Call Chain: Process index: 0000 Process name: NULL PCB: 827377C0 (CPU 6) --------------------------------------------------------------------------------- Procedure Frame Procedure Entry Return Address --------------- ------------------------------ --------------------------- 90FCBF88 Null 800E95FA SCH$WAIT_ANY_MORE_c 90FD9F88 Null 800E95F4 SCH$WAIT_ANY_MODE_C 90FD9FD0 Stack 800D0F80 SCH$IDLE_C 800E92D0 SCH$INTERRUPT+00BB0
In this example, CLUE/CPU ALL shows the call frame for all CPUs.
Performs housekeeping operations to conserve disk space.
CLUE CLEANUP performs housekeeping operations to conserve disk space. To avoid filling up the system disk with listing files generated by CLUE, CLUE CLEANUP is run during system startup to check the overall disk space used by all CLUE$*.LIS files.
If the CLUE$COLLECT:CLUE$*.LIS files occupy more space than the logical CLUE$MAX_BLOCKS allows, then the oldest files are deleted until the threshold is reached. If this logical name is not defined, a default value of 5,000 disk blocks is assumed. A value of zero disables housekeeping and no check on the disk space is performed.
SDA> CLUE CLEANUP %CLUE-I-CLEANUP, housekeeping started... %CLUE-I-MAXBLOCK, maximum blocks allowed 5000 blocks %CLUE-I-STAT, total of 4 CLUE files, 192 blocks.
In this example, the CLUE CLEANUP command displays that the total number of blocks of disk space used by CLUE files does not exceed the maximum number of blocks allowed. No files are deleted.
Displays the system, memory, and device configurations.
/ADAPTERDisplays only the part of the system configuration that contains information about the adapters and devices on the system.
/CPUDisplays only the part of the system configuration that contains information about the CPUs.
/MEMORYDisplays only the part of the system configuration that contains information about the layout of physical memory.
CLUE CONFIG displays the system, memory, and device configurations. If no qualifier is specified, the entire system configuration is displayed (memory, CPUs, adapters, and devices), plus additional system information.
Displays a crash dump summary.
CLUE CRASH displays a crash dump summary, which includes the following items:
- Bugcheck type
- Current process and image
- Failing PC and PS
- Executive image section name and offset
- General registers
- Failing instructions
- Exception frame, signal and mechanism arrays (if available)
- CPU state information (spinlock related bugchecks only)
SDA> CLUE CRASH Crash Time: 30-AUG-1996 13:13:46.83 Bugcheck Type: SSRVEXCEPT, Unexpected system service exception Node: SWPCTX (Standalone) CPU Type: DEC 3000 Model 400 VMS Version: X6AF-FT2 Current Process: SYSTEM Current Image: $31$DKB0:[SYS0.][SYSMGR]X.EXE;1 Failing PC: 00000000.00030078 SYS$K_VERSION_01+00078 Failing PS: 00000000.00000003 Module: X Offset: 00030078 Boot Time: 30-AUG-1996 09:06:22.00 System Uptime: 0 04:07:24.83 Crash/Primary CPU: 00/00 System/CPU Type: 0402 Saved Processes: 18 Pagesize: 8 KByte (8192 bytes) Physical Memory: 64 MByte (8192 PFNs, contiguous memory) Dumpfile Pagelets: 98861 blocks Dump Flags: olddump,writecomp,errlogcomp,dump_style Dump Type: raw,selective EXE$GL_FLAGS: poolpging,init,bugdump Paging Files: 1 Pagefile and 1 Swapfile installed Stack Pointers: KSP = 00000000.7FFA1C98 ESP = 00000000.7FFA6000 SSP = 00000000.7FFAC100 USP = 00000000.7AFFBAD0 General Registers: R0 = 00000000.00000000 R1 = 00000000.7FFA1EB8 R2 = FFFFFFFF.80D0E6C0 R3 = FFFFFFFF.80C63460 R4 = FFFFFFFF.80D12740 R5 = 00000000.000000C8 R6 = 00000000.00030038 R7 = 00000000.7FFA1FC0 R8 = 00000000.7FFAC208 R9 = 00000000.7FFAC410 R10 = 00000000.7FFAD238 R11 = 00000000.7FFCE3E0 R12 = 00000000.00000000 R13 = FFFFFFFF.80C6EB60 R14 = 00000000.00000000 R15 = 00000000.009A79FD R16 = 00000000.000003C4 R17 = 00000000.7FFA1D40 R18 = FFFFFFFF.80C05C38 R19 = 00000000.00000000 R20 = 00000000.7FFA1F50 R21 = 00000000.00000000 R22 = 00000000.00000001 R23 = 00000000.7FFF03C8 R24 = 00000000.7FFF0040 AI = 00000000.00000003 RA = FFFFFFFF.82A21080 PV = FFFFFFFF.829CF010 R28 = FFFFFFFF.8004B6DC FP = 00000000.7FFA1CA0 PC = FFFFFFFF.82A210B4 PS = 18000000.00000000 Exception Frame: R2 = 00000000.00000003 R3 = FFFFFFFF.80C63460 R4 = FFFFFFFF.80D12740 R5 = 00000000.000000C8 R6 = 00000000.00030038 R7 = 00000000.7FFA1FC0 PC = 00000000.00030078 PS = 00000000.00000003 Signal Array: 64-bit Signal Array: Arg Count = 00000005 Arg Count = 00000005 Condition = 0000000C Condition = 00000000.0000000C Argument #2 = 00010000 Argument #2 = 00000000.00010000 Argument #3 = 00000000 Argument #3 = 00000000.00000000 Argument #4 = 00030078 Argument #4 = 00000000.00030078 Argument #5 = 00000003 Argument #5 = 00000000.00000003 Mechanism Array: Arguments = 0000002C Establisher FP = 00000000.7AFFBAD0 Flags = 00000000 Exception FP = 00000000.7FFA1F00 Depth = FFFFFFFD Signal Array = 00000000.7FFA1EB8 Handler Data = 00000000.00000000 Signal64 Array = 00000000.7FFA1ED0 R0 = 00000000.00020000 R1 = 00000000.00000000 R16 = 00000000.00020004 R17 = 00000000.00010050 R18 = FFFFFFFF.FFFFFFFF R19 = 00000000.00000000 R20 = 00000000.7FFA1F50 R21 = 00000000.00000000 R22 = 00000000.00010050 R23 = 00000000.00000000 R24 = 00000000.00010051 R25 = 00000000.00000000 R26 = FFFFFFFF.8010ACA4 R27 = 00000000.00010050 R28 = 00000000.00000000 System Registers: Page Table Base Register (PTBR) 00000000.00001136 Processor Base Register (PRBR) FFFFFFFF.80D0E000 Privileged Context Block Base (PCBB) 00000000.003FE080 System Control Block Base (SCBB) 00000000.000001DC Software Interrupt Summary Register (SISR) 00000000.00000000 Address Space Number (ASN) 00000000.0000002F AST Summary / AST Enable (ASTSR_ASTEN) 00000000.0000000F Floating-Point Enable (FEN) 00000000.00000000 Interrupt Priority Level (IPL) 00000000.00000000 Machine Check Error Summary (MCES) 00000000.00000000 Virtual Page Table Base Register (VPTB) FFFFFFFC.00000000 Failing Instruction: SYS$K_VERSION_01+00078: LDL R28,(R28) Instruction Stream (last 20 instructions): SYS$K_VERSION_01+00028: LDQ R16,#X0030(R13) SYS$K_VERSION_01+0002C: LDQ R27,#X0048(R13) SYS$K_VERSION_01+00030: LDA R17,(R28) SYS$K_VERSION_01+00034: JSR R26,(R26) SYS$K_VERSION_01+00038: LDQ R26,#X0038(R13) SYS$K_VERSION_01+0003C: BIS R31,SP,SP SYS$K_VERSION_01+00040: BIS R31,R26,R0 SYS$K_VERSION_01+00044: BIS R31,FP,SP SYS$K_VERSION_01+00048: LDQ R28,#X0008(SP) SYS$K_VERSION_01+0004C: LDQ R13,#X0010(SP) SYS$K_VERSION_01+00050: LDQ FP,#X0018(SP) SYS$K_VERSION_01+00054: LDA SP,#X0020(SP) SYS$K_VERSION_01+00058: RET R31,(R28) SYS$K_VERSION_01+0005C: BIS R31,R31,R31 SYS$K_VERSION_01+00060: LDA SP,#XFFE0(SP) SYS$K_VERSION_01+00064: STQ FP,#X0018(SP) SYS$K_VERSION_01+00068: STQ R27,(SP) SYS$K_VERSION_01+0006C: BIS R31,SP,FP SYS$K_VERSION_01+00070: STQ R26,#X0010(SP) SYS$K_VERSION_01+00074: LDA R28,(R31) SYS$K_VERSION_01+00078: LDL R28,(R28) SYS$K_VERSION_01+0007C: BEQ R28,#X000007 SYS$K_VERSION_01+00080: LDQ R26,#XFFE8(R27) SYS$K_VERSION_01+00084: BIS R31,R26,R0 SYS$K_VERSION_01+00088: BIS R31,FP,SP
Extracts the error log buffers from the dump file and places them into the binary file called CLUE$ERRLOG.SYS.
CLUE ERRLOG [/OLD]
/OLDDumps the errorlog buffers into a file using the old errorlog format. The default action, if /OLD is not specified, is to dump the errorlog buffers in the common event header format.
CLUE ERRLOG extracts the error log buffers from the dump file and places them into the binary file called CLUE$ERRLOG.SYS.
These buffers contain messages not yet written to the error log file at the time of the failure. When you analyze a failure on the same system on which it occurred, you can run the Error Log utility on the actual error log file to see these error log messages. When analyzing a failure from another system, use the CLUE ERRLOG command to create a file containing the failing system's error log messages just prior to the failure. System failures are often triggered by hardware problems, so determining what, if any, hardware errors occurred prior to the failure can help you troubleshoot a failure.
You can define the logical CLUE$ERRLOG to any file specification if you want error log information written to a file other than CLUE$ERRLOG.SYS.
You need at least DECevent V2.9 to analyze the new common event header (CEH) format file. The old format file can be analyzed by ANALYZE/ERROR or any version of DECevent.
SDA> CLUE ERRLOG Sequence Date Time -------- ----------- ----------- 128 11-MAY-1994 00:39:31.30 129 11-MAY-1994 00:39:32.12 130 11-MAY-1994 00:39:44.83 131 11-MAY-1994 00:44:38.97 * Crash Entry
In addition to writing the error log buffers into CLUE$ERRLOG.SYS, the CLUE ERRLOG command displays the sequence, date, and time of each error log buffer extracted from the dump file.
Outputs the Field Replacement Unit (FRU) table to a file for display by DECevent.
The FRU command extracts the FRU table into an output file (CLUE$FRU.SYS), which can then be displayed by DECevent. This command works on the running system, as well as on dump files.
Updates history file and generates crash dump summary output.
CLUE HISTORY [/qualifier]
/OVERRIDEAllows execution of this command even if the dump file has already been analyzed (DMP$V_OLDDUMP bit set).
This command updates the history file pointed to by the logical name CLUE$HISTORY with a one-line entry and the major crash dump summary information. If CLUE$HISTORY is not defined, a file CLUE$HISTORY.DAT in your default directory will be created.
In addition, a listing file with summary information about the system failure is created in the directory pointed to by CLUE$COLLECT. The file name is of the form CLUE$node_ddmmyy_hhmm.LIS where the timestamp (hhmm) corresponds to the system failure time and not the time when the file was created.
The listing file contains summary information collected from the following SDA commands:
- CLUE CRASH
- CLUE CONFIG
- CLUE MEMORY/FILES
- CLUE MEMORY/STATISTIC
- CLUE PROCESS/RECALL
- CLUE XQP/ACTIVE
Refer to the reference section for each of these commands to see examples of the displayed information.
The logical name CLUE$FLAG controls how much information is written to the listing file.
- Bit 0---Include crash dump summary
- Bit 1---Include system configuration
- Bit 2---Include stack decoding information
- Bit 3---Include page and swap file usage
- Bit 4---Include memory management statistics
- Bit 5---Include process DCL recall buffer
- Bit 6---Include active XQP process information
- Bit 7---Include XQP cache header
If this logical name is undefined, all bits are set by default internally and all information is written to the listing file. If the value is zero, no listing file is generated. The value has to be supplied in hexadecimal form (for example, DEFINE CLUE$FLAG 81 will include the crash dump summary and the XQP cache header information).
If the logical name CLUE$SITE_PROC points to a valid and existing file, it will be executed as the final step of the CLUE HISTORY command (for example, automatic saving of the dump file during system startup). If used, this file should contain only valid SDA commands.
Refer to Chapter 2, Section 2.2.4 for more information on site-specific command files.
This command is obsolete.
The CLUE MCMK command has been withdrawn. Issuing the command produces the following output, explaining the correct way to obtain MACHINECHECK information from a crash dump.
Please use the following commands in order to extract the errorlog buffers from the dumpfile header and analyze the machine check entry: $ analyze/crash sys$system:sysdump.dmp SDA> clue errlog SDA> exit $ diagnose clue$errlog