450 likes | 1.54k Views
CICS Application Dump Reading. Russ Evans evaru01@ ca.com Computer Associates SHARE102 Session 1070. Objectives. Why What is Available Controlling Dump and Trace AEI0 Abend ASRA Abend. Why Read Dumps?. Detailed snapshot of time of failure Sometimes difficult to recreate problem
E N D
CICS Application Dump Reading Russ Evans evaru01@ ca.com Computer Associates SHARE102 Session 1070 Copyright (c) 2003 Russell Evans
Objectives • Why • What is Available • Controlling Dump and Trace • AEI0 Abend • ASRA Abend Copyright (c) 2003 Russell Evans
Why Read Dumps? • Detailed snapshot of time of failure • Sometimes difficult to recreate problem • Sometimes dangerous to recreate problem • Only way to identify problem with certainty • Can be faster and easier than recreating • Dumps are free! Copyright (c) 2003 Russell Evans
What Tools Are Available? • CICS Transaction Dump • CICS Aux Trace • Language Environment Dump and Messages • CICS System Dump Copyright (c) 2003 Russell Evans
Useful Resources Before starting the debugging process, you will need: • Compile listing with the LIST option • Link map of the program load modulewith MAP and XREF • CICS Application Dump • Access to the CEEMSGS output Copyright (c) 2003 Russell Evans
Looking at the Dump CICSAPPL --- CICS TRANSACTION DUMP --- CODE=AEI0 TRAN=APCT ID=1/0037 DATE=03/09/17 TIME=11:52:29 The title of the dump contains identifying data: Review to ensure you have the correct dump: • VTAM APPLID of the region • Trans ID of the abending transaction • Abend Code • Date and Time Copyright (c) 2003 Russell Evans
Looking at the Dump (continued) REGISTERS AT LAST EXEC COMMAND REGS 0-7 092132EC 092138B8 00000000 0000009D 09213AB6 092132FC 09213300 092138F0 REGS 8-15 092134E1 00000083 0002B630 0002C62F 09205D08 09213820 8002B7CE 00000000 Do NOT rely on these registers! In this dump, they are from the last LE EXEC command issued while creating the CEEMSG dump, not from the user program….. Copyright (c) 2003 Russell Evans
Looking at the Dump (continued) PSW & REGISTERS AT TIME OF INTERRUPT PSW 079D1000 88BFD700 00060007 00000000 REGS 0-7 00000000 0921E1A0 00000000 0921EBF8 08BFD5E4 00000000 00000000 001000D0 REGS 8-15 00000000 0921EBF0 08BFD620 08BFD6B4 08BFD610 0921F690 08BFD688 88C8E8C8 Registers at time of interrupt are not produced for AEIx abends PSW Points to Next Sequential Instruction R14 may point to last Exec CICS command Other registers contain useful information Copyright (c) 2003 Russell Evans
Looking at the Dump (continued) The trans envrionment section displays some interesting information... Transaction environment for transaction_number(0000298) transaction_id(APCT) orig_transaction_id(APCT) initial_program(ABEND1 ) current_program(ABEND1 ) facility_type(TERMINAL) facility_name(CP02) Start_code(TO) netname(SC0TCP02) profile_name(DFHCICST) userid(CICSUSER) cmdsec(NO) ressec(NO) spurge(NO) dtimeout(0000000) tpurge(NO) taskdatakey(USER) taskdataloc(BELOW) twasize(00000) twaaddr( ) remote(NO) dynamic(NO) priority(001) Tclass(NO) runaway_limit(5000) indoubt_wait(YES) indoubt_wait_mins(000000) indoubt_action(BACKOUT) cics_uow_id(D7D7E3C5091A8D00) confdata(NO) system_transaction(NO) restart_count(00000) restart(NO) Copyright (c) 2003 Russell Evans
Looking at the Dump (continued) The EIB also has interesting information EXEC INTERFACE BLOCK 0115228C 0103260F C1D7C3E3 0000298C C3D7F0F2 00000009 00007D0E 06010000 00000000 00000000 00000000 00000000 000000E3 D5D7E7E7 E7E74000 00000000 00000000 00000000 00000000 0000001B 00000003 00 EIBTIME DS PL4 0115228C EIBDATE DS PL4 0103260F EIBTRNID DS CL4 APCT EIBTASKN DS PL4 0000298C EIBTRMID DS CL4 CN02 EIBRSVD1 DS H EIBCPOSN DS H EIBCALEN DS H EIBAID DS CL1 EIBFN DS CL2 0E06-LOAD EIBRCODE DS CL6 EIBDS DS CL8 EIBREQID DS CL8 EIBRSRCE DS CL8 TNPXXXX EIBSYNC DS C EIBFREE DS C EIBRECV DS C EIBSEND DS C EIBATT DS C EIBEOC DS C EIBFMH DS C Copyright (c) 2003 Russell Evans
Looking at the Dump (continued) When viewing the trace table, look first at EIP ENTRY/EXIT pairs 000298 QR AP 00E1 EIP ENTRY WRITEQ-TD =000019= 000298 QR DD 0301 DDLO ENTRY LOCATE 08A4DBC0,00059AC7,DCTE,CESE =000020= 000298 QR DD 0302 DDLO EXIT LOCATE/OK 08ABE270 , C4C3E3C5 =000021= 000298 QR AP F600 TDA ENTRY WRITE_TRANSIENT_DATA CESE,08AEB2E0 , 00000001,YES =000022= 000298 QR DD 0301 DDLO ENTRY LOCATE 08A4DBC0,00070BE4,DCTE,CESE =000023= 000298 QR DD 0302 DDLO EXIT LOCATE/OK 08ABE270 , C4C3E3C5 =000024= 000298 QR AP F601 TDA EXIT WRITE_TRANSIENT_DATA/OK =000025= 000298 QR AP 00E1 EIP EXIT WRITEQ-TD OK =000026= Remember, the most recent trace entry is the last Copyright (c) 2003 Russell Evans
Looking at the Dump (continued) The expanded trace entries have more details…. AP F600 TDA ENTRY - FUNCTION(WRITE_TRANSIENT_DATA) QUEUE(CESE) FROM_LIST(08AEB2E0 , 00000001) RSL_CHECK(YES) TASK-00298 KE_NUM-0014 TCB-QR /008CFE88 RET-88CE76F4 TIME-11:52:29.0992101250 INTERVAL-00.0000019062 =000022= 1-0000 00500000 00000035 00000000 00000000 BC200000 00000000 01AE0103 C3C5E2C5 *.&..........................CESE* 0020 08AEB2E0 00000001 00000002 08A83070 01004000 02201101 00680000 00000028 *...\.........y.... .............* 0040 00000000 01000000 B5C00000 00000000 *.........{...... * 2-0000 40C3D7F0 F2C1D7C3 E340F2F0 F0F3F0F9 F1F7F1F1 F5F2F2F9 40404040 40404040 * CP02APCT 20030917115229 * 0020 404EF0F0 F0F040F0 F9F2F0F6 F7C2F840 40F0F0F0 F0F0F0F0 F040F0F9 F2F0F6F5 * +0000 092067B8 00000000 092065* 0040 C2F040F0 F9F2F0F6 F8F3F840 F8F9F5F1 F5C5F4F2 4040F0F9 F5F1F6F0 F5F840F0 *B0 09206838 89515E42 09516058 0* 0060 F9F2F0F3 F7F5F040 F0F8C1C5 C1C6C4F4 40F0F9F2 F0F6F8F3 F840404F 4B4B4B4B *9203750 08AEAFD4 09206838 |....* 0080 4B4B4B4B *.... * Copyright (c) 2003 Russell Evans
Looking at the Dump (continued) The Program Information section of the dump contains information related to all of the programs currently active in the transaction. If the transaction has issued EXEC CICS LINKs, theneach logical link level will be displayed. This section has the most recent entries first, andis read from the last entry backward. Copyright (c) 2003 Russell Evans
PROGRAM INFORMATION FOR THE CURRENT TRANSACTION Number of Levels 00000004 INFORMATION FOR PROGRAM AT LEVEL 00000004 of 00000004 Program Name LINK04 Invoking Program LINK03 Load Point 08BFD000 Program Length 00000D90 Entry Point 88BFD020 Addressing Mode AMODE 31 Language Defined COBOL Language Deduced Unknown Comoro Address 09217C70 Comoro Length 0000000A Execution Key USER Data Location ANY Environment User application INFORMATION FOR PROGRAM AT LEVEL 00000003 of 00000004 Program Name LINK03 Invoking Program LINK02 Load Point 098692A0 Program Length 00000C70 Entry Point 898692C0 Addressing Mode AMODE 31 Language Defined COBOL Language Deduced Unknown Comoro Address 09211650 Comoro Length 0000000A Execution Key USER Data Location ANY Environment User application INFORMATION FOR PROGRAM AT LEVEL 00000002 of 00000004 Program Name LINK02 Invoking Program ABEND1 Load Point 093FDF70 Program Length 00000C70 Entry Point 893FDF90 Addressing Mode AMODE 31 Language Defined COBOL Language Deduced Unknown Comoro Address 09208650 Comoro Length 00000009 Execution Key USER Data Location ANY Environment User application INFORMATION FOR PROGRAM AT LEVEL 00000001 of 00000004 Program Name ABEND1 Invoking Program CICS Load Point 093FC260 Program Length 00001C70 Entry Point 893FC280 Addressing Mode AMODE 31 Language Defined COBOL Language Deduced Unknown Comoro Address 00000000 Comoro Length 00000000 Execution Key USER Data Location BELOW Environment User application Copyright (c) 2003 Russell Evans
Looking at the Dump (continued) PROGRAM INFORMATION FOR THE CURRENT TRANSACTION Number of Levels 00000004 INFORMATION FOR PROGRAM AT LEVEL 00000004 of 00000004 Program Name LINK04 Invoking Program LINK03 Load Point 08BFD000 Program Length 00000D90 Entry Point 88BFD020 Addressing Mode AMODE 31 Language Defined COBOL Language Deduced Unknown Comoro Address 09217C70 Comoro Length 0000000A Execution Key USER Data Location ANY Environment User application Copyright (c) 2003 Russell Evans
Looking at the Dump (continued) Load Point 08BFD000 LINK04 PROGRAM STORAGE ADDRESS 08BFD000 TO 08BFDD8F LENGTH 00000D90 000 C4C6C8E8 C3F5F3F0 58F0021C 58F0F0D0 58F0F014 58F0F00C 58FF000C 07FF0000 *DFHYC530.0...00..00..00.........* 08BFD000 020 47F0F070 23C3E2C5 C3E3F0F4 4040C3F2 40F14BF4 4BF040F0 F761F0F9 61F0F140 *.00..CSECT04 C2 1.4.0 07/09/01 * 08BFD020 040 F0F54BF5 F54BF3F8 08BFD074 E0E87C0C 08002000 10800008 00000000 00000000 *05.55.38.....Y..................* 08BFD040 Copyright (c) 2003 Russell Evans
Looking at the Dump (continued) Commarea Address 09217C70 CURRENT COPY OF THE COMMAREA 0PROGRAM COMMUNICATION AREA ADDRESS 09208650 TO 09208658 LENGTH 00000009 000000000 C1D7C3E3 40C3C1D3 F1 *APCT CAL1 * 09217C70 Copyright (c) 2003 Russell Evans
Controlling Dump and Trace //LOAD EXEC PGM=DFHDU530,REGION=0M //STEPLIB DD DISP=SHR,DSN=CICSTS13.CICS.SDFHLOAD //DFHDMPDS DD DISP=SHR,DSN=TEST.CICS.DFHDMPA //DFHPRINT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSOUT DD SYSOUT=* //DFHDUPRM DD * Copyright (c) 2003 Russell Evans
Controlling Dump and Trace (continued) //LOAD EXEC PGM=DFHTU530,REGION=0M //STEPLIB DD DISP=SHR,DSN=CICSTS13.CICS.SDFHLOAD //DFHAUXT DD DISP=SHR,DSN=TEST.CICS.DFHAUXT //DFHAXPRT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSOUT DD SYSOUT=* //DFHAXPRM DD * Copyright (c) 2003 Russell Evans
Controlling Dump and Trace (continued) I DUMP STATUS: RESULTS - OVERTYPE TO MODIFY Dum Cur(A) Ope Use CEMT I DUMP to switch the dump datasets prior to printing Copyright (c) 2003 Russell Evans
Controlling Dump and Trace (continued) I AUX STATUS: RESULTS - OVERTYPE TO MODIFY Aux Cur(B) Sto Use CEMT I AUX to start aux trace. Trace must be stopped before printing. Copyright (c) 2003 Russell Evans
Solving An AEI0 Abend • Unhandled errors result in AEIx abend • Much useful information in EIB fields and Trace Table • Messages and Codes provides clear description of error • Programming Guide describes conditions each command can receive Copyright (c) 2003 Russell Evans
Solving An AEI0 Abend(continued) From the CICS Messages and Codes manual: Explanation: PGMIDERR condition not handled. This is one of a number of abends issued by the EXEC interface program. Because of their similar characteristics these abends are described as a group. See the description of abend AEIA for further details. Note that this information can also be found using the CMAC transaction. Copyright (c) 2003 Russell Evans
Solving An AEI0 Abend(continued) Because an AEI* abend is the result of a failure in an EXEC CICS command, the first place to look is the EIB: EXEC INTERFACE BLOCK 0115228C 0103260F C1D7C3E3 0000298C C3D7F0F2 00000009 00007D0E 06010000 00000000 00000000 00000000 00000000 000000E3 D5D7E7E7 E7E74000 00000000 00000000 00000000 00000000 0000001B 00000003 00 EIBFN = 0E06 = LOAD EIBRSRCE = TNPXXXX Note: EIBFN codes are found in the ApplicationProgrammers Reference Manual Copyright (c) 2003 Russell Evans
Solving An AEI0 Abend(continued) • Another good place to look for the cause of AEI* abends is the trace table: • Shows command that failed • Provides detail on the failure • Provides the R14 value to the program, identifying the failing line of code • Unfortunately, in this dump the only entries in the trace tableare from LE writing its dump to CESE. Recommendation: set TERMTHDACT(QUIET) or (MSG) to keep LE from overwriting your trace table Copyright (c) 2003 Russell Evans
Solving An AEI0 Abend(continued) An aux trace of the pgmiderr AP 00E1 EIP EXIT LINK PGMIDERR PG 1102 PGLE EXIT - FUNCTION(LINK_EXEC)RESPONSE(EXCEPTION)REASON(PROGRAM_NOT_LOADABLE) Copyright (c) 2003 Russell Evans
Solving An AEI0 Abend(continued) Another helpful trace item AP 00E1 EIP ENTRY LINK REQ(0004) FIELD-A(09719448 ..m.) FIELD-B(09000E02 ....) TASK-00632 KE_NUM-0035 TCB-QR /008DA728 RET-8A3019FE TIME-22:12:33.9754938750 The RET value shown in the EIP ENTRY trace is the return address within the application program. We can use this to identify the specific EXEC CICS command that resulted in this failure. Copyright (c) 2003 Russell Evans
Solving An ASRA Abend • An ASRA abend is the result of an 0Cx system abend in a user program. • EIB doesn’t point to the cause of the abend • Trace table doesn’t show the failing instruction • We need to look at the PSW to find the failing instruction • We need to look at the registers to find the cause of the failure Copyright (c) 2003 Russell Evans
Solving An ASRA Abend(continued) The PSW PSW & REGISTERS AT TIME OF INTERRUPT PSW 079D1000 88BFD700 00060007 00000000 The program interrupt code tells us what kind of S0C abend this is The NSI shows the address of the next instruction. The ILC tells us how long the failing instruction is Copyright (c) 2003 Russell Evans
Solving An ASRA Abend(continued) Finding the Failing Instruction CICS is not always aware of which program is running at the time of the abend. Scan the module index in the dump to find the failing program ------ MODULE INDEX ----- LOAD PT. NAME ENTRY PT LENGTH 08BFBFE0 DFHCRSP 08BFC000 00000DC8 08BFCDB0 DFHCNV 08BFCDD8 00000130 08BFCEE0 TNPEXT2 08BFCEE0 00000078 08BFD000 LINK04 08BFD020 00000D90 08BFDD90 TNP31UP 08BFDD90 00000258 Copyright (c) 2003 Russell Evans
Solving An ASRA Abend(continued) Finding the Failing Instruction Look at the program listing to find the failing instruction 000191 CALL 00030E 4120 9950 LA 2,2384(0,9) DFHEIV0 000312 5020 D170 ST 2,368(0,13) TS2=0 000316 9680 D170 OI 368(13),X'80' TS2=0 00031A 4110 D170 LA 1,368(0,13) TS2=0 00031E 4100 D14C LA 0,332(0,13) CLLE@=3 000322 58F0 A000 L 15,0(0,10) V(DFHEI1 ) 000326 05EF BALR 14,15 000328 50F0 D078 ST 15,120(0,13) TGTFIXD+120 00032C BF28 D089 ICM 2,8,137(13) TGTFIXD+137 000330 0420 SPM 2,0 000332 9120 D054 TM 84(13),X'20' TGTFIXD+84 000336 58B0 C00C L 11,12(0,12) PBL=1 00033A 47E0 B1D8 BC 14,472(0,11) GN=1(00034C) 00033E 5820 D05C L 2,92(0,13) TGTFIXD+92 000342 58F0 21CC L 15,460(0,2) V(IGZEMSG ) 000346 4110 A0B5 LA 1,181(0,10) PGMLIT AT +169 00034A 05EF BALR 14,15 00034C GN=1 EQU * 00034C 947F D15C NI 348(13),X'7F' IPCB=1 000350 5820 D05C L 2,92(0,13) TGTFIXD+92 000354 58F0 2224 L 15,548(0,2) V(IGZETRM ) 000358 4110 A0B4 LA 1,180(0,10) PGMLIT AT +168 00035C 05EF BALR 14,15 Copyright (c) 2003 Russell Evans
Solving An ASRA Abend(continued) Finding the Failing Instruction Look at the module link map to find the failing module *** M O D U L E M A P *** --------------- CLASS B_TEXT LENGTH = D90 ATTRIBUTES = CAT, LOAD, RMODE=ANY ALIGN = DBLWORD --------------- SECTION CLASS ------- SOURCE -------- OFFSET OFFSET NAME TYPE LENGTH DDNAME SEQ MEMBER 0 DFHECI CSECT 1E SYSLIB 03 DFHECI 8 8 DFHEI1 LABEL 8 8 DLZEI01 LABEL 8 8 DLZEI02 LABEL 8 8 DLZEI03 LABEL 8 8 DLZEI04 LABEL 20 CSECT04 CSECT 56C SYSLIN 02 **NULL** 590 ABENDIT * CSECT 3AC SYSLIB 01 ABENDIT Copyright (c) 2003 Russell Evans
Solving An ASRA Abend(continued) Finding the Failing Instruction Calculate the offset of the failing instruction NSI Address…… 08BFD700 Load point……… -08BFD000 Offset into module 700 Start of ABENDIT -590 Offset into ABENDIT 170 Copyright (c) 2003 Russell Evans
Solving An ASRA Abend(continued) Finding the Failing Instruction Look at the program listing to find the failing instruction 000018 COMPUTE 000164 F284 D168 8000 PACK 360(9,13),0(5,8) 00016A FC85 D168 A030 MP 360(9,13),48(6,10) 000170 F897 D180 D169 ZAP 384(10,13),361(8,13) 000176 D207 D178 A036 MVC 376(8,13),54(10) 00017C D204 D17B D185 MVC 379(5,13),389(13) 000182 4F20 D178 CVB 2,376(0,13) 000186 F144 D17B D180 MVO 379(5,13),384(5,13) 00018C 4F50 D178 CVB 5,376(0,13) 000190 5C40 C004 M 4,4(0,12) 000194 1E52 ALR 5,2 000196 47C0 B07A BC 12,122(0,11) 00019A 5A40 C000 A 4,0(0,12) 00019E GN=8 EQU * 00019E 1222 LTR 2,2 0001A0 47B0 B084 BC 11,132(0,11) The NSI Copyright (c) 2003 Russell Evans
Solving An ASRA Abend(continued) Finding the Failing Instruction Look at the program listing to find the failing instruction 02 GOOD PIC S9(8) COMP VALUE ZERO. LINKAGE SECTION. 01 PARMS-IN. 05 BAD PIC S9(5). 000018 COMPUTE ABEND = BAD * GOOD. Copyright (c) 2003 Russell Evans
Solving An ASRA Abend(continued) Using the R14 Value R14 usually points to where the last call returned (or would have returned) to. Sometimes this can help determine howprocessing arrived at the point of failure. Find the matching instruction using the process for finding the NSI. If it is a return value it should immediately followa BASR or BALR instruction. Copyright (c) 2003 Russell Evans
Solving An ASRA Abend(continued) Finding Data In The Dump REGS 0-7 00000000 0921E1A0 00000000 0921EBF8 08BFD5E4 00000000 00000000 001000D0 REGS 8-15 00000000 0921EBF0 08BFD620 08BFD6B4 08BFD610 0921F690 08BFD688 88C8E8C8 Length fields 00016A FC85 D168 A030 MP 360(9,13),48(6,10) Register Displacement Copyright (c) 2003 Russell Evans
Solving An ASRA Abend(continued) Finding Data In The Dump Register 10 Value . . . . . 08BFD620 Offset (from instruction) . . . . 30 Actual Address . . . . . . .08BFD650 Length (from instruction) 6 00000000000C0000 Packed zeros Copyright (c) 2003 Russell Evans
Solving An ASRA Abend(continued) Finding Data In The Dump Register 13 Value . . . . . 0921F690 Offset (from instruction) . . . .168 Actual Address . . . . . . .0921F7F8 Length (from instruction) 9 00000000 0921F83818000000 Not a valid packed number Copyright (c) 2003 Russell Evans
Solving An ASRA Abend(continued) Relating the dump to the program Failing command. . . . COMPUTE ABEND = BAD * GOOD Data Definition of Good. . . . . . PIC S9(8) COMP VALUE ZERO. Data Definition of Bad . . . . . . . PIC S9(5). IGYSC2025-W "PARMS-IN" or one of its subordinates was referenced, but "PARMS-IN" was a "LINKAGE SECTION" item that did not have addressability. This reference will not be resolved successfully at execution. Copyright (c) 2003 Russell Evans
Conclusion • Dumps and traces are valuable tools • Identify the failing line of code • Help determine the actual cause of the failure • “Cheat sheet” can be downloaded from www.reevans.com Copyright (c) 2003 Russell Evans