SOA Testing IMS applications

1. SOA Testing IMS applications Regression testing of Z/OS, IMS DB2 and MQ at the SCC by replaying production workload

2. Process Summary • Extract 01 and 03 logrecs for a time period • Create Input file from 01 and 03 data • Sort the input file • Transfer input file to the SCC • Run the test

3. Input file • Each input record and it’s responses are grouped together consecutively on the log • 00 control record • contains information about the message, userid, lterm,etc. Also contains the elapsed time since the last message • 01 record, contains the input message. • May be segmented over multiple messages • 03 record contains a ‘response snippet’ • A bit of the response to be compared for success • An indication of the expected response time • Includes Length and offset • One per response • May be 0 or more than one per input message

4. Log extract Process Apply optional filters Extract program Sort Take this file to the flash copy test systems Extract File Production IMS LOGS

5. Extract Program – LIVSOAL • Read IMS SLDS directly • Produces smallest amount of data • Use for normal testing • Each system extracted separately, then merged with a sort • Produce from IMSPI extract • Extract and merge 01 03 recs into a DASD dataset • Run the extract program against the DASD dataset • Useful for rerunning, developing, etc • Still requires sort step afterwards

6. Distributed Environment - Simplified AIX Z/OS MQ MQ MPR OTMA MPR WAS JVM DB2 MPR IMS JVM MPR JVM JVM MPR

7. Regression testing SOA applications Z/OS Address Space Subtask MPR OTMA Subtask MPR MQ Driver Subtask DB2 MPR IMS Subtask MPR Subtask MPR Extract file

8. Driver Program LIVSOAD • Uses subtasking to drive MQ at the required message rate • Mainline controls pacing and subtask scheduling • Subtasks issue MQPUT, MQGET with wait option. • Waits on message correlid, like our real SOA systems

9. Driver Program LIVSOAD • Finds available input queues dynamically • Accepts commands via sysin or modify • Coordinates startup and shutdown of subtasks • Reads the input file • Creates the input message from segments • Build the MQIIH in front • POSTs subtask to do the MQ calls

10. Subtask Program LIVSOAS1 • Connects to MQ, creates reply queue • First time only • Sends Message to IMS using MQPUT1 • Uses the original userid authority • Uses the same commit mode • Uses the same MQ input queue for a particular TPIPE • 3270 messages round robined • Waits for response with MQGET • Wait time factored on original response time • Waits on a specific correlid response message • Compares the response received to the expected messages • Accumulates response times • Waits for more work • Disconnects from MQ on termination

11. Error Processing • Count of errors maintained • Terminate after a maximum numbers • MQ errors • Completion and reason codes written to syslog • Comparison Errors • Snaps output record and snippets. • All subtasks busy • Wait 1 second and retry • Repeat for up to 30 seconds before termination • Number of subtasks is adjustable • But beware of MQ IDBACK=20 – can reassemble/zap table, or set with online command =MVMA SET SYSTEM IDBACK(500)

12. Checkpoint Processing • Replay log data until stop is triggered • By end of log data • By MAXMSGS processed • By MAXERRS reached • By ELAPSED minutes • By STOP command • Write the next record STCK to checkpoint file • Reposition here on next restart • Aim is only to replay a piece of log data once

13. Execution Parameters

14. Log extract Parameters • If not specified, reads from start to end of supplied log • Paired date and time must be both specified, or omitted

15. Log extract Filtering Examples T=ABC,M=CSQ*,O=12,L=10 T=ABC,O=12,L=10,U

16. Testing Results • About 95% of transactions worked OK ‘out of the box’, OTMA+3270 • With filtering, now 99.8% work • Not all transactions replay OK • SOA/OTMA workload is generally OK • Some transactions ‘call out’ via MQ to off mainframe – not available at SCC • Some transactions had a date/time in first 100 bytes – change the compare offset/length for these trans • Not all 3270 works • EDIT=UC and no MFS – gets uppercased when it didn’t previously • DRD could easily change to EDIT=ULC • 3270 traffic was not the reason for writing – a bonus if it worked

17. Testing Approach • Get a clean run, excluding stuff we can’t get to work • RC=0 means function worked without errors • Tune the comparison offset for some transactions • IMSGEN to remove EDIT=UC, or online command with DRD • Exclude transactions we cannot get to work • Run the test before a change • Run the test after a change • Need to pull off and compare IMF data for performance profile comparisons

18. Testing Notes • Startup mainview on SYSA/B • Start up summary trace and MWAIT for all transactions • Start up MQ MVMA and MVM1 • Nobble JCL for CHIN • Make sure DB2s are started and connected, MSC links started, regions up • Start with a small, slow sample. • Expect ‘first time’ timeouts • Can run overnight, set STRTDATE and STRTTIME in SYSIN. Job will wait until that time to begin test

19. Testing Notes • Jobs on IV.MASTER.JOB(SOATESTn) • 1 – log extract IMSA • 2 – Log extract IMSC • 3 – SORT SOA file • 4 - DUMP/TERSE SOA FILE • 5 - FTP GET/UNTERSE SOA FILE at SCC • 6 - Execute driver program • 7 – Terse IMF logrecords to take back to real systems • 8 – Restore IMF logrecs • 9 – LOAD IMS logrecs into SAS

20. Enhancements? • Do a before/after comparison job for IMF log data • Report the offset of compare failures • Summary report – list by highest errors • Can we do anything for stored procs? • Extract program – Use DBRC API, Dynamic allocation of SLDS, SORT in one step • IMS Connect • OTMACI