460 likes | 804 Views
Prevent and Troubleshoot Systems and Performance Issues with PeopleSoft Applications. Mark Dirrim – Center of Expertise Aditya Deepak – FMS Support Manager.
E N D
Prevent and Troubleshoot Systems and Performance Issues with PeopleSoft Applications Mark Dirrim – Center of Expertise Aditya Deepak – FMS Support Manager
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Program Agenda • Introduction • Best Practices: Use a Lifecycle Advisor • Overview: Troubleshooting and Performance Advisors • Resolution Details • Q&A
Top Reasons for Creating Service Requests What we see the most: • Customer needing troubleshooting assistance • Performance planning missing from some project plans • Tools available to customers are unknown
Prevent and Resolve Performance Advisor My Oracle Support Doc ID 1437143.1
Prevent and Resolve Performance Issues • Use Cases • Evaluate • Test • Resolve PHASES
Use Cases When to use the Prevent and Resolve Performance Issues Advisor • Performance Issue • New Implementations • Changes to your user population • Upgrades • Architecture Changes • Patching and Maintenance PREVENT
Phases Prevent and Resolve Performance Issues • Evaluate • Asses the environment and set goals • Test • Plan and conduct tests • Resolve • Use best practice tools and concepts to diagnose
Troubleshooting PSFT Issues Advisor My Oracle Support Doc ID 1524185.1 – Live soon
Troubleshooting Advisor • Use Cases • Describe the Problem • Recreate the Problem • Determine the Source • Plan and Resolve PHASES
Use Cases When to use the Advisor? • System down or down regularly/intermittently • End users cannot work at all, or seeing unexpected results • Errors • You have a problem, and you don’t know where to start
Determine the Source • Identify a Diagnostic Tool
Oracle Communities User Community and Support forum
Online tracing • Identify the ‘problem’ transaction • Select approach • Initiate tracing at login • Initiate tracing before ‘problem’ operation
Tracesql file basics PSAPPSRV.1664 (1097) 1-2826 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 COM Stmt=SELECT VERSION FROM PSVERSION WHERE OBJECTTYPENAME = 'SYS' PSAPPSRV.1664 (1097) 1-2827 15.31.34 0.001000 Cur#2.1664.HCM1Q RC=0 Dur=0.001000 EXE PSAPPSRV.1664 (1097) 1-2828 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Fetch PSAPPSRV.1664 (1097) 1-2829 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Commit PSAPPSRV.1664 (1097) 1-2830 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Disconnect PSAPPSRV.1664 (1097) 1-2841 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 COM Stmt=SELECT OBJNAME, FLAG, PTCUSTOMFORMAT FROM PSUSEROBJTYPE WHERE MENUNAME = :1 AND PNLGRPNAME = :2 AND PNLNAME = :3 AND OPRID = :4 AND FIELDTYPE = :5 PSAPPSRV.1664 (1097) 1-2842 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Bind-1 type=2 length=13 value=ROLE_EMPLOYEE PSAPPSRV.1664 (1097) 1-2843 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Bind-2 type=2 length=18 value=TL_MSS_EE_SRCH_PRD PSAPPSRV.1664 (1097) 1-2844 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Bind-3 type=2 length=1 value= PSAPPSRV.1664 (1097) 1-2845 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Bind-4 type=2 length=7 value=adahlbe PSAPPSRV.1664 (1097) 1-2846 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Bind-5 type=18 length=2 value=-1 PSAPPSRV.1664 (1097) 1-2847 15.31.34 0.001000 Cur#2.1664.HCM1Q RC=0 Dur=0.001000 EXE PSAPPSRV.1664 (1097) 1-2848 15.31.34 0.001000 Cur#2.1664.HCM1Q RC=1 Dur=0.001000 Fetch PSAPPSRV.1664 (1097) 1-2849 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Commit PSAPPSRV.1664 (1097) 1-2850 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Disconnect PSAPPSRV.1664 (1097) 1-2887 15.31.34 0.000000 >>> start Nest=00 TL_MSS_EE_SRCH_PRD.GBL.PreBuild PSAPPSRV.1664 (1097) 1-2888 15.31.34 0.000000 call ext FUNCLIB_TL_WEEK.TR_WEEK_INIT_FUNC FieldFormulafcn=TimeSheetPreBuild #params=0 PSAPPSRV.1664 (1097) 1-2889 15.31.34 0.001000 >>> start-ext Nest=01 TimeSheetPreBuildFUNCLIB_TL_WEEK.TR_WEEK_INIT_FUNC.FieldFormula PSAPPSRV.1664 (1097) 1-2890 15.31.34 0.000000 call constructor TL_COMMON:Determine_DateAccess:DateAccess PSAPPSRV.1664 (1097) 1-2891 15.31.34 0.000000 call constructor TL_COMMON:Determine_DateAccess:DateAccess #params=0 PSAPPSRV.1664 (1097) 1-2892 15.31.34 0.000000 >>> start-ext Nest=02 DateAccessTL_COMMON.Determine_DateAccess.DateAccess.OnExecute
Online tracing - results SQL trace 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 COM Stmt=SELECT OBJNAME, FLAG, PTCUSTOMFORMAT FROM PSUSEROBJTYPE WHERE MENUNAME = :1 AND PNLGRPNAME = :2 AND PNLNAME = :3 AND OPRID = :4 AND FIELDTYPE = :5 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Bind-1 type=2 length=13 value=ROLE_EMPLOYEE 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Bind-2 type=2 length=18 value=TL_MSS_EE_SRCH_PRD 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Bind-3 type=2 length=1 value= 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Bind-4 type=2 length=7 value=mdirrim 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Bind-5 type=18 length=2 value=-1 15.31.34 0.001000 Cur#2.1664.HCM1Q RC=0 Dur=0.001000 EXE 15.31.34 0.001000 Cur#2.1664.HCM1Q RC=1 Dur=0.001000 Fetch 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Commit 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Disconnect • “The 5+2 trace” SQL Statements • SQL Checkbox 1 – SQL Statement • SQL Checkbox 2 – Binds – highlighted in red above • SQL Checkbox 3 – Commit / Disconnect • SQL Checkbox 4 – Fetch • SQL Checkbox 5 – Shows EXE lines which is execution time
Online tracing results • “The 5+2 trace” PeopleCode results 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 COM Stmt=SELECT CALC_EST_GROSS, TEST_LV_BAL, TEST_COMP_BAL, INTRFC_PERF_MEAS, PAYROLL_REQ_NUM, LAST_SEQ_PAY_TIME, TA_BATCH_SIZE, TA_LAST_ALIAS, TA_LAST_SECTION, TA_LAST_SQL_ID, TL_IN_PROD, LAST_ST_INSTANCE, 15.31.34 0.001000 Cur#2.1664.HCM1Q RC=0 Dur=0.001000 EXE 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Fetch 15.31.34 0.001000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Commit 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Disconnect 15.31.34 0.000000 >>> start Nest=00 TL_MSS_EE_SRCH_PRD.GBL.PreBuild 15.31.34 0.000000 call ext FUNCLIB_TL_WEEK.TR_WEEK_INIT_FUNC FieldFormulafcn=TimeSheetPreBuild #params=0 15.31.34 0.001000 >>> start-ext Nest=01 TimeSheetPreBuildFUNCLIB_TL_WEEK.TR_WEEK_INIT_FUNC.FieldFormula 15.31.34 0.000000 >>> start-ext Nest=02 DateAccessTL_COMMON.Determine_DateAccess.DateAccess.OnExecute 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 COM Stmt=SELECT ROWSECCLASS,PPA_ACCESS,PPA_ALLOW,USE_TL_SYSDATE,(CONVERT(CHAR(10),TL_SYSTEM_DT,121)),FUT_ACCESS,FUT_ALLOW,PY_ACCESS,PY_MONTH FROM PS_TL_OPR_SECURITY WHERE ROWSECCLASS=:1 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Bind-1 type=2 length=6 value=mdirrim 15.31.34 0.001000 Cur#2.1664.HCM1Q RC=0 Dur=0.001000 EXE 15.31.34 0.000000 Cur#2.1664.HCM1Q RC=0 Dur=0.000000 Fetch 15.31.34 0.001000 <<< end-ext Nest=02 DateAccessTL_COMMON.Determine_DateAccess.DateAccess.OnExecuteDur=0.001499 CPU=0.000000 Cycles=40 15.31.34 0.000000 <<< end Nest=00 TL_MSS_EE_SRCH_PRD.GBL.PreBuildDur=0.009230 CPU=0.000000 Cycles=368 15.31.34 0.000000 >>> start Nest=00 GP_ABS_EVENT.EMPLID.FieldDefault 15.31.34 0.000000 <<< end Nest=00 GP_ABS_EVENT.EMPLID.FieldDefaultDur=0.000067 CPU=0.000000 Cycles=5
Online tracing • Navigate to where you want to trace • Launch ‘New Window’ • Navigate to Trace pages in the ‘New Window’ • PeopleTools -> Utilities -> Debug -> Trace PeopleCode • PeopleTools -> Utilities -> Debug -> Trace SQL • Use same or more detailed trace options • Return to first window to perform the problem operation • Return to the ‘New Window’ and turn off trace options • Specific Operation
COOL TOOL! • Analyze .tracesql trace files • Download at: My Oracle Support Document ID: 1470578.1 (NOTE: No licensing required) • Learn more via the Recorded Webcast: Using TraceMagic to analyze *.tracesql files • Join the Community: Communities – Install/Upgrade: search on “TraceMagic” • TraceMagic Utility
Tracing Settings to use for TraceMagic
Application Server • Tips • Enable LogFence=4 • Configuration Parameter in psappsrv.cfg • Review results • Displays component and duration • Results from all users intermixed • Results from all App Servers intermixed • Consider including LogErrorReport=Y
Extended WebLogic HTTP Logging • WebLogic can be configured to capture details of each request by the Web server • WebLogic Admin Console is used to enable this feature • KM Doc: “How to Enable HTTP Extended Access Logging“ (Doc ID 662319.1) provides configuration details. • Logs will contain activities with long ‘time-taken’ value. • The URL should ‘point’ you to the problem component. • Using WebLogic to assist in problem diagnosis
Database monitoring • Track Expensive SQL back to application • AWR Report in Oracle • Licensed product • Section on SQL Statistics will contain details on ‘Top 10’ SQL • Organized by Elapsed Time, CPU Time, Gets, Reads and others • Data collected for ‘sample intervals’ and report generated later • DB2 lock monitor and related reports • More difficult to parse through • Much of the same information is available except Component
Batch Performance Diagnosis • Run Control can be modified to include TraceSQL and TracePC flags to capture diagnostics • TraceSQL 128 will provide a summary; KM DocID 1094728.1 • Other TraceSQL values will add details • 1 adds each AE Step information • 2 Adds SQL Statements – each execution • 4 Adds Temp Table allocation information • 1024 Adds timings to database • Sum of desired values determines trace flag to use • 128 for summary, 135 for detail and summary • Application Engine processes
Batch Performance Diagnosis • Modify Run Control to include Trace 128 • Cobol process will generate file named “cobsql_pid.trc” • Trace will contain the name of each stored SQL statement, execution count, fetches and runtime. • Cobol SQL statements stored in database in table PS_SQLSTMT_TBL • How to Acquire a COBOL SQL Trace (KM Doc ID 1092955.1) • Cobol
Web Server Logs • Web Server logs are generated automatically and are located in: • <web-home>/<domain>/servers/PIA/logs • The types of files there are:
PIA.pidPIA_access.logPIA_servlets0.log.0…PIA_servlets2.log.4PIA_shutdown.logPIA_shutdownerr.logPIA_stderr.logPIA_stdout.logPIA_weblogic.logdiagnostic_imagespeoplesoft.logPIA.pidPIA_access.logPIA_servlets0.log.0…PIA_servlets2.log.4PIA_shutdown.logPIA_shutdownerr.logPIA_stderr.logPIA_stdout.logPIA_weblogic.logdiagnostic_imagespeoplesoft.log Web Server Logs - files
Error Logs Generated by PeopleTools when there is a serious application error To log the process state information within the application server log itself:
Summary • Feedback – we welcome your comments • Future plans for the Advisor • Role based navigation • Task selection
4th Annual My Oracle Support Community Meet Up Event • Monday, September 23 • 6:30 – 9:30 pm • Join us for fun, networking, beverages and appetizers • Meet fellow Community members and Oracle moderators • Admission is free but registration is required • Details and registration at: • www.bit.ly/MeetUpEvent2013
Oracle University Training Services Prepare Your Organization to Achieve Optimal Performance for Your PeopleSoft Applications Top Courses for Administrators and Support Engineers “Companies that invest more in training reduce critical severity service calls by up to 49%” - Capital Analytics Study, 2008 • PeopleSoft Application Engine Rel 8.53 • PeopleSoft Security Rel 8.53 • PeopleSoft PeopleTools I Rel 8.53 • PeopleSoft PeopleTools II Rel 8.53 End-User Training • Partner with education consultants to build customized content and training plans • Implement change and achieve results in less time and with greater success Premier Support customers eligible to save 20% on learning credits.
Q&A and Important Support Resources • Discover more about Get Proactive: • http://www.oracle.com/goto/proactivesupport • MOS Community:https://communities.oracle.com/portal/server.pt/community/peoplesoft/375 • Product Information Center (PIC):https://support.oracle.com/epmos/faces/DocContentDisplay?id=1427453.2 • Upgrade/Maintenance Advisors:https://support.oracle.com/epmos/faces/DocContentDisplay?id=250.2