Troubleshooting 11i issues

1. Troubleshooting 11i issues Adam Janbolat www.dba4aday.com adam@dba4aday.com www.dba4aday.com

2. DBA JOKE • No one really knows what the Database Administrator does, and no one is smart enough to know if the DBA is doing it or not. But every shop must have one DBA, because no place can afford two of them. www.dba4aday.com

3. Different issues with each user • A user needs a trace file to upload to support • A user is waiting on a payroll process that should not take more than 10 min but it has been running for three hours • A user heard someone talk about locking issues and now they have locking issues! • A user is getting error ORA-1652: unable to extend temp segment when running his/her code from sqlplus • Your manager says the application is running slow; can you improve performance • A developer wants you to tune her code because it is a dba thing to do www.dba4aday.com

4. Overview • Tracing application forms • Tracing database processes • Tracing concurrent requests • Locking issues • Temp space issues www.dba4aday.com

5. Diagnostic Tools • Enterprise Manager • unix/sql scripts www.dba4aday.com

6. MIS: DEMO Tracing Forms • Verify diagnostics is turned on in the Help menu. • You can turn it on for any user by setting this profile option to No: Hide Diagnostics menu entry www.dba4aday.com

7. Turn diagnostics on for user BROWN www.dba4aday.com

8. Set the profile option to No www.dba4aday.com

9. Tracing forms www.dba4aday.com

10. Tracing forms www.dba4aday.com

11. MIS: DEMO Tracing forms • The form will produce a trace file in the udump directory. • The trace file will be ora_173xxx.trc • Use tkprof on the trace file: tkprof ora_173.trc ora_173.out explain=apps/password sort=prsela • Which users have trace enabled: www.dba4aday.com

12. Who has diagnostics privs Run this query to find who has diagnostics privs: select a.user_name from fnd_user a, fnd_profile_option_values b where a.user_id=b.level_value and b.profile_option_id=3214 and b.profile_option_value='N' and b.level_id= 10004 www.dba4aday.com

13. Tracing database processes • Identify the process • Trace the process • Trace file will end up in the udump directory • Use tkprof on the trace file www.dba4aday.com

14. Tracing concurrent request • Find out the request_id • Find out if the process has child processes • Use this query to find which process it is: select oracle_process_id from fnd_concurrent_requests where request_id=941206 • Oracle_process_id corresponds to the spid in OEM. www.dba4aday.com

15. www.dba4aday.com

16. Debugging performance issues • Use top to find which session is consuming lots of cpu time. • Identify the process www.dba4aday.com

17. Process 2034132 www.dba4aday.com

18. MIS: DEMO PID translates to SPID in TOAD and OEM; find out what the process is doing www.dba4aday.com

19. Tune the code • Run an explain plan • Try passing a hint • Try creating an index • Consider partitioning • Verify there are no blocking locks www.dba4aday.com

20. MIS: DEMO Locks created by open forms • How do I know which forms are open and who is using them: Code is attached www.dba4aday.com

21. Monitor the health of the db • Check the alert log daily (automate) • Check for space issues on unix and db • Verify there are no trace files generated • Monitor for processes that consume a lot of temp space • Monitor sga utilitzation • Check application component logs on regular basis • Use statspack to monitor performance www.dba4aday.com

22. MIS: DEMO Monitor for processes that consume a lot of temp space • ORA-1652: unable to extend temp segment by xx in tablespace TEMP • Look for the number of free_extents: select * from v$sort_segment • Monitor what process is using lots of temp space. Here is a way to do that: www.dba4aday.com

23. Temp space scripts • Create a table in the tools tablespace: Sql> create table mon_temp_tab (COL_SID, VARCHAR2(10) COL_MODULE VARCHAR2(200), COL_NAME VARCHAR2(40), COL_EXTENTS NUMBER, COL_BLOCKS NUMBER, TIME_STMP DATE, GROUP_ID NUMBER) • Create a sequence: create sequence mon_temp_seq start with 1; • Create a procedure that does the monitoring: www.dba4aday.com

24. create or replace procedure mon_temp_space • as • v_free_extents number; • v_max_free_extents number; • v_seq number; • cursor c1 is • select s.sid "SID", • s.module "MODULE", • s.username "USERNAME", • u.extents "EXTENTS" , • u.blocks "BLOCKS" • from v$session s , v$sort_usage u • where s.saddr=u.session_addr • and u.contents='TEMPORARY' • and s.status='ACTIVE' • order by extents desc; • begin • select mon_temp_seq.nextval into v_seq from dual; www.dba4aday.com

25. select free_extents into v_free_extents from v$sort_segment; • if v_free_extents <=1000 • then • for i in c1 • loop • insert into mon_temp_tab (col_sid, col_module,col_name,col_extents,col_blocks,time_stmp,group_id) • values (i.sid,i.module, i.username,i.extents,i.blocks, sysdate,v_seq); • commit; • end loop; • else • null; • end if; • end; • / www.dba4aday.com

26. Create a dbms_job to run every hour declare  job_num integer; begin dbms_job.submit(job_num,'mon_temp_space;', sysdate + 1/24, 'sysdate + 1/24');  end; www.dba4aday.com

27. Monitor the mon_temp_tab www.dba4aday.com

28. Monitor the health of your applications • Verify that your are purging concurrent request logs often • Verify that your are analyzing your schemas often • Verify you are purging workflow runtime data often • Monitor apache logs for errors and warnings • Verify that debugging is disabled except for specific modules you are monitoring: www.dba4aday.com

29. Use this query to see which executables have debugging enabled • select * from fnd_concurrent_programs where enable_trace=‘Y’ • Turn off tracing www.dba4aday.com

30. Enable_trace_example www.dba4aday.com

31. Questions? www.dba4aday.com