Does Change Management Include Patches?

1. Does Change Management Include Patches? Joel Howard, RingMaster Software Northern California OAUG San Ramon 2004

2. Agenda • Change Management • Answer the Question • Realities of 11i • APM - a Change Mgmt Solution

3. What does Change Management cover? • Upgrades – Software • Upgrades – Hardware • Document Management • Configuration Management • Software Development • Maintenance

4. Change Management Why is Change Mgmt important? Disparate components share arelationship to one another… SHARED DEPENDENCIES!!

5. Changes…Dependencies • Maintenance Pack >> Apache Pre-reqs • Apache upgrade >> DB Pre-requisites • One off patch >> Pre- and Post-requisites • Apps Upgrade >> DB Upgrade Does CM include patches? YES !!

6. Release 11i Dependencies • The biggest, most complex release ever • Release 10.7 40 products • Release 11.5.5 161 products • Release 11.5.7 179 products • Release 11.5.8 191 products • Release 11.5.9 197 products ALL OR NOTHING!

7. Release 11i • Release 11.5.5 > 20,000 bug fixes • Release 11.5.7 > 40,000 bug fixes • Release 11.5.8 > 64,000 bug fixes • Release 11.5.9 > 100,000 bug fixes

8. Not Unique to Oracle What are the two most difficult issues in maintaining software? • Patching • Not Patching

9. The Impact of Patching • Problems Resolved • New, more stable technology • Added Functionality • Improved Security • Faster Performance Who is impacted? EVERYONE.

10. The Impact of Patching • Large amounts of new code introduced • Exact Impact unknown…undocumented • 2000-5000 distinct bug fixes in an average family pack • 2-50 different modules impacted • User unaware of changed processes/attributes • Disabled forms/reports • Extensive testing • Disruption of production system

11. The impact of NOT patching Why not patch? Stability… Perceived From Metalink FAQ: “At a minimum, apply maintenance packs to stay within two maintenance releases” of the most current release. Timely support: Call Oracle support for a problem and you’re not patched current? Performance Latest performance updates as part of a patch.

12. The impact of NOT patching Security Latest security updates: issued in the form of patches New Functionality Improvements in functionality and stability Politics User complaints, frustration over a lack of fixes, requests for added functionality

13. Why avoid it? • Multiple manual steps per patch, per environment • No detailed impact analysis • No structure to enforce pre-requisites • Inability to flag affected customizations • No mechanism to manage testing, approvals or workflow

14. Run Patchsets.sh Check Applied Patches Download Patch Transfer Patch Update Spread Sheets UnZip Patch Create Summary Customization Impact Review “Readme” Database Impact Issue search File System Impact Review Drivers Review Invalid DB Objects Parse Actions Find Pre-reqs Review Log File Compare Envir’s For Each Language Run Adadmin Apply to test Run Adpatch Run Adctrl Each Driver UnZip Patch Transfer Patch Run Datainstall Prepare Envir. Server Login Each Tier Manual Steps Maint Pack Download Change Reporting Maint Pack Research Change Testing Maint Pack Application

15. Strategy • AUTOMATE the maintenance of your applications • Patch Proactively: Stay current on your watch… not Oracle’s Software issues are typically encountered when you least have time for them • Month end (Mgmt reports due, Payroll filing) • Quarter end (SEC Reporting) • FDA or other certification requirements • Year end (1099s, W-2s, Sarbanes Oxley) Not patched current = major project when you can least afford it

16. APM A Change Management Solution • Identify the change • Control the change • Communicate the change

17. APM Real-time analysis by APM file system impact (APPL_TOP) database impact (table,view,sequence,index) list of affected customizations required prerequisite patches is the prereq obsolete? determine “patch impact ratio” summary of bug numbers in patch links to Oracle bug database list environments where patch/bug already applied Patch Prediction Change Management • Change requested • Research performed on Metalink • Patch obtained • Patch impact analyzed • Dependency information obtained • Knowledge disseminated • Implementation planned • Plan implemented • Testing performed • Patch Approved • Patch Migrated • Concurrent Documentation

18. APM Automatic scheduled download of patch from MetaLink Notification of obsolescence Enforcement of pre-requisites Establish pre-reqs Automate manual tasks Change Management • Change requested • Research performed on Metalink • Patch obtained • Patch impact analyzed • Dependency information obtained • Knowledge disseminated • Implementation planned • Plan implemented • Testing performed • Patch Approved • Patch Migrated • Concurrent Documentation

19. Automated processes within APM patch download transfer patch to multiple servers schedule single patch or patch group merge patches in background ADADMIN, ADCTRL, sqlplus/shell scripts compile / list invalid objects start/stop Oracle processes email/pager notification Change Management • Change requested • Research performed on Metalink • Patch obtained • Patch impact analyzed • Dependency information obtained • Knowledge disseminated • Implementation planned • Plan implemented • Testing performed • Patch Approved • Patch Migrated • Concurrent Documentation

20. Test Patch TRAIN DEV1 TEST1 PROD DEV2 Ok? Yes DBA DBA Ok? Yes DBA Ok? Yes No DBA Ok? DBA STOP Workflow / Approval Process Yes

21. Change requested Research performed on Metalink Patch obtained Patch impact analyzed Dependency information obtained Knowledge disseminated Implementation planned Plan implemented Testing performed Patch Approved Patch Migrated Concurrent Documentation APM Automatic Alerts: emails, pager, etc. Central Repository for all environments for all patching information Dozens of useful reports Environment comparisons, patch detail, patch impact analysis, patches applied by date, by environment, by application, by person, etc. find patchset levels, find where patch was applied, compare your patches against those released from Oracle Change Management

22. Functional Analyst Alerts, queries, reports, email, pagers, approvals Patching Information Information Flows Application DBA

23. Environment Comparison

24. The Impact of APM Functional Group • Reduced, focused user testing • Verify testing took place • Faster support from Oracle Technical Group • Reduced workload through automation • Reduced risk through impact analysis • Complete, accessible audit trail • Faster problem resolution Organizational Effect • Enforced procedures • Decreased workload • Communication, accountability between business groups • Improved problem resolution • Reduced costs

25. Oracle and RingMaster 11.5.9, Oracle Application Manager (OAM) • Improved Reporting • No automation, scheduling, workflow • No pre-requisite checking • No mechanism to manage testing or approvals • No detailed impact analysis • No environment comparison reports 11.5.10, Oracle Application Manager (OAM) • Improved Impact Analysis on file system only • Pre-requisite checking • No automation, scheduling, workflow • No mechanism to manage testing or approvals • No impact analysis for DB changes • No environment comparison reports

26. RingMaster Software Corporation • 1995: Founded as a consulting organization with expertise in Oracle Applications • 1998: Began developing and selling software specifically for Oracle Applications Users • Offices in Boston, Dallas, Los Angeles, Atlanta, Chicago, St. Paul, and Vermont • More than 100 customers in the US, Canada, and Europe

27. Representative Customers