Disaster Recovery Using TDP with RMAN

1. Disaster Recovery Using TDP with RMAN Wylie Merritt ORACLE DBA Oklahoma Dept. of Human Services

2. Goals • Determine viability of TDP for z/Linux • Develop a ‘hot’ backup strategy • Test various recovery scenarios, including (especially) disaster recovery • Make a recommendation by YE 2003

3. Background • OKDHS had previously always done file-level backups of a cold database • Migration to a new environment meant new tools • Institutional memory: Warm backups = trouble

4. Background • RMAN had never been implemented, much less used • IBM was already under contract and Tivoli was already in widespread use on the mainframe side

5. The BIG Question…. • Can we trust this mission-critical, high-visibility database to RMAN/TDP?

6. Plan of Attack • Tivoli Storage Manager was already installed and in use (cold backups) • Tivoli Data Protector (beta) installed in late July ‘03 • RMAN hooked up to TDP – it works! • Now – How do we use RMAN?

7. Plan of Attack, Cont. • Preliminary testing promising • Backing up and restoring tablespaces, data files, etc. works fine • Not excessively complex (though not graphical) • Ability to do warm backups = a good thing BUT... What about disaster recovery?

8. Replication as Disaster Recovery POP • ‘Need to replicate’ already existed • Identical Linux guest created for replication – or so we thought! • Lesson learned #1 • Tivoli has to be ‘tricked’ into seeing the new host as the original one • Some critical parameters: • SErvername – in dsm.sys and dsm.opt • TDPO_NODE – in tdpo.opt

9. Replication as Disaster Recovery POP (Cont.) • What if your RMAN Catalog is gone? • Lesson learned #2 • Always use ‘controlfile autobackup’ • Allows use of backup file (if accessible) to determine which is the latest full backup, which log files will be required (and need restoring), etc. • As well as insuring you always have a current control file!

10. Replication as Disaster Recovery POP (Cont.) • How do you connect to a non-existent database instance? (Hint: look at your listener.ora) • Lesson learned #3 • Some ‘obscure’ listener.ora parameters come in handy during ‘disaster recovery’ • GLOBAL_DBNAME • SID_NAME • ORACLE_HOME • In the SID_LIST_<lsnr> section

11. Replication as Disaster Recovery POP (Cont.) • Lesson learned #3 (cont.) • Also need to have defined (in init.ora) • DB_NAME • DB_DOMAIN • DB_NAME + DB_DOMAIN = GLOBAL_DBNAME (from listener.ora) • Allows you to connect remotely to a non-existent (or down) database instance

12. Bottom Line – Must Haves • Backup or some other way to replicate the environment, including: • Operating system • Oracle (including RMAN) • Tivoli (including Storage Manager and Data Protector files) Don’t forget... All relevant (and relatively static) parameter files, such as tdpo.opt, dsm.opt, dsm.sys, init.ora, listener.ora & tnsnames.ora

13. Bottom Line – Must Haves (Cont.) • Connection to the new host • Local is good, but remote will work also (if you’re prepared) • A recent RMAN backup that includes • Controlfile autobackup • Log files

14. Proof of Principle Testing Implementation Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Schedule • You are here SO... We have satisfied ourselves that this is a viable approach, and that TDP will perform as advertised

15. Current Plan • High-availability database will be backed up weekly using RMAN/TDP combination. Est. planned downtime at < 30 min./wk. (currently 1 hr.) • Monthly (?) Flash Copy • All other z/Linux Oracle databases will be backed up weekly using Storage Manager (cold backup, file level) • Go-live date: April 10th

16. Contact Information • DBA – Wylie Merritt • Wylie.Merritt@okdhs.org • 405.522.1364 • Sys Admin (OS) – Chris Little • Chris.Little@okdhs.org • 405.522.1306 • Sys Admin (Tivoli) – Frank Hubble • Frank.Hubble@okdhs.org • 405.522.1314