NDMP BOF #2

1. NDMP BOF #2 Network Data Management Protocol 53nd IETF - NDMP BOF #2

2. Agenda • Agenda bashing • Summary of 1st BOF (52nd IETF) • Work group and protocol name • NDMP technical overview • - NDMP v5 specification organization • - Work group charter revisions • - Milestone/deliverable revisions • - Summary and next steps 53nd IETF - NDMP BOF #2

3. Summary of first BOF • Key feedback points: • Use of the term ‘Management’ possibly inconsistent with IETF meaning • Need for an NDMP architectural overview presentation • Some NDMP functionality outside of IETF core competency • Questionable ability to review a lengthy, broad spectrum specification • Need for a focused NDMP v5 protocol design mailing list • Possibility of significant architectural change required to satisfy IETF 53nd IETF - NDMP BOF #2

4. Summary of first BOF continued • Positive consensus: • Is NDMP evolution a useful endeavor? YES • Is NDMP appropriate as an IETF effort? YES • Should a NDMP work group be formed? YES • Are there volunteers to support a NDMP WG? YES 53nd IETF - NDMP BOF #2

5. Work group and protocol name • Issue: • Concern over the ‘M’ in Network Data Management Protocol • Perceived as inconsistent with commonly understood IETF meaning • However the acronym ‘NDMP’ enjoys certain end user recognition • Recommendation: • Preserve NDMP as the acronym • Change ‘Management’ to ‘Movement’ • Rename WG and protocol: Network Data Movement Protocol 53nd IETF - NDMP BOF #2

6. NDMP Technical Overview Harald Skardal <hskardal@netapp.com> 53nd IETF - NDMP BOF #2

7. Table of Contents • NDMP v1-v4 Problem Statement, • A typical NDMP Session: Backup • NDMP Key Concepts • NDMP v1-4 Summary • Issues with NDMP v4 • New Requirements for NDMP v5 • Proposed Work Areas for NDMP v5 53nd IETF - NDMP BOF #2

8. NDMP v1-4: Problem Statement • Initial focus: Backup and Recovery • Some file system or tape replication • Centralized control of data management • Separation of control between: • Data Management Application, (DMA) • Data Source, Data Sink • Allow for network based tape sharing • Enable interoperability => “best of breed” data management solutions. 53nd IETF - NDMP BOF #2

9. Control Messages Data Connection Payload Data Content Index Notifications, file history, log messages TCP/IP TCP/IP NDMP Data Service NDMP Tape Service TCP/IP, IPC A typical NDMP Session: Backup DMA Host DMA IP Network Primary Storage System Secondary Storage System 53nd IETF - NDMP BOF #2

10. NDMP: Key Concepts • NDMP session: • DMA and NDMP services performing an NDMP “job” • Data Management Application: • NDMP Session Coordinator and Console • Maintains “file history” index database. • NDMP Services: • The Agent managing each participating storage system • NDMP Connections: • Control connections: Between the DMA and each participating service • XDR encoded control messages, notifications • Data connection: Service to service transfer of payload data • Opaque format: Primary storage “understands” format, secondary storage does not understand format. 53nd IETF - NDMP BOF #2

11. NDMP v1-v4 Summary • Moving a byte image across the network between two parties: • Primary or secondary storage systems. • For protection or alternate storage in a primary or secondary storage system • Recovery of data from a secondary storage system • Recording internal byte image addresses for the purpose of direct file/dir access • Data Security: MD5 based Authentication • V4 Extensibility: enables proprietary and standard extensions for additional functionality. 53nd IETF - NDMP BOF #2

12. NDMP v4 Issues • No restartability. • A failed session need to be restarted from the beginning • No standardized handling of timeouts. • Weak security: • Clear text, MD5 authentication • No privacy • Non-standard interface to the application environment • Based on a mix of Unix or proprietary environment variables • Unix and file system based • Weak support for Windows based storage • No support for block and database storage 53nd IETF - NDMP BOF #2

13. NDMP v5 Scope • NDMP is fundamentally a network protocol • NDMPv5 scope is expanding beyond backup, restore. • New application areas are • Data replication, migration, mirroring • Remote backup/restore, Archiving, • These applications increases the need for a solid framework for dealing with new network issues 53nd IETF - NDMP BOF #2

14. NDMP v5 Proposed Requirements • Restartability • Restarting failed session from recent checkpoint • Standardized timeout definitions • Speed-matching of storage systems • Maximize i/o capacity: Connect N sources to M sinks • Support additional in-path data processing: • compression, encryption, virus scanning, etc. • Finding “optimal path” when multiple interfaces and paths exist • Abstraction of secondary storage resources • Tape devices with proprietary/complex features 53nd IETF - NDMP BOF #2

15. NDMP v5 Proposed Areas of Work • Improved Network Services: • Service discovery, binding • Bandwidth management, “QoS” • Improved security: • Authentication, Firewall compatibility, Privacy, Integrity • Session Wide Check Point Management • Infrastructure for restartability, DAR • Includes application level timeout support • Translator services: • Support for in-path data processing, incl. data stream multiplexing • Abstraction of Secondary Storage Systems • Hiding/abstracting device specific properties • Support for Internationalization/Localization 53nd IETF - NDMP BOF #2

16. NDMP Summary: • Practical solution which enables the creation of: • Interoperable data storage, protection and management products, • Enables customers to build “best of breed” data storage and management solutions. • Several new requirements for improving the capabilities of NDMP based solutions. • Several of these fall in the area of networking: security, interoperability, • Clean network based architectural model provides for existing and new data management needs • NDMP architecture can easily be extended to meet new requirements 53nd IETF - NDMP BOF #2

17. NDMP v5 Spec Organization Jim Ward <jimw@worksta.com> 53nd IETF - NDMP BOF #2

18. NDMP v5 Spec Organization Goals: • Delineate those portions of v5 that will leverage IETF expertise from those that may not • Foster parallelism by distributing content and editorial responsibility for discrete protocol areas to subgroups within the WG • Facilitate broad contribution and review by focusing each document on one topic, or a related set of topics 53nd IETF - NDMP BOF #2

19. NDMP v5 Spec Org continued Categories: • Architecture (one document) • Describes concepts and models on which NDMP is based • Core protocol (one or more documents) • Defines those portions of the protocol that • Implement underlying architecture • Are independent of layered services • Consume services of underlying protocols (TCP, XDR, etc.) • Layered services (multiple documents) • Defines each primary storage, secondary storage, and translate service 53nd IETF - NDMP BOF #2

20. NDMP v5 Spec Org continued Architecture Specification: • Problem domain • Key terms and concepts • Models • Computational components • Connection topologies • Finite state machines • Extension mechanism • Underlying standards • Compatibility with prior protocol versions 53nd IETF - NDMP BOF #2

21. NDMP v5 Spec Org continued Core Protocol Components: • Protocol namespace • Transport management • Core protocol state machine • Service discovery & binding • Common service operations • Diagnostics and management • Security • Service checkpointing and restartability • Extensions management 53nd IETF - NDMP BOF #2

22. NDMP v5 Spec Org continued Layered Services: • Primary storage services • Data service • File history reporting • Snapshot management • Secondary storage services • Mover service • Tape service • Disk service (CD-RW, e.g.) • SCSI service (Library, e.g.) 53nd IETF - NDMP BOF #2

23. NDMP v5 Spec Org continued More Layered Services: • Translate services • Multiplex/demultiplex • Data compression • Data encryption • Virus scanning • Other services (via extensions) 53nd IETF - NDMP BOF #2

24. Work group charter The Network Data Movement Protocol (NDMP) defines a network-based mechanism and protocol for controlling backup, recovery, and other remotely managed operations involving the movement of data between primary and secondary storage entities. The NDMP architecture separates the network attached Data Management Application (DMA), Data Servers and Tape Servers participating in data movement operations (archival, backup, recovery, migration, etc.). NDMP also provides tunneled control of secondary storage devices including tape devices and SCSI media changers. 53nd IETF - NDMP BOF #2

25. Work group charter continued The XDR and TCP/IP protocols are key foundations for NDMP. The key goals of NDMP include centralized remote management, interoperability, contemporary functionality, extensibility, internationalization, security, and insuring data integrity. The objective of the NDMP work group is to advance the state of NDMP standards by producing a set version 5 specifications including a requirements definition, an architectural definition, a core protocol definition, a service abstraction framework definition, and an extensible collection of data movement services. The core protocol will focus on control, security, reliability and efficiency of moving opaque file system and non-file system data between sources and destinations. 53nd IETF - NDMP BOF #2

26. Work group charter continued NDMP version 5 will be submitted as multiple Internet Standards Track RFCs. The first phase of work group activity is to produce new requirements and architecture documents for NDMP version 5. The requirements document will describe the limitations and deficiencies of current NDMP (v1-v4), propose high-level solutions for improvements, and rate the importance of different solutions. Following the publication of this document, work will begin on producing the NDMP v5 core protocol, service abstraction definitions and key service specifications. 53nd IETF - NDMP BOF #2

27. Work group charter continued Not In Scope: The NDMP work group will not directly address issues on the periphery of NDMP. NDMP considers a number of areas outside its focus (e.g. tape formats and multiplexing formats). As such, the NDMP work group is expected not to address those issues directly. Instead, NDMP should be designed to support all types of data formats. 53nd IETF - NDMP BOF #2

28. Deliverables NDMP v5 Requirements Document NDMP v5 Architecture Document NDMP v5 Core Protocol and Service Abstraction Document NDMP v5 Key Service Specific Documents NDMP v5 Implementation Guidelines Document 53nd IETF - NDMP BOF #2

29. Milestones Done Submit 1st draft of NDMP v5 Requirements Document as Internet Draft Done Meet at Utah 52nd IETF to discuss and record NDMP v5 requirements Done Organize ongoing work on NDMP v5 specification structure Mar02 Meet at Minneapolis to revise charter, objectives, and WG formation issues Apr02 Finalize NDMP v5 requirements May02 Release existing NDMP v4 specification as Informational RFC Jul02 First draft of v5 Architecture Specification Aug02 First draft of v5 Core Protocol Specification Sep02 First draft of v5 Key Services Specifications 53nd IETF - NDMP BOF #2

30. Milestones continued Oct02 Second draft of v5 Architecture Specification Nov02 Second draft of v5 Core Protocol Specification Dec02 Second draft of v5 Key services Specification Jan03 First draft of v5 Implementation Guidelines Jan03 Final draft of v5 Architecture Specification Apr03 Final v5 Implementation Guidelines May03 Final v5 Core Protocol Specification Jun03 Final v5 Key services Specification 53nd IETF - NDMP BOF #2

31. Summary and next steps • - Is NDMP evolution a useful endeavor? • Is NDMP appropriate as an IETF effort? • Should a NDMP work group be formed? • Do we have volunteers willing to move the effort forward? • What are the next steps? 53nd IETF - NDMP BOF #2

32. Documents and links • developer web site: www.ndmp.org • technical discussion: ndmp-tech@ndmp.org • protocol design: ndmp-v5design@ndmp.org • NDMP v4 specification: draft-skardal-ndmpv4-03.txt • NDMP v5 requirements: draft-skardal-ndmpv5-requirements-00.txt • To subscribe to either mail list, mail <majordomo@netapp.com> with the following command in the body of your email message: • subscribe ndmp-xxxx@ndmp.org <your email address> 53nd IETF - NDMP BOF #2

33. Work group staff positions • Co-Chairs: • Lawrence Barnes BakBone Software • Greg Linn Network Appliance • Document Editors: • Clive Hendrie Blue Arc • Harald Skardal Network Appliance • Work group Scribe: • tbd 53nd IETF - NDMP BOF #2