WebDAV and DeltaV: Collaborative Authoring, Versioning, and Configuration Management for the Web

1. WebDAV and DeltaV:Collaborative Authoring, Versioning, and Configuration Management for the Web Jim Whitehead University of California, Santa Cruzejw@cs.ucsc.edu

2. WebDAV and DeltaV • WebDAV and DeltaV are: • Application layer network protocols • Extensions to the Hypertext Transfer Protocol (HTTP 1.1) • Developed by the Internet Engineering Task Force (IETF) • WebDAV • Remote collaborative authoring of Web resources • Overwrite prevention, metadata management, namespace operations • DeltaV • Versioning and Configuration Management • Extends base WebDAV capabilities • Checkout/checkin, version history, logical change tracking, workspaces, configuration management

3. Major WebDAV Clients • Application Software: • Microsoft: Office 2000/XP (Word, Excel, PowerPoint, Publisher) • Adobe: Photoshop 6, Illustrator 10, Acrobat 5, In Design 2 • Web Site Authoring • Adobe: Go Live 5 • Macromedia: Dreamweaver 4 • Remote File Access: • Apple: Mac OS X webdavfs • OS X also ships with Apache and mod_dav (have to configure mod_dav to make it work) • Microsoft: Windows Web Folders • Wind River Software: WebDrive • Goliath (Mac, open source) • WebDAV Explorer (UC Irvine, Feise/Kanomata, open source) • XML editors • Excosoft: Documentor • Altova: XML Spy • SoftQuad: XMetal

4. Major WebDAV Servers Microsoft: IIS 5/6, Exchange 2000, Sharepoint Apache: mod_dav (over 95,000 sites) Oracle: Internet File System Adobe: InScope Xythos: Web File Server Novell: Netware 5.1, Net Publisher W3C: Jigsaw Endeavors: Magi-DAV IBM: DAV4J (DeveloperWorks) FileNet: Panagon ECM Intraspect: 4i Merant: PVCS Dimensions, Content Manager Hyperwave: Information Server 5.5 4D: WebSTAR V

5. Collaborative Document Authoring • Three collaborators, in different cities, use Word 2000 to collaborate on a report they are producing together.

6. Collaborative Web Site Authoring • Two homes each develop their family Web site using WebDAV to interact with their ISP.

7. Remote Authoring, Part of Staged Production

8. Visions for WebDAV • Participants in WebDAV have many views on what it is: • A protocol for collaborative authoring of all document types • XML, HTML, word processing, spreadsheets, • A Web-based network file system, with nice high-latency behavior • Better performance than NFS and Samba over the Internet • A data integration technology for accessing a wide range of repositories • Document mgmt. systems, configuration mgmt. systems, filesystems, etc. • Remote software engineering infrastructure • A replacement protocol that can handle email, calendaring, directory lookup and more • Could replace: POP, IMAP, CAP, LDAP… • All views are correct!

9. Filesystem View • Exemplars: Web Folders, WebDrive, WebIFS, TeamDrive, Mac OS X

10. Document Authoring • Exemplars: Office 2000/XP: Word, Excel, PowerPoint, as well as Photoshop, Documentor, and XML Spy Office: uses filesystem metaphor for WebDAV location

11. Photoshop • Workflow metaphor for WebDAV location

12. Web Site Authoring • Exemplars: Go Live 5/6, Dreamweaver • Site metaphor for WebDAV location

13. Remote Collaborative Annotation • Acrobat 5 views a WebDAV location as a storage location for document annotations • Annotations are stored in resources separate from the PDF document • One collection per document • One annotation resource per user (in collection)

14. Facets of WEBDAV • There are many ways to view the DAV work: • Collaboration infrastructure • Metadata repository infrastructure • Namespace management infrastructure • Access control infrastructure • Searching infrastructure – DASL

15. Collaboration Infrastructure • Whole resource locking supports: • remote collaborative authoring of any media type • Web pages, Word processing, Presentations, XML, … • Lock characteristics: • Long-duration locks • Not associated with network connection • Client receives a lock token identifying the lock • Can disconnect from network after receiving lock token • Locks automatically expire after a client specified timeout period • Lock single resources, or hierarchies of resources • Infrastructure for asynchronous, widely distributed, hypertext-aware, collaborative editing tools.

16. Metadata Recording Infrastructure • Metadata support • Properties are(name, value) pairs that can be created, modified, deleted, and read on Web resources. • Consistency of properties can be maintained by the server or the client • Property values are well-formed Extensible Markup Language (XML) • Can store RDF as well • Property name is a URI or URL • Extensible, global property namespace • Infrastructure for recording information about Web data • A general purpose metadata repository

17. Access Control Infrastructure • Access Control: • The ability to remotely control who can read and write a resource • Key challenges: • Expose the access control capabilities of the repository… • …while ensuring the client-side user interface can be simple (I.e., avoid lots of feature discovery) • Provide some notion of principal, without duplicating LDAP • Handle large numbers of principals • Access control lists • Each entry grants/denies a privilege to a principal (or group) • Infrastructure for remotely creating collaboration groups

18. Searching Infrastructure • Searching a WebDAV repository - DASL: • Search for resources with a given property, or a given property value • Search for a substring inside a resource body • Search scope can be one resource, a collection of resources, a hierarchy of resources, or a whole server • Search syntax is extensible • Search specification specified as body submitted with SEARCH command • Could accommodate XML Query syntax • Infrastructure for remote searching

19. WebDAV Methods and Data Model

20. WebDAV Methods • Overwrite Prevention: • LOCK – prevents non-lock holders from writing to the resource • UNLOCK – removes a lock • Metadata Management: • PROPFIND – read properties from a resource • Allprop – all property names and values • Propname – only return property names • Prop – just return specified properties • PROPPATCH – write properties on a resource • Namespace Management • COPY – duplicate a resource • MOVE – move a resource (preserving identity) • MKCOL – create a new collection

21. Scope of WebDAV Methods Web Resource PROPFIND PROPPATCH† Properties (name, value) pairs LOCK UNLOCK COPY MOVE† DELETE† MKCOL† (PUT†) Body (primary state) GET PUT† † - affected by LOCK

22. DeltaV: Versioning and Configuration Management

23. Remote Collaborative Document Authoring Scenarios • A task force of people from geographically dispersed business units need to develop a report together. • Team needs to solicit feedback, so they send out copies of the report. • As a result, the team needs a permanent copy of the exact report version they are having other people review.

24. What is DeltaV? • DeltaV is: • An application layer network protocol • Extends the WebDAV protocol • WebDAV itself extends HTTP 1.1 • Thus DeltaV is also an extension to HTTP 1.1 • HTTP is the core network protocol of the Web • An interoperability standard (RFC 3253) • Allows document authoring and software development applications to interoperate with versioning and configuration management systems using a common interface • A data model • DeltaV embodies a data model that is capable of modeling multiple versioning and configuration management repositories • The data model is independent of network protocol • A data integration infrastructure • A common place for cross-application data sharing

25. Version History URL path of Versioned Controlled Resource Foo.htm 1 initial Line of Descent Beta1 2 Branch Revision History 3 Version Name 4 5 6 Label Test1 Merge 7 Beta2 Successor Predecessor

26. Representation of a Version History

27. Autoversioning • There are many WebDAV-aware applications • They know nothing about versioning! • Would like to provide automatic versioning support • Two styles of autoversioning: • Every modification creates a new version • PUT/PROPPATCH  CHECKOUT  PUT/PROPPATCH  CHECKIN • Works best for authoring clients that replicate resources to a local disk, and only write to the server at the end of an editing session. • Every LOCK/UNLOCK pair creates a new version • LOCK  LOCK  CHECKOUT • UNLOCK  CHECKIN  UNLOCK • Works well for authoring clients that work directly on the WebDAV server and take out locks, like Office 2000. • Style is controlled by DAV:autocheckout and DAV:autocheckin properties on version controlled resource

28. Workspaces • A workspace is a location… • … where a person can work in isolation • … from ongoing changes made by other collaborators • to the same set of resources. • Workspace use scenarios: • Two people each modify a common source code file, and want to test their work before integrating changes • The directory structure of a large project is being changed • Don’t want to require everyone to stop working while this takes place • Keep namespace changes invisible to team until complete • A developer is trying a change that might not pan out • New files are in their workspace, but not under version control until the final viability of the change is known

29. 4 5 3 1 2 Server Workspace Example • Version history of main.c: version controlled resources /users/chris/ projectX/main.c /users/geoff/ projectX/main.c version history resource /projectX/main.c /repo/o522/v1 /repo/o522/v2 /repo/o522/v3 /repo/o522/v4 /his/o522 /repo/o522/v1 /repo/o522/v2 /repo/o522/v3 /repo/o522/v4 /repo/o522/v5 version resources

30. 1 1 1 2 2 2 3 4 3 Server Workspace Example defs.h makefile main.c /users/geoff/projectX/ main.c, 5 makefile, 3 defs.h, 2 3 /users/chris/projectX/ main.c, 4 makefile, 3 defs.h, 3 5

31. WebDAV Projects at UC Santa Cruz

32. Recontextualizing WebDAV • WebDAV and DeltaV gathered decontextualized requirements for collaboration and version control • Abstracted away characteristics of specific application domains • Used Web authoring & software development as driving problems • Led to a powerful, inexpensive, generic content management capability • Now, goal is to apply the general-purpose capabilities of WebDAV/DeltaV to solve specific problems: • Electronic records management • Digital libraries • … as well as ensuring they satisfy original goals: • Document management • Software development

33. Advanced Document Collaboration Infrastructure • Goal: Create advanced open-source WebDAV server with following characteristics: • Highly scalable • store and search millions of documents • Data integration • knit together multiple document and metadata repositories • Access Postdoc, legacy stores via DAV/DeltaV • Access Control • support the emerging ACL protocol standard • Versioning • document versioning and auto-versioning support

34. Advanced Document Collaboration Infrastructure (2)

35. Advanced Document Collaboration Infrastructure (3) • Research challenges: • Abstract repository interface • Develop API between mod_dav_repos and repository wrappers • Scalability • Ensuring the architecture supports large numbers of documents and storage • Supports efficient searching • Cross-repository searching • Support for efficient DASL searches • Accommodating server-defined and client-defined properties • Allowing searches that span the contents of multiple repositories • Access Control Implementation • Create an open reference implementation of the access control protocol • Develop an access control client that can adapt to multiple access privilege hierarchies

36. Advanced Document Collaboration Infrastructure (4) • Work performed so far: • Created mod_dav_dasl prototype • Single database, MySQL • Storage and searching of arbitrary dead properties • Implemented DASL • Some performance characterization • Client support for DASL in cadaver client • Work is underway to release the code • Work performed by Elias Sinderson (also working at ARC), Sung Kim, Kai Pan

37. Electronic Records Management • US Government agencies are required to support standard DoD 5015.2 • Describes metadata items useful for archiving electronic records • 5015.2 is not an interoperability standard • Approach • Develop an XML representation of 5015.2 metadata items • Store 5015.2 metadata in WebDAV properties • Develop server support for • automatic archiving/disposal of records • automatic setting of 5015.2 property values • Develop client support for entry/viewing/searching 5015.2 items • Work performed by Dorrit Gordon, Leila Naslavsky

38. Electronic Records Management (2) • Work performed so far: • Preliminary XML metadata schema for 5015.2 metadata items • Modular approach allows work to potentially be used for ERM in other countries (e.g., VERS standard in Australia) • Initial work on identifying items with controlled vocabularies • Containment data modeling of 5015.2 data model • Payoff • Wide deployment of interoperable, low-cost infrastructure for archiving electronic records • Infrastructure for personal records management • In the future, how will you manage that 10TB hard drive on your PC?

39. Personal Digital Library • Researchers interact with a number of institutional digital libraries • ACM and IEEE are significant for CS researchers • Limited existing support for users interacting with these institutional DLs • Either read files online, or • Store files in a filesystem • Institutional DLs have poor track record for reliability, QoS • Users want to have local copies of relevant documents • Disk sizes are large, and growing • It’s reasonable to keep a copy of every document ever read • How do you manage all these documents?

40. Personal Digital Library (2) • Most researchers know other researchers in their area • If another researcher in your area finds an interesting paper, you’ll want a copy • This should happen automatically, just by being a member of a research peer group • Our approach: • Use a WebDAV server to store a personal, or small workgroup’s digital library • Augment documents with bibliographic metadata • Special-purpose document browsing and searching client • Provides a document UI, not a file UI • Emphasizes bibliographic metadata in browsing UI • Integrate P2P capability for document sharing • User-selectable policies for IP enforcement • Current status: NSF proposal in-progress

41. Adaptive Configuration Management Client • DeltaV specification defines multiple packages (feature sets) of functionality • Core versioning • Basic server workspace • Basic client workspace • Advanced server workspace • Advanced client workspace • A server may support one or more of these • Research challenge: • Create a DeltaV client that can adapt to any server it encounters • Implies it can adapt to different workspace semantics, and different supported features • Existing CM clients are tailored to a single system • Status: looking for a sponsor

42. Software Engineering at UC Santa Cruz

43. Software Engineering @ UCSC • Currently in the process of building up an SE research group • Hiring for three SE faculty this year • In-process of developing a Masters degree program in Software Engineering • Aimed at working professionals in Silicon Valley • Goal is to eventually offer many courses at the UC Silicon Valley Center (on the Ames campus) • Coursework in Software Engineering plus a significant multi-quarter (perhaps including summer) project • Also developing a Masters program in Web and Internet Engineering • Courses in SE, databases, networking, storage systems • Also aimed at SV working professionals

44. WebDAV Resources

45. WebDAV • WebDAV Resources • http://www.webdav.org/ • A central collection of pages and links to all things WebDAV. • WebDAV Working Group • http://www.ics.uci.edu/pub/ietf/webdav/ • Contains links to active documents, and a complete list of WebDAV-supporting applications. • RFC 2518 – WebDAV Distributed Authoring Protocol • http://www.ics.uci.edu/pub/ietf/webdav/protocol/rfc2518.pdf • This is the WebDAV Distributed Authoring Protocol specification • WebDAV: A network protocol for remote collaborative authoring on the Web • Proc. of the Sixth European Conference on Computer-Supported Cooperative Work, Sept. 12-16, 1999, Copenhagen, Denmark, pp. 291-310. • http://www.ics.uci.edu/~ejw/papers/dav-ecscw.pdf • An academic paper giving an overview of the WebDAV Distributed Authoring Protocol.

46. DeltaV • Delta-V Working Group web page • http://www.webdav.org/deltav/ • The home page for the IETF Delta-V Working Group, with links off to the most recent specifications. • RFC 3253 – DeltaV Protocol • http://www.webdav.org/deltav/protocol/rfc3253.html • Extensions to the WebDAV protocol (RFC 2518) for versioning and configuration management • The Future of Distributed Software Development on the Internet • Web Techniques, Vol. 4, No. 10, October, 1999, pages 57-63 • http://www.webtechniques.com/archives/1999/10/whitehead/ • An introduction to WebDAV and DeltaV that describes the advantages of DeltaV over CVS for remote collaborative software development

47. DASL: DAV Searching & Locating • DAV Searching and Locating page • http://www.webdav.org/dasl/ • A web site containing links to the most recent WebDAV searching protocol specifications. • J. F. Reschke, S. Reddy, J. Davis, A. Babich, “WebDAV Search” • http://greenbytes.de/tech/webdav/draft-reschke-webdav-search-latest.html • The most recently edited DASL protocol specification. • Significantly increased momentum recently • Likely to see specification approved this year.

48. Access Control Protocol • Access Control page • http://www.webdav.org/acl/ • A web site containing links to current access control protocol specifications. • WebDAV Access Control Protocol • draft-ietf-webdav-acl-07, November, 2001. • http://www.webdav.org/acl/protocol/draft-ietf-webdav-acl-07.htm • The most recent revision of the access control protocol specification.

49. Advanced Collections • WebDAV Bindings • draft-ietf-webdav-binding-protocol-02, December 17, 1999. • http://www.ics.uci.edu/pub/ietf/webdav/collection/draft-ietf-webdav-binding-protocol-02.txt • The most recent revision of the WebDAV Bindings Protocol. • WebDAV Redirect Reference Resources • draft-ietf-webdav-redirectref-protocol-02, December 17, 1999. • http://www.ics.uci.edu/pub/ietf/webdav/collection/draft-ietf-webdav-redirectref-protocol-02.txt • The most recent revision of the Redirect Resources protocol. • WebDAV Ordered Collections Protocol • draft-ietf-webdav-ordering-protocol-02, December 20, 1999. • http://www.ics.uci.edu/pub/ietf/webdav/collection/draft-ietf-webdav-ordering-protocol-02.txt • The most recent revision of the Ordered Collections protocol.

50. Distributed Authoring Locking, Properties, Copy/Move RFC 2518 complete Access Control Protocol for remote access control Finish: Spring 2002 Searching Requirements and protocol for searching a WebDAV repository Finish: late 2002 Document Roadmap WebDAV Working Group: Advanced Collections Requirements and protocol for bindings, redirectors, ordered coll. Finish:June 2003 DASL: Delta-V Working Group: Versioning & CM Protocol just sent to IESG RFC 3253 complete