Managing Shared Content

1. Managing Shared Content Part 1 - Susan J. Sullivan, CRM; NARA Part 2 - Tim Shinkle, Gimmal

2. Target Outcomes Knowledge of: • How NARA combined people and technology to meet the shared drive cleanup challenge, and • How cleaning up shared drives can lead to efficient management of all electronic content. • How people interact with Indexing and Categorization (I&C) Management Tools to manage content • Start thinking how you can do this in your organization

3. First meeting with the CIO as Records Officer Clean up those shared drives! ARMA is in Florida this year. *This is a dramatization

4. A bit of luck at ARMA May I have your card? • ARMA 2009 • - How to Clean Shared Drives *This is a dramatization

5. Services Contract • Award winning RM experts • Approach rooted in records management • Appliance with crawlers • Data gathering period, pre-crawls • Pilot project, draft rules – cleanup pilot • Information Management Plan • Communications Plan (posters wiki, meetings) • Met and worked with 36 SMEs, crawled and reported, validated, cleaned content

6. Overall Process Business Assessment Content Assessment Information Management Plan Human Validation Litigation Preservation Cleanup Policies Organization Migration

7. People + Tools = Cleanup

8. What is eTrash • Non-business value, non-records • Temporary files • Obsolete applications • Files with zero content • Un-openable/un-readable content • Not so important content that is not FOIA, Litigation related, retention scheduled and is: • Duplicates • Large • Voluminous • Obsolete

9. Content identified for auto-clean • System generated temporary files • System generated backup files • Zero Content files and folders • Abandoned applications • Obsolete install files • System status reporting files

10. Content reviewed for cleanup • Non accessed drafts older than 1 year • Human identified backup files • Human-identified as delete-able content • Non-business images music or audio video or media • Office Document Duplicates • Old files • Large files • Large duplicate archive • Identified old files • Compressed file duplicates • Renditions convenience copies

11. Content Identified for Risk • PII and eDiscovery • Litigation hold files • Credit card numbers • Social security numbers

12. A new way of viewing content

13. Outcomes • Shares cleaned or under review • Bulletin issued - NARA Bulletin 2012-02, December 06, 2011; Guidance on Managing Content on Shared Drives [http://www.archives.gov/records-mgmt/bulletins/2012/2012-02.html] • Inputs to Presidential Memorandum -- Managing Government Records, November 28, 2011, [http://www.whitehouse.gov/the-press-office/2011/11/28/presidential-memorandum-managing-government-records] • Policies identified • cleaning shares, • managing content on shared drives • Most importantly…..clear path forward

14. Data Management Policies • Reliability and content – Preserving file attributes • Context – Preserve file path, include recordkeeping metadata within user profiles • Usability • Align storage with usage, preservation and access needs. Widely accessible storage location for high value content • Use robust search capability eliminate complex hierarchical folder structures. • Authenticity - only authorized users can add, change and delete content for a specified data set.

15. Benefits • Storage space – when cleaned all identified, can provide upwards of 47% in reclaimed storage. • Money - To maintain 27 TB clean per year, save $187K per year (shares and email). • Time – Staff time culling through volumes of content to find or migrate their information (FOIA – E-discovery) • Access efficiencies – Right information, right place right time

16. Enterprise Value of Cleaning • If users are to migrate their own content to a target system (e.g., new share or ECRM), they decide: • Should it migrate (30 seconds) • How should it be tagged (30 seconds) • If 55% of 10TB should not be migrated: • There are 16.8M documents (at 360KB/DOC) • 140K hours of wasted decision making • $10.4M of staff time (at $75 per hour)

17. Next Opportunity: Align management with value Permanent Mid to long term retention Short term retention E-trash

18. Beyond cleanup – Auto-class • Keyword/Boolean - group • Concept Search • Ontology (file plan) based • Analyze word/phrase relationships (Bayesian) • Clustering (Bayesian) • Auto-classifier • Near duplicates • Predictive coding • Learning by example

19. People will still have a role • Connect with business owners • Develop / enforce file integrity & metadata policies • Locate and categorize content • Develop rules Boolean / Bayesian / Train tools by example • Run & review crawls and refine rules • Determine and assign value (metadata) • Search legal terms, financial terms, health terms • How often accessed, how often duplicated? • Develop / enforce storage / management policies according to value • Manage by exception

20. Vision of future I&C Tools Maintain current item-level, categorized index Tweek index crawler, learns Workers store content Crawler indexes content nightly

21. Retention Bell Curve Granular retention schedules Granular retention schedules Big bucket retention schedules I&C Tools Large categories by business function / creator role Indexed and Categorized at item level Paper-based file plans

22. Vision Evolution • Get clean and lean • Mature cleanup program • Cleanup and network policies enforced • Comfortable with crawlers and rules • Develop rules to index at various value levels • Transitory, 3 year, 7 year, 10 year, etc. • Permanent • Group, cluster, train, metadata extraction, migrate, and tag with retention • Go granular as needed • Crawl nightly, manage by exception

23. Automated Policies • Run continuously • Cleanup policies for files that are no longer needed • Security risk policies for files meeting PII criteria • Legal policies for files meeting legal hold criteria • IT drive management policies for databases, multimedia, images, applications, web content and other categories • Lifecycle management policies for • inactive files • draft files • active files • records • reference material • Content integrity policies for abandoned files • Migration policies for files that require migration to target systems

24. I&C Tools • Indexes & categorizes content according to rules • Clusters content around trends (retention) • Ingests content samples and learns. • More crawling = more learning • Manage by exception = more learning • Act upon content (move, lock)

25. Part 2 Tim Shinkle

26. Professional Services Approach • Phase 0 – Source repository assessment • Phase I – Program setup • Phase II – Pilot • Phase III – Enterprise Transformation

27. Phase 0 – Source Repository Assessment • Perform preliminary analysis, e.g., Shares, SMEs, business processes • Scan a percentage of enterprise shares that represent a good cross section of the data • Produce an assessment report and plan moving forward • Percentage of candidate eTrash, migrate, relocate, review leave • PII analysis • Metadata extraction • Classification and categorization • Strategy and approach • Expected cost savings and benefit analysis

28. Phase I – Program setup • Meet with sponsors • Identify scope (shares, groups, SMEs) • Develop communication plan • Stand up program communications portal (e.g., corporate portal, wiki) • Identify pilot group(s) (shares and SMEs) • Meet with RM and Legal to identify candidate records, holds and PII • Approve policies, cleanup and classification rules

29. Phase II – Pilot • Interview pilot group • Perform analysis and refine rules • Crawl and report on pilot group shares • Review results with SMEs and move approved files to cleanup location • Refine global cleanup and classification rules and update published policies • Develop benefit analysis report and enterprise strategy • Publish updates to program portal (wiki)

30. Phase III – Enterprise Transformation • Outline schedule, project plan and group priority • Update communications on portal (e.g., wiki) • Repeat for each group • Meet, refine share mapping to SMEs and perform high level analysis, size and scope • Crawl shares, generate reports and publish for review • Meet with SMEs, review results and start the clock • Move approved files to temporary storage location • Perform any additional migration and transformation based on classification, categorization and metadata extraction of results • Repeat until completed • Publish final reports, develop and implement strategy for on going policy information governance automation

31. Technology Demonstration Example Data

32. Classification Best Practices • Create categories that can be leveraged by machines (e.g., Put like content in like containers, e.g., expense reports) • People can provide context that is not found in the content (e.g., the folder name is a project number where anything to do with the project is dumped into the folder, such as proposal, meeting notes, deliverables, reference material, directions, local hotels, expenses, etc...) • Folders can only sometimes tell you what can be found in the folder, other times it cannot (e.g., \Invoices vs. \Misc)

33. Classification Best Practices • File names are mixed and matched (e.g., \Proposals\Proposal XYZ.doc, \Meeting notes\Proposal XYZ.doc) • Don’t confuse how individuals want to organize their content with the classification of the content • The more the information is shared, the more it needs to be generally classified (big buckets) • Combine algorithm extracted topics and identified classifications with subject matter expert search rules

34. Q&A • Susan Sullivan, CRMDirector - Corporate Records Management National Archives and Records Administrationsusan.sullivan@nara.govV: (301) 837-2088 • Tim ShinkleDirector, Gimmal Grouptim.shinkle@gimmal.com(703) 927-5650