1 / 43

Building the Right Infrastructure for Microsoft SharePoint

Building the Right Infrastructure for Microsoft SharePoint . EMC and Microsoft Alliance. EMC and Microsoft have been engaged as strategic partners for over 16 years Shared view of importance of information Joint sharing of intellectual property

oona
Download Presentation

Building the Right Infrastructure for Microsoft SharePoint

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Building the Right Infrastructure for Microsoft SharePoint

  2. EMC and Microsoft Alliance • EMC and Microsoft have been engaged as strategic partners for over 16 years • Shared view of importance of information • Joint sharing of intellectual property • EMC participates in Microsoft Technology Adopter Programs (TAPs) and Rapid Deployment Programs (RDP) • CEO-sponsored worldwide go-to-market alliance • Over 300 published technical solutions • Over 100 data sheets on Microsoft technologies • EMC has Hundreds of Microsoft Certified Professionals (MCPs) • Microsoft is one of EMC’s Top 5 customers 16+ Microsoft Gold/Silver Competencies 21x Time Partner of Year 2,700+ consultants worldwide Cooperative Support Agreement Equipment in Redmond labs Leading Microsoft consultancies

  3. VNXe3100 VNXe3300 VNX5100 VNX5300 VNX5500 VNX5700 VNX7500 Affordable. Simple. Efficient.Powerful.

  4. Optimized for Microsoft Complete Integration Across the Microsoft Stack On-Premise / Cloud • Manage Storage from SharePoint UI • Shrink SQL ContentDB by 90-95% via RBS • 30-40% reduced search RT using FAST Cache Business Productivity • FAST Cache for 9x increase in TPS • VFCache for 87% reduction in RT • Combined for 11x increase in TPS Application Platform • ESI/SCOM/SCVMM Packs • SMB3.0 Support • Thin Provisioning • ODX Copy Offload Core Infrastructure

  5. Challenge: Relentless Growth of Unstructured Data People / Business Processes: • Disconnected information silos • Information not available to those who need it in real-time • Governance IT Related: • TCO / Asset Utilization • Backup Windows / Recovery Processes • Version Control, user mapping, permissions Source : IDC, The Digital Universe Decade

  6. Why SharePoint? Why Now? Gartner Magic Quadrant for Enterprise Content Management • Why SharePoint? • Fastest growing platform in MSFT’s history • $2B in 2011, 20,000 seats/DAY! • Accessible, extensible, unified content platform • Why now? • 67% of customers report rolling out SharePoint to the entire organization • New release (Wave 15) coming in November

  7. Infrastructure Challenges with Current Sharepoint Deployments • Unstructured Data Takeover • Primary document types stored in SharePoint are PDF, Word, PowerPoint, and large Excel spreadsheets • Documents typically well over 1MB • Files take over valuable server disk space and resources • Avalanche of large media • SharePoint often stores large files like media images and videos • SQL not optimized for file level access in general and streaming in specific • Applications will not return control until file is fully loaded • File size and concurrency will increase the chance of browser timeouts

  8. Infrastructure Challenges with Current Sharepoint Deployments • Old and unused files hogging valuable SQL storage / CPU • No means to intelligently tier off / archive older stale content • 60-80% of SQL BLOB corpus rarely used or not used at all • Equates to over 50% of the SQL Content DB Corpus if not externalized! • Lack of scalability • Proliferation of server resources to address performance bottlenecks • Poor economic scaling of SQL server licenses • Native backup / recovery tools too slow for sizable documents • Only 600GB backed up in six hours using native tools • Only several TB’s backed up in six hours using MSFT DPM • Lack of consistency in recovery of farm, ContentDB, and BLOB Store • Lack of tools to load-balance server resources

  9. Building a SharePoint Architecture for Scale Today Centralize & Take Control Consolidate & Optimize 5% File Shares Metadata Legacy ECM BLOBs 95% Future content growth

  10. Metalogix StoragePoint Librarian Contoso Finance AP Invoices Purchase Orders StoragePoint Creates SharePoint Structure and Items StoragePoint Catalogs File share AR HR Employees Training Materials File Share Librarian Handbooks and Policies List Items and Metadata Content stored on file system. Content stays on file share SQL Content DB Training Materials HR Contoso Purchase Orders Invoices Handbooks and Policies Employees Finance AR AP

  11. The Challenge with SharePoint BLOBs • Performance for large files, especially write intensive • Large file upload time can be improved by as much as 50% with RBS • Can’t easily & economically scale • Asset Utilization SQL Servers • BLOBs don’t benefit from SQL query engine. Why are they there? • Long backups / long restores • Lack of tiered storage = Poor TCO 1TB of SQL Content Database Storage 90-95% unstructured, rest metadata “Typically, as much as 80 percent of data for an enterprise-scale deployment of SharePoint Foundation consists of file-based data streams that are stored as BLOB data. These BLOB objects comprise data associated with SharePoint files. However, maintaining large quantities of BLOB data in a SQL Server database is a suboptimal use of SQL Server resources. You can achieve equal benefit at lower cost with equivalent efficiency by using an external data store to contain BLOB data.” Source: http://msdn.microsoft.com/en-us/library/bb802976.aspx DOC PDF TIFF PPT XLS 10 x 100GB SharePoint Content Databases

  12. Metalogix StoragePoint RBS Architecture StoragePoint Windows SharePoint Services Endpoint Manager Shared Services Adapter Isilon EBS .NET Profile Management Compression Encryption Logging Notifications Filters (Size and Type) Monitoring Migration Caching Adapter Framework SPRequest Adapter SOAP VNX RBS WebDAV Adapter Atmos SharePoint Object Model REST Metadata Content Database

  13. Metalogix StoragePoint Data Flow BLOB write handed off to RBS Provider Document uploaded SP Web Application Service RBS Provider Is there a storage profile? NO Metadata uploaded Document YES Invoice # Determine Endpoint Date BLOB written to SQL Server Amount Metadata saved to SQL Server Vendor Metadata Synchronous Asynchronous BLOB Migrator Agent Object Store / Cloud Share SQL Server

  14. StoragePoint – Using Aging Rules Metadata BLOB Invoice Date > 3 years Invoice Date > 1 year VNX/NL-SAS VNX/NL-SAS Compressed Isilon • On-Premise • Private Cloud • Public Cloud Archive Active Inactive Older Content to Lower $/GB Tiers

  15. StoragePoint - Advanced Filters Share1 FileSystem (Sync) Filter: File Type = PDF, TIF, TIFF Share2 FileSystem (Sync) Storage Profile Filter: List = HR DOCS BLOB Store 3 FileSystem (Sync) Filter: Site (Web) = Finance

  16. EMC / Metalogix Reference Architecture • Flash 1st 75% Cost Reduction via VNX FAST and smart use of SSDs and NL-SAS • FAST Cache  Up to30-40% reduction in response times for Search • FAST VP  80% less performance tuning for SQL • Lower SQL Licensing Costs due to server virtualization and RBS Metadata 5-10% FC Network VNX FAST Cache enables faster indexing/crawl/search SQL Search DB tempDB Search Query VNX FAST VP to reduce performance tuning Share SAS BLOBs 90-95% StoragePoint Rule moves BLOB from Tier 1 to Tier 2 At Invoice Date + 6 months Share NL SAS CIFS Network StoragePoint Rule moves BLOB from Tier 2 to Tier 3 At Invoice Date + 1 year Share Deep Compressed Reference Architectures: http://www.emc.com/collateral/hardware/technical-documentation/h8185-sharepoint-vnx-metalogix-psg.pdf http://powerlink.emc.com/km/live1/en_US/Offering_Technical/Presentation/remote-blob-storage-sharepoint-tp.pptx

  17. Applying StoragePoint Advanced Filters VNX File Level Retention: FC Network Metadata LUN FAST Cache LUN FAST VP Encrypt sensitive data before it is moved to storage endpoint (up to 256-bit) Share NL SAS Use Advanced Filters to move HR data to where retention / compliance is ensured Share FLR Active Data CIFS Network • Set retention periods at file system creation • Default/minimum/maximum • Automatic File Lock and Delete Archive / Big Data

  18. Performance • Maintenance • Load Balancing • Business Continuity • Consolidation • Availability Virtualizing SharePoint is Essential • Achieve 2-10x consolidation ratio, especially for larger deployments • Lower SQL Licensing by consolidating vCPU onto physical CPU Simplified DR management (Geo-Clustering, vCenter Site Recovery Manager) • Live migration of virtual machines (Hyper-V Live Migration, VMware vMotion) Improved front end performance with more, smaller WFEs rather than few large WFEs • VM based protection for SharePoint provides homogeneous high availability (WSFC, VMware HA) Maximized overall performance with balanced cross farm HW utilization (SCVMM PRO , VMware DRS)

  19. Three Sources of Savings • Better Asset Utilization • File Share typically not well utilized • VNX Utilization ~70% on SAS / NL-SAS and up to 90% on SSD • More Efficiency via Flash 1st • Results in fewer drives, lower energy bills, and lower $/GB • Also lower $/IOPS via FAST Suite • Lower SQL licensing via RBS and virtualization • Reduce database size and increase VM density per physical server

  20. FLASH 1st Data Strategy Hot data on FAST Flash SSDs—cold data on dense disks “Hot”high activity As data ages, activity falls, triggering automatic movement to high-capacity disk drives for lowest cost Highly active data is stored on Flash SSDs for fastest response time Data Activity High- capacity HDD FlashSSD “Cold” Low Activity Data Age Movement Trigger

  21. FLASH 1st Example 32 TB Usable. 50% Y/Y Growth. 60 Days Data Shelf-life. 210 15K DRIVES VNX5500

  22. FLASH 1st Example 32 TB Usable. 50% Y/Y Growth. 60 Days Data Shelf-life. 210 60 VNX5300 15K DRIVES DRIVES FAST Cache & FAST VP 20200GB SSDs 16600GB 15K SAS 243TB NL-SAS VNX5500

  23. FLASH 1st Example 32 TB Usable. 50% Y/Y Growth. 60 Days Data Shelf-life. Power IOPS Footprint 60 VNX5300 74K 5% FLASH 45RU 3,938W DRIVES 33K FAST Cache & FAST VP 18RU 990W 20200GB SSDs 16600GB 15K SAS 243TB NL-SAS

  24. VNX FAST Suite A FLASH 1st strategy enabler Real-time caching with FAST Cache • FAST Cache continuously ensures that the hottest data is served from high-performance Flash SSDs • FAST VP* optimizes storage pools automatically, ensuring that active data is being served from SSDs, while cold data is moved to lower-cost disk tiers • Together they deliver a fully automated FLASH 1st storage strategy for optimal performance at the lowest cost attainable • Monitor and tune the whole system with the complementary Unisphere Quality of Service Manager and Unisphere Analyzer 15KHDD FlashSSD Flash Scheduled optimization with FAST VP * Not available for VNX5100 High-capacity HDD

  25. VNX FAST Cache boosts SQL2012 IOPS • FAST Cache: 14 x 100GB SSDs • Working Set < FAST Cache • Results: • 5x Improvement of IOPS and TPS • 30 Minute Cache Warming TPS

  26. Lower $/GB and $ / IOPS with RBS With RBS No RBS Metalogix StoragePoint LOWEST$/IO Metadata 5-10% SSD$.99/IO BLOBs 90-95% Average$/GB LOWEST$/GB SAS–15K $2/GB NL-SAS $.43/GB • Example: • Avg. Capacity VNX5300 = 50TB • Without RBS on SAS-15K: $100K • With RBS and 5% SSD and 95% NL-SAS: $40-50K

  27. Lower SQL Licensing Costs • SQL Server is rarely CPU constrained • For SharePoint the performance bottleneck tends to be the WFE • Idle CPU = Wasted licensing $$ • SQL2012 Server Enterprise Licensing: • Physical: per physical core • Virtual: per vCPU (all you can eat per physical core) • No cost difference between a physical core and vCPU • Virtualization = greater VM density and lower SQL licensing costs • Savings determined by vCPU : Core Ratio (e.g. 4:1 ratio =75% savings) • RBS = greater VM density and lower SQL licensing costs • Lower CPU load on SQL servers • Have to weigh against performance / CPU utilization • SQL ContentDB recommended CPU limit 50% • Perfmon helps determine the status quo

  28. RBS + VNX = More efficient SQL IO • Overall more efficient handling of IO • Reduction in overall IO by 20-30% • IO coalescing on VNX • Most IO handled by VNX CIFS share • 70-80% of backend IO handled by CIFS share • RBS reduces SQL CPU cycles by only storing the varbinary file attribute in SQL • Actual BLOB is stored in more efficient file system Source: http://powerlink.emc.com/km/live1/en_US/Offering_Technical/Presentation/remote-blob-storage-sharepoint-tp.pptx

  29. Large Government Contractor EMC VNX and Metalogix StoragePoint Improves Operational Agility and Performance • Challenge • Internally facing communications site, which included large amounts of rapidly changing large objects • Page load times and backup / recovery times Derek Martin Cloud Solutions Architect “The dashboard reports showed that I was now saving >90% of my storage on the expensive SQL drives.” • Results • Average page load time improved by 4x after RBS • Backup times from SQL and SharePoint reduced by a factor of 13x • >90% moved from expensive SQL drives to NL-SAS

  30. SharePoint Performance Considerations • Performance for SharePoint highly use case dependent • Focus on Enterprise Content Management* • SharePoint workload for content management • Can be characterized by 60/20/10/10 for Browse/Modify/Search/Upload • File size, distribution, and concurrent access impact performance • Symptoms vs. Remedy • Symptoms often occur during individual tasks (e.g. file upload by user) • Sizing however needs to factor in system level workloads for the entire farm • In addition need to size system for blended workload (e.g. Exchange, SQL, SharePoint)

  31. SharePoint Performance Considerations • The storage subsystem is rarely the bottleneck • Generally web front end (WFE) CPU represents the bottleneck • VNX FAST Suite enables very efficient configurations (majority NL-SAS, 2-5% SSDs) • Search functions are the workloads that happen to benefit the most • Search critical as SharePoint environments and document repositories grow • Microsoft Test Report for Very Large-Scale Document Repositories validates VNX performance • VNX typically deployed with mixed workloads (e.g. Exchange, SQL, and SharePoint) • Case for RBS: SQL Database not efficient at handling files IO • Each write represents an update of the table and the log • Non-externalized storage formats use the SQL buffer pool when the data pages are accessed • Large files and high concurrency can lead to buffer overruns compounding the issue • Other processes such as index rebuilds can also compete for the same server resources

  32. Best Use of VNX FAST Suite for SharePoint

  33. Recovery Options / Tradeoffs • What are the recovery objectives? • Recovery Level: Database, Database + BLOBs, Database + BLOBs + Farm • Recovery Time: how long does it take to recover • Recovery Point: How much can I afford to lose vs. how much am I willing to spend • Distance: Local vs. Remote • Granularity: Item level vs. entire corpus • Management Overhead: how much time do I need to spend on backup / recovery • Challenges with native tools: • Limited in scale, scope, and performance (e.g. STSADM, Central Administration, Recycle Bin) • Configuration data not backed up • Low backup / recovery rate: rated at 600GB in 6 hours (TB’s with DPM) • Manual re-creation of configuration • STSADM / Central Administration not recommended above 85GB for site collections or Content DB larger than 100GB

  34. Availability & Protection Methods / Tradeoffs

  35. Replication Manager / Item-Level Restore • RM supports recovery of Content DB and Remote BLOB Store via script • Order of restore: • Single ContentDB BLOB Store • StoragePoint DB • Singe ContentDB • For item-level restore: • Point Selective Restore Manager or Ontrack PowerControls to mounted RM database files • Browse to content and restore the required file(s) or Source: http://powerlink.emc.com/km/live1/en_US/Offering_Technical/Presentation/remote-blob-storage-sharepoint-tp.pptx or

  36. Replication Manager Protection Full farm protection 250 GB Single Content Database Restore

  37. SharePoint 2010 Disaster Recovery:Virtualized Farm (Full Replication) SharePoint Farm A Primary Site Secondary Site • Resource/Protection Group level granularity • Active/Active (Sync distances) or Active/Passive (Async distances) • Failover automation: • VMware Site Recovery Manager (SRM) Protection Groups for all server roles • EMC Cluster Enabler with Hyper-V failover clustering VMs VMs WFE Application Servers VMs VMs VMs VMs Crawl RecoverPoint SQL Cluster VMs VMs Databases Databases BLOB Store BLOB Store

  38. Designing DR consistency for SharePoint Automated DR Externalized Content consistency Search consistency

  39. Open / browse local or remote SharePoint farm Provision SharePoint Web Application: Create disks Create content database Create Web Application Attach the content database Create content database only (no WFE) Web Application can be created later using SharePoint admin and the database created in ESI Provisioning SharePoint - ESI leave unchecked Users can select any SQL server that is part of the farm and any instance in that server.

  40. Health Monitoring in the Private Cloud • EMC System Center Operations Manager (SCOM) Management Packs: • Array discovery & health monitoring • Full hierarchical storage schema and relationship view • Dashboards incl. physical view (SPs, DAEs, NICs, HBAs, Drives) and logical view (RAID Groups, Volumes/Pools, LUNs, Protocols…) • Performance Monitoring • End to end application service view • Map / visualize relationships between storage and SharePoint farm • EMC System Center Orchestrator (SCO) Management Packs: • Focal point for automation of heterogeneous datacenter management • Service Manager for workflows and pre-defined run books • 21 storage activities and counting • Example Drive failure: • Gather SP Collects • Post on pre-defined share • Send alerts to admin

  41. Leveraging VSPEX Partner Bundle ISV Application Hypervisor x86 Server Network Storage Proven Infrastructure *not committed

  42. RBS Options & Feature Comparison

More Related