Large-scale SharePoint Architecture

1. Large-scale SharePoint Architecture Eli Robillard eli@erobillard.com Twitter: @erobillard November 2011

2. Thanks to our sponsors!

3. Agenda for this hour • The process, language and patterns of architectural design. • Principals of sound architecture • Perspectives of SharePoint architecture • Large-scale considerations

4. The Design Process “The usual problem with process improvement is not having one to begin with.”

5. The Design Process • Define the elements. • Organise the elements into entities. • Among the entities, identify the relationships. • Describe the processes that give the system life.

6. Shared Language

7. Design Patterns • Christopher Alexander • A Pattern Language, 1979 • Gang of Four • Design Patterns: Elements of Reusable Object-Oriented Software, 1994

8. Change “The only constant is change.” –Heraclitus Good architects plan for day 1. Great architects plan for every day after that.

9. Principals of Sound Architecture Have a process and get crisp on scope. Express solutions in a shared language. Identify and reuse design patterns. Plan for change. Introducing the worst acronym ever: SCLAPACH Scope Language Patterns Change

10. Out-of-scope Business value “Feature <x> will be delivered in release <y>.”

11. Architectural Stack Governance : Logical : Physical : Information : Solution : Team Platform, farms, services and integration Servers, networks and communications Taxonomies and navigation Applications

12. Architectural Stack

13. Governance Governance is the architecture of the team. For every what, have a who and be sure who knows why. Scope  Language  Patterns  Change

14. Governance Teams and Tasks Write and compile code Assemble and document the package Install and configure the release UAT design, QA and load testing Monitor and maintain the platform Provide training and manuals Explain benefits, drive adoption Development : Integration : Deployment : Test : Operations : Training: Marketing:

15. Governance Demo Scope  Language  Patterns  Change

16. Governance • Recommendations: • Plan teams like server farms; think of roles as capabilities or services; decompose them until they can scale. • People can wear multiple hats when it presents no conflict of interest.

17. Logical Architecture Scope: Farms, zones, authentication, application pools, web applications, site collections, service applications, external services. Scope  Language  Patterns  Change

18. Logical Architecture Demo Scope  Language  Patterns  Change

19. Logical Architecture New recommended limits: • 200 GB per Content DB • 4 TB supported limit for all use cases • >4TB supported by Record Center and Document Center Scope  Language  Patterns  Change

20. Physical Architecture Scope: Servers (farm composition), load-balancing, firewalls and port configuration, authentication providers, availability and failover. Scope  Language  Patterns  Change

21. Physical Architecture Demo Scope  Language  Patterns  Change

22. Physical Architecture Recommendations: • Plan for full automation • Cookie-cutter provisioning everywhere Scope  Language  Patterns  Change

23. Information Architecture Scope  Language  Patterns  Change

24. Information Architecture Demo Scope  Language  Patterns  Change

25. Information Architecture Recommendations: • Create empty top-level content types that inherit from Item and Document • Build for discovery • Keep taxonomies simple

26. Solution Architecture Common problems: • How are solutions versioned? • How do I deploy globally? • How is content upgraded? • What is a SharePoint application? • Solution package payload • Features or sets of features • Site Template

27. Solution Architecture • Understand the solutions being hosted • Make knowledge transfer part of change management • Know: • What patch levels are expected? • Where is the payload going? • GAC, BIN, 14 hive, SP Sandbox, Galleries, elsewhere? • Does it use elevated privileges? • How do I monitor and trouble-shoot?

28. Solution Architecture • Profile your content and applications • Co-locate compatible solutions • Profile according to: • SLA: Service windows and usage patterns • Users: zones and authentication • Corpus: Size, shape and growth of content • Physical isolation requirements • Feature availability vs. isolation

31. Data Access Model Client API: May be Javascript or JQuery clients for web services. Service Layer: Web services, a service bus, or other protocols used to communicate across application domains. Business Logic: Organizes raw data and relationships into concrete entities, with methods to allow interop among entities. Data Access: Stored procedures or primitive classes to implement business rules, validation and security. Data store: Relational, hierarchical or flat-file data.

32. Solution Architecture • Insist on an n-tier architecture • Data: .NET Provider model • Logic: .NET Define common business entities • Presentation: Consider as an SDK • Take a layered approach to building and provisioning application components

33. Where should these be documented? • Physical vs. virtual servers • My Sites • Service accounts • Disaster recovery environment

34. Rules of thumb “If it’s worth doing, it’s worth overdoing.” -ER “The problem with doing something right the first time is that no one will appreciate how difficult it was.” –Walt West “Whatever you are, be a good one.” – Abe Lincoln

35. Thank-you! Please complete your evaluations to enter in the prize drawing.

36. References • Software Release Management Best Practices, Kevin A. Lee, [Link].

37. Large-scale SharePoint Infrastructure Eli Robillard eli@erobillard.com

38. Large-scale considerations • Patch management • Database scaling • Caching • Latency • Multi-lingual

39. Patch management issues • When content farms are patched before service farms • Applications rely on patches • Manual vs. scripted processes

40. Patch management • Take a top-down approach to upgrades • Decide between shared and central ownership of the process • Track all regional upgrades and associated tickets centrally • Consider your DB upgrades (PSConfig)

41. Database scaling issues • Effects of long running operations • Indexing, backup, profile imports, etc. • Inability to scale onto new hardware • The single instance effect

42. Database scaling • Separate spindles for: • OS, server logs (RAID 10 or 1), tempdb (RAID 10), SharePoint databases (RAID 10 or 5) • Separate tempdb data files per CPU • Separate instances for: • Content DBs, SP Analytics and Config DBs • Change the default autogrowth setting

43. Caching • ASP.NET Page output cache • SharePoint object cache • BLOB cache • Content distribution networks (CDNs)

44. When you don’t use the object cache. . .

45. When you don’t use the object cache. . . • What went wrong? • The hero control (which was implemented as a Sandboxed Solution) was using the non-cached SharePoint API, instead of the cached API. • The site was launched, and as soon as it got real load, the Sandboxed Solution throttling system (correctly) stopped executing the control.

46. Dealing with Latency • IIS Compression • WAN Acceleration • Remote terminals

47. Multi-lingual considerations • Configure variations sooner than later and consider on-demand page propagation. • If you baseline at SP1 plus either the August or October CU, language packs no longer need to be installed separately.

48. References • SPC382: Scaling SharePoint Records Centers to Terabytes and Beyond, Russ Houberg, SharePoint Conference 2011. • SPC 311: Best Practices for Building your Website for Scale with SharePoint 2010, Ethan Gur-esh , SharePoint Conference 2011.