ARC06 SharePoint Search Deployment

2. ARC06SharePoint Search Deployment Mike FitzmauriceSenior Technical Product ManagerMicrosoft Corporationmikefitz@microsoft.com

3. Agenda • SharePoint Version Overview • Improvements to Search in MOSS 2007 • Finding People and Expertise • Indexing Unstructured Data • Indexing Structured Data • Customizing the Search UI • Search Usage Reporting

4. SharePoint Version OverviewSharePoint Products and Technologies • Search features differ with specific version • Microsoft Office SharePoint Server 2007 • Enterprise CAL • Standard CAL • Microsoft Office SharePoint Server 2007 for Internet Sites • Microsoft Office SharePoint Server for Search Standard Edition • Microsoft Office SharePoint Server for Search Enterprise Edition • Windows SharePoint Services 3.0 • A details of the features found in each version can be found on www.microsoft.com/office • This session assumes all search functionality is available to you • (i.e., SharePoint Server 2007 w/Enterprise CAL)

5. Search in SharePoint Products and Technologies • Windows SharePoint Services 3.0 • Subset of SharePoint Server search features • No index rebuild during upgrade to SharePoint Server • Indexing and query always on the same machine • Search scope limited to site and sub site content only • Only content stored in WSS 3.0 can be indexed • Microsoft Office SharePoint Server 2007 • Substantial additional features and functionality (e.g. Indexing BDC) • Indexer and query server roles can be distributed • Custom Search scopes across multiple content sources • Managed via a Shared Service Provider • Local and remote content sources can be indexed

6. SharePoint Search ArchitectureSearch topology model User Requests Index Files Property Store Load Balancer Web front ends Query servers Propagationof indexes SQL Server Indexer Crawling Content Sources

7. Relevancy Improvements • Substantially improved relevancy was the number one goal for this release • New algorithms for Web and Enterprise Content • New relevancy elements including • Click Distance • Anchor Text • URL Depth • Metadata Extraction from documents • Testing has shown significantly better relevancy • 500% improvement on common queries • 100% improvement on all queries

8. Security Improvements • Query-time security trimming (Same as SPS 2003) • Custom security trimming API. • Minimum crawler permission is “Full Read” • Still provides the same security trimming functionality • Automatically configured for new sites • Search visibility options • Prevent sites or specific lists appearing in search results • “Security only” crawl and single item removal

9. Shared Service Provider • Unique to Microsoft Office SharePoint Server • i.e., not in Windows SharePoint Services by itself • Groups a set of services • High-value • Resource intensive • Administration can be delegated • Portals consume shared services associated with the SSP • Configurable at the site collection level to add scopes, keywords and best bets. • One index per SSP • Multiple shared service providers in a farm

10. Scalability and Performance Improvements • Tested to about 50 million documents • Fast incremental crawls using the new change log • Continuous Index Propagation • Items appear in results minutes after being indexed • Crawler Impact Rules • Control the impact of crawling on remote sources

11. Improved Management • Simplified and streamlined Admin UI • Improved management and mapping of crawled properties • One index per shared service  no needto worry about managing discrete indexes • Multiple crawler start addresses for each content source • Improved management of search scopes • Scopes decoupled from Content Sources • Scopes can span multiple content sources • Scope by Property, Site, Content Source, URL • Granular rules for scope creation

12. Improved Management • Web Based Index Log • Browse and filter crawler log • Complete admin API • Microsoft Operations Manager Management pack

13. Indexing Management Streamlined experience and more control One index per shared service  no needto worry about managing discrete indexes Multiple start addresses per content source MOSS indexes can drive the WSS search experience Allow upgrade from WSS to MOSS

14. Managing Search

15. Improved End User Experience • Search Center • Clean UI, Search Site Template, Web Parts • New Search Syntax • Implicit AND, Property Search, +/-, Advanced Search • New Results Features • Context Summary, “Did you mean”, Duplicate Collapsing, Hit Highlighting, Best Bets, Auto Definitions • Improved Alerting and Search RSS Feeds • Customization • Web Parts, ASP.NET 2.0, Results with XSLT

16. CoreResults ActionLinks SearchTabs Search Box Statistics Pagination HighConfidence MatchingKeywords AndBest Bets Search Center

17. Improved End User Experience

18. Finding People and Expertise • SharePoint Profile data • Built from AD, BDC, LDAP v3 and manually entered • Privacy controls • My Colleagues • Stored in a list in My Site • Outlook 2007 suggested or manually entered • Privacy controls • Results grouped by social relevancy • My colleagues, my colleagues colleagues, everyone else

19. People and Expertise Search

20. Indexing Unstructured DataExtensible Connectivity • Protocol Handler • Connects to a content source and enumerates the documents • Ships with support for • NTFS File System, Exchange Public Folders, SharePoint sites, HTTP, Lotus Notes, Active Directory • Partners providing support for • Documentum, Hummingbird, FileNet, Interwoven and others • iFilter • Reads a document and extracts the plain text content and properties for the indexer to index • Ships with support for over 200 file types • Partners/Vendors providing support for many more e.g. PDF

21. Protocol Handlers Available with Vista/Desktop Search SharePoint Technologies Known Microsoft protocol handlers: Vista – File, CSC, MAPI SharePoint Server 2007 – HTTP, File Share, Notes, WSS Others – One Note IFilters Available with Indexing Service Vista/Desktop Search SharePoint technologies SQL Server Exchange Server Commonly Used IFilters: Vista/WDS – Office, MIME, HTML, Text, RTF Office 12 – Office, Office12, MSG (and uses OS filters) Others – PDF, TIFF, Zip Protocol Handlers & IFilters

22. Indexing Unstructured Data

23. Indexing Structured DataBusiness Data Catalog • Index and Search Structured Data • NO CODE! XML Application Definition • Sample SAP and Siebel definitions included OOB • Access to data through ADO.NET or Web Service • Render BDC Data in SharePoint Server • Business DataView Web Part • SharePoint Designer • Surface Data in SharePoint Lists • Business Data Column • Create Custom Actions • Import data into User Profiles

24. Indexing Structured Data

25. Customizing the Search UI • Customize the Query • Query Web Part Properties • Custom Search Scopes • Custom Search Center Tabs • Customize the Advanced Search Page • Object Model and Search Web Service • Customize the Results • Core Results Web Part Properties • ASP.NET 2.0 Master Pages and Page Layouts • Apply custom XSL to results set • Custom page for BDC Search results • Object Model and Search Web Service

26. Customizing the Search UI

27. Search Usage Reporting • MOSS 2007 now includes usage reporting out of the box. • Best way to improve search is to understand current usage. • Several built-in search activity reports • Search query reports: Number of queries and top queries. • Search result reports: Top Results, Queries With Zero Results, Results with low click-through and a few more. • Available at both site collection level and SSP level. • Export data for extended reporting in Excel. • Respond to feedback with configuration changes or editorial results (Best Bets).

28. Usage Reporting

29. Search Performance Planning Things to keep in mind Indexing is resource intensive – balance it… Think about freshness goals, change rate Evaluate remote server responsiveness Upgrade WSS 2003 servers with WSS 2007 to index them faster Scale strategies might include Dedicated WFE for crawling Separate indexer machine Separate SQL DB for Search Fast disk for SQL, fast CPU for Indexer, more memory

30. Additional Information • SharePoint Team Blog • http://blogs.msdn.com/sharepoint • SharePoint Server 2007 Developer Portal • http://msdn.microsoft.com/office/sharepoint • Product information • http://www.microsoft.com/sharepoint