FAST Search for Sharepoint 2010

1. FAST Search for Sharepoint 2010 Changing the Enterprise Search game Dan Wilson Microsoft Switzerland dan.wilson@microsoft.com

2. Introducing FAST Search for SharePoint OOB User Experience Tailoring General Productivity Search Platform and Architecture Search Driven Applications Deployment and Administration Summary and Resources

3. Session Objectives and Takeaways • Objective: help you understand FAST Search for SharePoint • Key Features and Capabilities • How to deploy and customize • Applications and examples • FAST Search for SharePoint is NEW • Combines best of high-end search, SharePoint and Microsoft • Extends SharePoint Server 2010 • Covers general productivity AND search-driven applications

4. Today’s Choices Force Compromise Out-of-the-Box Products High-end Products OR easy to manage highly capable but but limited in capability hard to manage, expensive

5. Our Vision: A New Choice in Search Enterprise Search from Microsoft Best of High-end • Best of SharePoint • Out-of-the-box intranet search • People Search • SharePoint platform integration • No code customization • Dynamic user experiences • Advanced content processing • Broad repository connectivity • Extreme scale and performance • Open and Extensible Platform Best of Microsoft • Simplicity and low TCO • Familiar, integrated developer tools • Windows 7 Integration • Robust partner and platform eco-system

6. Search solutions can be broad and deep Search Driven Applications Productivity Search Experience • Drive measurable ROI by helping a business group make the most of a specific set of information • Customer insight • Competitive intelligence • Research portals • Connect all of your people with a broad set of information • Enterprise content and data • Intranet sites • Team workspaces • People and expertise

7. Which search offering is for me? General productivity search Light customization Customized productivity search and search driven applications

8. Introducing FAST Search for SharePoint OOB User Experience Tailoring General Productivity Search Platform and Architecture Search Driven Applications Deployment and Administration Summary and Resources

9. DEMO Contoso Consulting

10. Enables conversational experience across all of the results You will never miss any content Enabling better findability and exploration Discover non-obvious relationships across the entire result set Exact counts shows relative weight Provides analytic view of your results Indicates priority and importance The right lever to slice and dice your content Sort on any field Empower the user to use the relevance model that best fits their needs Rearrange the result set to meet specific criteria Alphabetical, numeric, and date Enables precise control of results Deep Refinement and Sorting Out of the Box Sorting Options Exact Counts

11. Introducing FAST Search for SharePoint OOB User Experience Tailoring General Productivity Search Platform and Architecture Search Driven Applications Deployment and Administration Summary and Resources

12. Custom Query Suggestions

13. Query Suggestions & the Term Store Maintaining a custom query suggestion list using the SharePoint Term Store repository, instead of putting the information in a list on a site. • # retreive values from termstore • $ts = Get-SPTaxonomySession -Site http://esgismoss14 • $groups = $ts.DefaultKeywordsTermStore.Groups • $groups | select Name, Description, TermSets • # Get a reference to the query application • $searchapp = Get-SPEnterpriseSearchServiceApplication -Identity "FAST Query SSA" • # Add new query suggestions • new-spenterprisesearchlanguageresourcephrase -SearchApplication $searchapp -Language En-Us -Type QuerySuggestionAlwaysSuggest -Name "faster“ • # Start the job that deploys the query suggestions (runs once every 24 hours) • Start-SPTimerJob -Identity "prepare query suggestions" • # List query suggestions • Get-SPEnterpriseSearchQuerySuggestionCandidates -SearchApplication $searchapp • # Remove query suggestions • Remove-spenterprisesearchlanguageresourcephrase -SearchApplication $searchapp -Language En-Us -Type QuerySuggestionAlwaysSuggest -Identity "faster"

14. What’s for Lunch? Visual Best Bets at Hafslund

15. Audience-specific search experiences Use User Context to meet the needs of diverse groups Renee Lo Engineering Contoso Consulting ”What should I know about implementing ERP?” Alan Brewer Sales Manager Contoso Consulting ”What should I know about selling ERP consulting?” Business Unit Department Team Time of Day Preferred Sites SharePoint Audiences Interests & Current Projects Context of Current Task Username & Group Memberships Location Languages

16. DEMO Search Configuration Example:Refinement Example:Visual Best Bets

17. Introducing FAST Search for SharePoint OOB User Experience Tailoring General Productivity Search Platform and Architecture Search Driven Applications Deployment and Administration Summary and Resources

18. SharePoint Server(s) FAST Search for SharePointHigh Level Architecture FAST Server(s) … … Other Server(s) Lemmatization Format Conversion EntityExtraction Language Detection Mapper Content Processing Pipeline Content OpenSearch Query Processor Indexer Content Processor Crawler Federation People Search Search Center User Profiles Index Partition

19. How does the pipeline work? A systematic approach to interpreting your content • Sequential stages perform specific tasks while ingesting content • Breaks down content to the smallest addressable chunks to build meaning • Understands file encoding, data formats, and written languages • Supports 400+ file formats, 80+ languages • Process your content to make it searchable • Normalizes content so that a consistent relevancy model can be applied • Identifies structured and unstructured metadata in your content • Maps document metadata to SharePoint Crawled Properties

20. Extending Pipeline capabilities Straightforward way to add custom text analysis functionality Configure Optional Processing Steps • XML Properties mapper • Offensive Content Filter • Field Collapsing • Verbatim (wholeword) extractor • Use a dictionary for custom extraction • Pipeline Extensibility • Calls external applications for custom item processing Add Custom Processing Pipeline Extensibility is a specially defined stage that takes a set of crawled properties, as flat text, as input and maps output to another crawled property Sandboxed execution Executable arguments and temporary files are automatically handled with timeouts. Runs just before the Crawled Property Mapper, providing accessibility within SharePoint

21. Tune relevancy to improve accuracy Changing content and users need require a flexible solution • Start with great relevance OOB • Tuned for great general productivity experience • Automatically improves relevancy with social click-throughs and link text analysis • Create new relevance models • MultipleRankProfiles • Blend static and dynamic ranking parameters to instantly improve search results • Create with simple PowerShell commands • Expose as new sorting options Standard Sorting Options Custom Rank Profiles

22. How to Tune Relevancy Improve accuracy and control with Rank Profiles Rank Profiles are made by combining multiple ranking elements Create custom ranking algorithms to combine multiple ranking properties Rank Profiles are exposed by modifying the sorting web part.

23. Search 2010 “Stack”The platform for Search Customization What’s New in 2010? • Primary Search Web Parts now Unsealed • Federation now a key Public OM layer • All Web Parts built on federation • Query alteration, custom Runtimes, blending results from multiple sources • Web Service / RSS Enhancements FAST Search / SharePoint Search: • Shared Web Parts, RSS, Web Service • Shared Federation OM • Index and Crawling Separate • FAST unique Content Processing Pipeline SearchWeb Parts SearchWeb Parts Federation / Query OM Federation OM Web Service, RSS Web Service, RSS OpenSearch / CustomSource SharePoint Search Index FASTSearch Index FAST Indexer SharePoint Indexer Content Processing

24. Robust query language Use FAST Query Language (FQL) for precise query development • FQL provides a robust and expressive query language • Wildcard support - *, ? • Numeric Data types (Integer, Float, Decimal, Datetime) • Operators • Direct field access (e.g., title:othello, author:shakespeare) • Numeric (COUNT, RANGE, <, <=, >, >=) • Boolean (AND, OR, ANY, NOT) • Rank (RANK, XRANK) • Proximity (NEAR, ONEAR) • Sorting (SORT, SORTFORMULA) • String (operator support for strings) • Boundary (starts-with, ends-with, equals) • Filter

25. Query Language ExpressivenessSoft boost using the Xrank operator Boost all documents that contain the name Diane Tibbott Boost all documents that contain the name Mary Baker Or, in the FAST Query Language: Single (documents containing either Mary, Diane or both gets a boost of 5000): Xrank(string(”performance”), or(person:string(”diane tibbott”), person:string(”mary baker”)), boost=5000)) Accumulative (documents with both Mary and Diane gets a boost of 10000): Xrank(Xrank(string(”performance”), person:string(”diane tibbott”), boost=5000), person:string(”mary baker”), boost=5000)

26. Introducing FAST Search for SharePoint OOB User Experience Tailoring General Productivity Search Platform and Architecture Search Driven Applications Deployment and Administration Summary and Resources

27. Search Driven Applications Meet all the search application needs you have across your business Sales: 360o Customer Insight “How do I support theunique search needs of teams and work that impact our business?” Services: Knowledge Browser Marketing: Competitive Intelligence • To do so, you need a search platform that has • A deep understanding of your information • Flexible relevance to meet diverse needs • A customizable UX to increase user efficiency Research & Development: Innovation Portal Support: Call Center Advisor Operations: Systems/Logistics Portal Legal, HR, IT, Finance, ……

28. Top information from Woodgrove…new market view report to send to clients Set of Customers to explore, with rollup Experts to help, with availability and rating View of information across different pivots, with drilldown Immediate actions on selected items News and external opinion to monitor and send to clients Drilldown to single view with all clues about a customer: portfolio, holdings, communications, annual and quarterly customer plans, etc…

29. Content Crawling: bring in data from lots of places Connectors to SharePoint (reports, account documents), exchange public folders, shared files; BDC with customization in SPD (no code) for customer portfolio/holdings Content processing: creating metadata Names of holdings, offerings, key concepts, companies, people Synonyms for key concepts (real estate ~ REIT) OOB web parts configured for style Federation, People Search, Search actions Custom web parts for visual navigation Roll-up configured via results collapsing Custom relevance profile SharePoint workflows for act-on-selected-items How would you create this?

32. Introducing FAST Search for SharePoint OOB User Experience Tailoring General Productivity Search Platform and Architecture Search Driven Applications Deployment and Administration Summary and Resources

33. SharePoint Server(s) FAST Search for SharePointSummary of architectural elements FAST Server(s) Other Server(s) • Site Collection Level Admin UI • Keyword Management • User Context Management • Site Promotion/Demotion • PowerShell • Schema configuration • Admin configuration • Deployment configuration • Central Administration UI • Property mapping • Property extraction • Spell-checking Administration and Schema Object Model • Connectors • Web Crawler • JDBC Advanced Content Processing Linguistics Web Link Analysis Security Access Module Indexing SharePoint Front-end • Connectors • SharePoint • File Traverser • Web • BDC • Exchange • Notes • Documentum Query Object Model Query and Result Processing Search Core Custom Front-End Query Web Service Federation Object Model Monitoring Services Content Microsoft System Center Operations Manager OpenSearch or Other Sources People Search

34. FAST Search for SharePoint Scaleout • Back-end with extreme and flexible scale out options • Scale-out multiple “dimensions” • Query Volume • Content Volume • Indexing freshness • Redundancy options • Search • Indexing • Performance targets* • 15M Docs/node • 10 QPS/node Query Volume Search and Indexing Query and Result Processing Content Volume No theoretical upper bounds! Crawling and Content Processing * Dependent on document and HW charateristics

35. Simplified, powerful administration A high-end enterprise search solution that’s easy to deploy and manage Manage efficiently with full support for Microsoft System Center and PowerShell scripting to automate tasks Deploy easilyusing wizard-driven installation, a topology designer, and native support for 64-bit virtualization Streamline administrationwith a simplified admin console that helps you manage search services across your enterprise

36. Introducing FAST Search for SharePoint OOB User Experience Tailoring General Productivity Search Platform and Architecture Search Driven Applications Deployment and Administration Summary and Resources

37. Turn information into business outcomes through engaging experiences. Social Connect and collaborate to apply your talent more effectively Conversational Change how you interact with information, giving you better answers ContextualTailored to your role, task, and perspective, making you more productive Visual Help you identify patterns and discover new insights

38. Target, Differentiators+ Fundamentals General Productivity search Light customization Social, Conversational Connectivity, Security, Relevance, Performance, Administration, Scale Customized productivity search and search driven applications Social, Conversational++, Visual, Contextual Connectivity+, Security, Relevance+, Performance, Administration+, Scale++ • SharePoint Enterprise CAL + FAST Search for SharePoint server license • SharePoint Standard CAL + SharePoint server license

39. Public sites Technet.microsoft.com/enterprisesearch Enterprise resource center on MSDN Search areas on Sharepoint.microsoft.com Microsoft.com/searchserverexpress Enterprise Search Blog Enterprise Search Content

40. Thank you for your Attention! • For more Information please contact • Dan Wilson • Enterprise Search Specialist, Microsoft • dan.wilson@microsoft.com • Mobile 078 844 6104