Customizing and Extending Microsoft Office SharePoint Search

1. Customizing and Extending Microsoft Office SharePoint Search Tom Rizzo Director, SharePoint thomriz@microsoft.com

2. Session Objectives • Session Objectives: • Modify the SharePoint Server search UI • Consume SharePoint Search within external applications • Extend the reach of SharePoint search • Build a custom security trimmer • Learn more about Microsoft Search Server and Federation

3. Agenda • Customizing the built in user experience • Extending the OOB experience

4. ActionLinks SearchTabs SearchBox CoreResults Statistics Pagination HighConfidence MatchingKeywordsandBest Bets Search Center

5. Sources, Scopes and Tabs • All Sites and People tabs provided OOB • Add your own Search Tabs • Tabs can be added for additional scopes • Specific site address • Specific property • Specific content source • Include/Exclude • Consists of a layout page and a results page

6. Search Web Parts • 9 OOB web parts • Search Box • Core Results • High Confidence • Statistics • Pagination • Action Links • Matching Keywords and Best Bets • Search Summary (Did you mean?) • Advanced Search • 2 New ones for MSS – Federated Results and Top Federated Results • Web part properties such as • Formatting • Turning stemming on/off • Number of results returned • XSL

7. Thesuarus and Synonyms • Customize Thesaurus and Noise Words • Located in drive:\Program Files\Microsoft Office Servers\12.0\Data\Config • tsxxx.xml and noisexxx.txt where xxx is the language code • Thesaurus supports Replacement or Expansion sets • Substitution weights and stemming are supported

8. Thesaurus Example • <replacement> • <pat>WinNT</pat> • _{Windows Server} • </replacement> • <expansion> • <sub weight="1.0">Office SharePoint Server</sub> • <sub weight="0.6">SharePoint</sub> • <sub weight="0.8">MOSS</sub> • </expansion>

9. Tuning Results • Best bets • Setup best bets at the site collection level • Simple things you can do for relevance • Authoritative pages • Demoted sites • Exclude specific items • Managed Properties • Add your own • Add them to Advanced Search • Change weighting of properties (via the OM, requires a bit of code)

10. Customize UI With XSLT • Create a custom look and feel or results layout • Modified through web part properties • Leverage the power of XSLT • Formatting • Logic • Math

11. Walk through of the XSLT • Parameter Section • Parameters passed by results, web part properties • No Keyword Section • You didn’t enter a keyword • No Results Section • No results were returned • Customize with your own message to help users • Main Body Result Section • Where it all happens • Results Template Section • Matches results in XML • Selected column matching – Title, URL, etc. • Hit Highlighting, Auto definition

12. demo Customizing the Search UI

13. Recap of Options • Add/Remove/Change Tabs • Add/Remove OOB Web Parts • Add custom web parts • Tune results • Change Web Part Properties • Change XSL

14. Agenda • Customizing the built in user experience • Extending the OOB experience

15. Extending Search • Customizing the Query and Results • Query Object Model • Customizing the Index • Index custom data - Protocol Handlers, IFilters, BDC • Custom Query Time Security Trimmer

16. Query SyntaxURL Syntax • Use Case • Launching a URL from a custom application • Windows Desktop Search • Save searches • Keywords • results.aspx?k=fish • Scopes • results.aspx?k=fish&s=BDC • Sort/View • results.aspx?v=date • results.aspx?v=relevance • Page • results.aspx?start=21 • Property matching • results.aspx?k=“author:Tom –site:http://tom.com”

17. Query Object Model • Use query OM • To build custom search UI web parts • To have direct access to query and results properties • To invoke custom queries • 2 types of query syntaxes • Keyword • SQL

18. Query SyntaxKeyword Syntax • Use Case • Business user • Simple and easy to use • Required and excluded term support • Bike –fitness • SharePoint search -author:”Tom Rizzo” • Filter by any string or integer property • Author:Tom Rizzo • Employees:30 • Implicit “AND” behavior • Consistent property:value syntax across Office, Windows and Live search gallery hinges –brass site:http//supportdeskscope:Products

19. Query SyntaxSQL Syntax • Use Case • Advanced search solutions • Complex queries • Consistent SQL across enterprise and desktop SELECT URL, Title, Description FROM portal..SCOPE() WHERE FREETEXT(‘gallery hinges’) AND SITE = “http://supportdesk” AND SCOPE = “Products” AND NOT CONTAINS(‘brass’)

20. Query SyntaxSQL Syntax • Full strength, complete coverage • FREETEXT() • Arbitrary groupings for AND, OR, NOT • CONTAINS() • LIKE • ORDER BY ASC | DESC • Removed in MOSS 2007 • UNION ALL • MATCHES • SELECT * • COALESCE TABLE  

21. Query Object ModelFeatures • Managed code API • Single request – multiple results • Result Types • Relevant results • High confidence results • Special terms • Definitions • Optional parameters • # of Sentences in Summary • Implicit - AND/OR • Number of results • Ignore noise words • Enable stemming • Language

22. KeywordQuery Result Table Collection SQLQuery Relevantresults Definitions Execute() QueryEngine Highconfidence OptionalParameters Specialterms Local Query Object Path Input Query OM Output ResultTable:IDataReader Site UI CustomClient

23. However… • Adding Custom Web Parts • OOB Web Parts use sealed hidden object • You’ll need your own query object to share with other Web Parts • You *can* use URL request parameters we pass between pages

24. Query Web Service • Use Case • Office Research Pane • Windows Desktop Search • Remote application • ASP.Net 2.0, Win Forms, Java apps • Similar features to query object model • Results in XML or data table formats • http://o12server/_vti_bin/search.asmx

25. Query Web Service • Methods • Query – returns results as XML format • Research and Reference Service-compatible XML • Used in the Office Research pane • QueryEx – • Returns .NET DataSet • Different result types, features • GetSearchMetaData – Retrieve available properties and scopes • Registration • Status

26. demo Query Web Service

27. Refine results using various properties View byrelevance orsocial distance Finding People

28. Finding People • People Search • Implements the core results • Customizes the search experience • Refine by Job Title/ Department • Sort by social distance • Display results differently

29. Indexing Custom DataProtocol Handler • MOSS ships with support for • Web Content, NTFS File Shares, Exchange Public Folders, Lotus Notes DBs, SharePoint Content, SharePoint Profiles, Business Data Catalog • Develop a Protocol Handler to gather data from custom repositories • e.g. Vorsite for Documentum • Connects to a content source and enumerates the documents • Interfaces are the same as in SPS 2003 • New registry path: • Register PH under HKLM\Software\Microsoft\Office Server\12.0\Search\Setup\ProtocolHandlers

30. Indexing Custom DataIFilter • Install OneNote IFilter (requires OneNote on the server ) • Install PDF IFilter (64 bit now available) • Develop IFilter for custom file formats • Reads a document and extracts the plain text content and properties for the indexer to index • Support reading from streams

31. Indexing Custom DataBusiness Data Search • Integration without writing code • Information in LOB systems is often hard to access • MOSS 2007 can bring that data to your users • Data is accessed through the Business Data Catalog • Exposed through Search, Meta Data, BDC web parts • Once a web service or a relational data source is registered with the BDC, search can easily index the data • No need to write code • Highly customizable results • Integrated with scopes and search center

32. Security • Built-in Query-time security trimming (Same as SPS 2003) • File shares, WSS/SPS 2003, Exchange, Lotus Notes(via mapping) • New! Support for result time custom security trimming • ISecurityTrimmer interface (Microsoft.Office.Server.Search.Query namespace) • Implement Initialize() and CheckAccess() • Add the custom security trimmer assembly to the GAC • Register the security trimmer using stsadm command

33. New Federation Capability

34. Federation Overview • Enables the display of results from other search engines or applications to be displayed alongside local results • Search Server will • Send a query to other search engines or applications • Format and render the results • Work out of the box or with minimal effort with many existing search engines and applications • Search Server will not • Aggregate multiple result sets into a single result set • Relevance rank results from other search engines or applications

36. Federated Locations • Each search engine or application must be configured as a Federated Search Location • Each Federated Search Location • Defines how and when queries will be federated to the location • Controls the formatting of results • Defines authentication settings • Can be connected to one or more Federation Web Parts to render results • Is easily imported and exported as a Federated Location Definition file (.FLD)

37. Supported Location Types • OpenSearch 1.0/1.1 • Query is sent in a parameterized URL • http://search.live.com/results.aspx?q={searchTerms} • {searchTerms} is replaced with query terms • Results must be returned as XML • Local Search Index • Query is sent through the search object model • {searchTerms} scope:customers • {searchTerms} is replaced with query terms • Results are returned as XML

38. A Note On OpenSearch • Standard created by Amazon A9 for search syndication and aggregation • An exported Federated Location Definition file is >= an OpenSearch Description Document • Search Server extends the OpenSearch schema to include properties such as; • Triggers • XSL • Location Type • More Results Link… • See www.opensearch.org for more info

39. Location Triggers • Control when to send query terms to a location based on the terms submitted • Always - Will always send all the query terms to the location • Prefix - If the prefix matches will only send the query terms after the prefix to the location • Pattern - Will send some or all of the query terms based on the regular expression match. A pattern trigger can be used to build complex triggers including “OR” statements

40. Federation Web Parts • Federated Results Web Part • Connected to a single Federated Location • Displays the top X results returned from the location • Top Federated Results Web Part • Connected to multiple Federated locations • Displays the top X results from the first location to return a result in the order of the configured locations • Neither Web Part will render if the location is triggered but it returns zero results

41. Rendering Results • Built in XSL for • OpenSearch Locations that return results as RSS or ATOM • Local Search Index results • Built in XSL can be customized or replaced • Configured in the Location Definition and can be overridden on individual Web Parts • Separate XSL definition for both of the Federated Results Web Parts • “More Results” Link Template available for click through to HTML results

42. Authentication • Three modes of authenticating to a location • Anonymous – No credentials required • Corporate – Shared credentials for every user • User – Per user credentials • Per user credentials requires custom work • Code to capture credentials in the UI • Code to store credentials if required • Code to retrieve stored credentials if required • Sample code will be available on MSDN by RTM • Exception is Kerberos which does work OOB • Per user credentials are sent to the location in the correct format automatically

43. An Example: www.live.com • How to connect Search Server to Live Search • Query Template • http://search.live.com/results.aspx?q={searchTerms}&format=rss • “More Results” Link Template • http://search.live.com/results.aspx?q={searchTerms} • Works without any extra effort because live.com can • Process a query in a URL • Return results formatted as XML (RSS) • Be accessed anonymously • There are many other search engines and applications that provide the same connectivity

44. Federated Search Connectors • Develop or purchase if the location does not support one or both of the following • Process a query in a URL parameter • Return results as XML • Federated Search Connectors do one or both of the following • Accept a query as a URL parameter and convert it to a format compatible with the location • Convert and render results in XML • Developed as a web page (aspx) • Packaged and deployed as a Solution Deployment Package

45. An Example: SQL Server • Connect Search Server to the full text search engine in SQL Server • Cannot send a query to SQL in a URL • Build an ASPX page that • Provides a URL for the Query Template • Extracts the {searchTerms} • Contains code to run the query against SQL • Formats and renders the results as XML (RSS) • If required include code to leverage either corporate or per user authentication • Packaged up and installed as a Solution Deployment Package

46. Connector Gallery • Federated Location Definition Gallery • Linked from within the admin UI • Selection of FLD’s to download and import from Microsoft and Community submissions • Quick, easy and free • Federated Search Connector Gallery • Partner built Federated Search Connectors • Primarily for LOB applications and services • Available for purchase

47. Summary • The user interface is easily customized to achieve a different look and feel • It’s easy to extend search to develop custom applications, local or remote • You can plug in any data source and search it securely

48. Resources

49. © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.