Solr Facets in Alfresco 4

1. Solr Facets in Alfresco 4 Dan Tuffery

2. Who Am I? About Me • Senior Software Engineer at Ixxus • Working with Alfresco & Solr for 2.5 years • Alfresco Certified Engineer and Solr Certified • Winner of Alfresco’s 2012 Dashlet Challenge

3. Agenda Talk Outline • Alfresco Solr facets demonstration • Solr facets overview • Walkthrough of implementation • Plans for the module

4. Alfresco Solr Facets Demonstration

5. Solr Facets Overview 3 Different flavours of facets • Field facets • Date facets • Number range facets

6. Indexing Facet Fields • Indexed • Not Tokenised • Not Stored

7. Facet Field Parameters • facet = enables faceting • facet.field = the field that you want to facet on • facet.mincount = constraint count >= facet.mincount • facet.limit = limit the number of constraints that are displayed

8. Facet Field Parameters • facet = enables faceting • facet.field = the field that you want to facet on • facet.mincount = constraint count >= facet.mincount • facet.limit = limit the number of constraints that are displayed q=myfield:solr&facet=true&facet.field=myfield&facet.mincount=1&facet.limit=20

9. Date Facet Parameters • facet.date= name of date field • facet.date.start= start of the date range • facet.date.end = End of the date range • facet.date.gap = period of time in between each date range

10. Date Facet Parameters • facet.date= name of date field • facet.date.start= start of the date range • facet.date.end = End of the date range • facet.date.gap = period of time in between each date range q=myfield:solr&facet=true&facet.date=created&facet.limit=20&facet.date.start=2012-0101T00:00:00.0Z&facet.date.gap =1MONTH&facet.date.end=2012-10-22T00:00:00.0Z

11. Solr Facets In Alfresco 4 What Needed To Be Done?

12. Alfresco’s Facet Implementation Out Of The Box • Facet field POJO • Logic to add facet field params to Solr request URL • Facet results are mapped to JSON object

13. My Implementation Five Main Tasks • Add facets to Alfresco (Facet Manager) • Solution to add facets to search request • Solution to return facets in the search response • Modify search results page to display facets • Add support for filter queries

14. 1)Add Facets to Alfresco Facet Manger • Tool in Admin Console • Based on current Tag Manager • Create, Read, Update and Delete facets • REST API in Alfresco (POST, GET, PUT, DELETE) • Custom content model – ix:facetField

15. 1)Add Facets to Alfresco What facets to display • Content Type, Mime type, Property, Date How the facet should be displayed • Display Name, Minimum Count, Limit When the facet should be displayed • Individual Site, All Sites, Repository Search

16. Indexing Facet Fields in Alfresco Best Practices • Indexed • Not Tokenised • Not Stored

17. Indexing Facet Fields in Alfresco Best Practices • Indexed • Not Tokenised • Not Stored

18. Indexing Facet Fields in Alfresco Best Practices • Indexed • Not Tokenised • Not Stored

19. Search Flow of Execution OOTB

20. Search Flow of Execution

21. 2) Add Facets to Search Request • Search request sent from Share • search.lib.js var queryDef = { query: ftsQuery, language: "fts-alfresco", page: {maxItems: params.maxResults * 2}, templates: getQueryTemplate(), defaultField: "keywords", onerror: "no-results", sort: sortColumns, searchScope: searchScope, siteId: params.siteId, filterQueries: filterQueries }; nodes = search.query(queryDef);

22. 2) Add Facets to Search Request • Extended Search.java String searchScope = (String)def.get("searchScope"); String siteId = (String)def.get("siteId"); .. .. if(searchScope != null) { List<FieldFacet> fieldFacets = facetService.getFieldFacets(searchScope, siteId); searchParameters.setFieldFacets(fieldFacets); } .. results = this.services.getSearchService().query(searchParameters);

23. 2) Add Facets to Search Request • Extended SolrQueryHttpClient.java • Generates Solr Request URL • Add parameters (including facets) to URL List<FieldFacet> fieldFacets = searchParameters.getFieldFacets(); for(FieldFacet facet : fieldFacets) { url.append("&facet.field=" + facet.getField()); url.append("&facet.mincount=" + facet.getMinCount());, url.append("&facet.limit=" + facet.getLimit()); }

24. 2) Add Facets to Search Request • Extended SolrQueryHttpClient.java • Generates Solr Request URL • Add parameters (including facets) to URL List<FieldFacet> fieldFacets = searchParameters.getFieldFacets(); for(FieldFacet facet : fieldFacets) { url.append("&facet.field=" + facet.getField()); url.append("&facet.mincount=" + facet.getMinCount());, url.append("&facet.limit=" + facet.getLimit()); } facet.field=@{http://www.ixxus.com/model/1.0}location.__.u

25. 3) Return Facets in Search Response • Results mapped to SolrJSONResultSet.java • Response returns set of ScriptNode objects Collection<Object> set = new LinkedHashSet<Object>(results.length(), 1.0f); .. //OOTB code to add scriptNodes to the set .. if(searchParameters.getFieldFacets().size() > 0) { List<FieldFacet> fieldFacets = sp.getFieldFacets(); for(FieldFacet facetField : fieldFacets) { List<Pair<String, Integer>> facets = ((SolrJSONResultSet)results).getFieldFacet(facetField.getField()); set.add(new ScriptFacet(facetField, facets, getScope(), this.services)); } } return set;

26. 3) Return Facets in Search Response • ScriptFacet Implements Scopable • Represents a facet field and its properties • Constraint filter query is constructed here public ScriptFacet(IxxusFieldFacet facetField, List<Pair<String, Integer>> facetList, Scriptable scope, ServiceRegistry services) { this.services = services; this.facetField = facetField; this.scope = scope; this.constraints = getFacetResults(facetList); } public Boolean isFacet() { return isFacet; }

27. 3) Return Facets in Search Response • Back up to search.lib.js to process results var results, i, j, facetResults; for (i = 0, j = nodes.length; i < j; i++) { if(nodes[i].isFacet) { var facet = nodes[i]; if(facet.facetHasAnyHits) { facetResults.push(facet); } } else {//it is a ScriptNode object, so process it in the usual way .. //OOTB code to add search results to results list } } return ({ items: results, facetItems: facetResults });

28. 3) Return Facets in Search Response • search.get.json.ftl "items": [ ... ], "facets": [ { "facetDisplayName":"Location", “constraints" : [ { "constraint": "San Jose (7)", "fq":"@{http://www.alfresco.org/model/content/1.0}title.__.u:San Jose" }, { "constraint":"Berlin (4)", "fq":"@{http://www.alfresco.org/model/content/1.0}title.__.u:Berlin" }, ... ] } ]

29. 4) Modify Search Results Page • Search Results page layout changed dynamically • Nested divs to display facet column

30. 4) Modify Search Results Page • search.js (YUI) var facets = oFullResponse.facets, resultsContainer, facetsDiv, results, body; if(facets && facets.length > 0) { resultsContainer = new Element(document.createElement("div"), { id: “results-container" }).addClass("results-container"); facetsDiv = new Element(document.createElement("div"), { id: "facetsDiv" }).addClass("search-facets"); Ixxus.search.solr.generateFacets(me, facets, facetsDiv); body = new Element(Dom.get("body")); results = new Element(Dom.get("results")); results.addClass("results-right"); resultsContainer.appendChild(facetsDiv); resultsContainer.appendChild(results); body.appendChild(resultsContainer); }

31. 5) Add Support For Filter Queries • Each facet constraint is a link • Original search is executed + filter query/queries • Same flow of execution as facets • Filter queries added to the Solr request URL

32. 5) Add Support For Filter Queries • SolrQueryHttpClient.java List<String> filterQueries = searchParameters.getFilterQueries(); for(String filterQuery : filterQueries) { filterQuery = filterQuery.replace("{", "\\{"); filterQuery = filterQuery.replace("http", "http\\"); filterQuery = filterQuery.replace("}", "\\}"); filterQuery = filterQuery.replace(" ", "*"); filterQuery = filterQuery.replace("TO", " TO "); url.append("&fq=").append(encoder.encode("{!lucene}" + filterQuery, "UTF8")); }

33. Module Plans IxxusSolr • Code has been contributed back to Alfresco • Alfresco Enterprise 4.2 (Spring 2013) • Maybe other community release before then

34. Module Plans TODO • Number range facets • Multi field faceting • Hierarchical faceting

35. Additional Information • Search Terms Dashlet • Uses faceting in Alfresco 4 • https://github.com/ixxus/ix-search-terms-dashlet • Ixxus blog • http://www.ixxus.com/blog/2012/10/solr-facets-alfresco-walkthrough/

36. Summary • Easy and configurable way to add facet fields • Solr facets • Field Facets • Date Facets • Support for filter queries

37. Questions?