Searching for Data: Powered by Google

1. Searching for Data:Powered by Google Paul H. Bern Numeric Data Services Librarian Syracuse University 2007 IASSIST Conference Montreal, Canada

2. Searching for Data and Statistics • Is rather difficult, especially for the novice • Various sites to make it easier: • ICPSR, ESDL, DISC, etc. • Could even use some social bookmarking sites • Or even create your own database of sites/data sets

3. Very Useful but… • Require a great deal of maintenance • Included sites change URLs, add/drop pages • Each site must be annotated for searching • Requires more than basic knowledge of programming • Searching can still be difficult • User has to know it exists

4. Google and Yahoo, etc. • Well known, easy to use • No maintenance • Can return many results that have nothing to do with the topic

5. What we need is: • A way to return many results, but only from relevant sites • It has to be something the novice can use with little or no help • It has to be easy to set up and maintain

6. Google Custom Search Engine • Essentially, a CSE is a way of telling Google “search only these sites” • Up to 5,000 of them to be exact • Relatively easy to set up and maintain • Can be done entirely from web forms • Can have several people contribute to it • Can link or nest several CSEs • Arranged topically • Way to get around 5,000 site limit

7. Some Downsides: • Using the more advanced features can be complicated • It is still a “Google” search, so more popular rather than more appropriate pages may come to the top • At present, it’s still in Beta, so they make changes and things break from time to time

8. So Let’s Take a Look • Numeric Data Services CSE • http://libsites.syr.edu/mgi/google/iassist2007.html • Can host on Google’s site or your own • Hosting on your own allows more customization • May have to highly customize depending on what you do • Hosting on Google’s is easier

9. What You Need to Start: • http://www.google.com/coop/cse/overview • A Google account – it’s free • A list of sites to include • Labeled would be good • A basic idea of how you want to set up the refinements

10. Working with Annotations • An annotation is a URL • Can be of a standard format or it can include wildcard character to include only parts of certain domains • Easiest to generate a tab-separated file and upload • Be sure it is in the proper format • Use Google Marker bookmark to add new sites • Download again in XML to use some of the advanced features and as a backup • Gotcha: when you download your sites, ALL sites associated with your account will be downloaded, not just those for a particular CSE!

11. Working With Refinements • Refinements can either redirect the user to another search engine (Google or otherwise) or can be used to modify the search results • Include/exclude only sites with a specific label • Give a “Boost” labeled sites so they move up in the results list • Only good if you search entire web • Inconsistent results • You can have up to 16 refinements in each CSE