Terence K. Huwe Director of Library & Information Resources

Track A | Tech Developments & Trends | A105Meaning-Based Computing:New Functionalities from the World of Enterprise Search Terence K. Huwe Director of Library & Information Resources Institute for Research on Labor & Employment University of California, Berkeley thuwe@library.berkeley.edu

Overview • Meaning-Based Computing (MBC): • What it is, where it came from • Its rapidly evolving potential • Some examples of its impact on work, and the professions • Its applicability to the research process • Some forecasts on what MBC may bring to the information professions Tech Trends | A105 | IRLE | University of California, Berkeley

The Importance of Forecasting Probability • How should we modify our beliefs in the light of new information? • “When the facts change, I change my opinion. What do you do, sir? • John Maynard Keynes From: The Theory That Would Not Die, Sharon BertschMcGrayne, Yale, 2011 Tech Trends | A105 | IRLE | University of California, Berkeley

Bayseian Theory Sheds Light • Thomas Bayse’ work was published after his death in 1763 by Richard Price • Bayse was interested in proving the existence of God • He labored in obscurity but has since gained fame Tech Trends | A105 | IRLE | University of California, Berkeley

What is Bayesian Analysis? “Scientific inquiry is an iterative process of integrating and accumulating information. Investigators assess the current state of knowledge regarding the issue of interest, gather new data to address remaining questions, and then update and refine their understanding to incorporate both new and old data. Bayesian inference provides a logical, quantitative framework for this process. It has been applied in a multitude of scientific, technological, and policy settings.” --The International Society for Bayseian Analysis Tech Trends | A105 | IRLE | University of California, Berkeley

Tech Trends | A105 | IRLE | University of California, Berkeley

1. Applications and Potential Uses • Used to help break the Enigma Code • Handwriting and speech recognition • Military uses • Manufacturing and sales efficiencies • Legal compliance and due diligence • Pure scientific research • And—information & records management Tech Trends | A105 | IRLE | University of California, Berkeley

New Life from British Roots • Michael Lynch (Cambridge) saw the broad potential of probability study • His academic research led to the founding of Autonomy, a FTSE 100 firm, recently acquired by Hewlett-Packard • The catalyst for market success: enterprise search, and the ascendance of unstructured data Tech Trends | A105 | IRLE | University of California, Berkeley

Enterprise Search As Test Bed • 80 percent of a firm’s info assets are unstructured, and thus hard to retrieve conventionally • Conventional search “imposes” structure onto data to categorize and retrieve it • The Intellgient Data Operating Layer (IDOL) searches both structured (databases) and unstructured data Tech Trends | A105 | IRLE | University of California, Berkeley

Two Events Furthered the Growth of MBC • In 2007, the U.S. Federal Rules of Civil Procedure made all data forms admissible for litigation—remember Enron? • The explosion in social media has created new challenges for firms, requiring that they track unstructured information more effectively Tech Trends | A105 | IRLE | University of California, Berkeley

Since So Much Data Are Now Admissible • If firms have liability attached to online discourse, social media, etc, they need protection • Michael Lynch founded Autonomy to provide just that: enterprise search across all media types, enabling fuller awareness of data assets Tech Trends | A105 | IRLE | University of California, Berkeley

Enterprise Search is Booming • Enterprise Search is now Pan-Enterprise • Many Fortune 500 firms recognize that they need new tools for managing both structured and unstructured data • It’s big business—MBC thrives in commercial and pure research settings Tech Trends | A105 | IRLE | University of California, Berkeley

Autonomy’s MBC-Based Tools • Implistic Query— “hotkey” to related information without leaving a primary task • Hyperlinking—live links, diverse sources • Smart, or Active Folders • Automatic Taxonomy Generation • Sentiment Analysis • Automatic clustering of all data types Tech Trends | A105 | IRLE | University of California, Berkeley

2. The Lawyer vs. the Algorithm • The “Discovery” process meets “E-Discovery” • Tracing meaning by linking varied word use • Firms Like Blackstone Discovery are building a market niche • Result: No armies of lawyers billing their time • The client saves on legal bills, and law firms confront a new challenge Tech Trends | A105 | IRLE | University of California, Berkeley

John Kelly, CEO, Blackstone Discovery, Palo Alto, CA: “Data are talking to each other in the ‘third person’” MBC-driven techniques can uncover crucial data for litigation by tracing relationships Tech Trends | A105 | IRLE | University of California, Berkeley

3. Impact on the Information Professions • Coming our way soon? • Still seeping from the enterprise search world. Some highlights: • “Meaning Based Healthcare” • Universities use it at the enterprise level (admission, etc) • Consulting • Telecommunications Tech Trends | A105 | IRLE | University of California, Berkeley

Potential Applications • Turbo-charged meta-search • Effective search of unstructured data (including social media) • Establish relationships between structured information (libraries and databases) and unstructured information (social media, voicemail, audio) Tech Trends | A105 | IRLE | University of California, Berkeley

MBC and Taxonomy-Based Search • Taxonomies continue to gain market share • Taxonomy & MBC solutions might coexist • Why? Because MBC can manage social media categorization as an automated process • For this to happen, developers need to get involved Tech Trends | A105 | IRLE | University of California, Berkeley

Trend: 21st Century Reference • Pattern recognition is practiced at the reference desk; MBC proves that it is a high-level skill • “Better” data requires more interpretation and analysis, not less • More machine assistance is a good thing • We need a place at the table, perhaps without invitation “There is a massive space for information professional to analyze data” --John Kelly, Blackstone Discovery Tech Trends | A105 | IRLE | University of California, Berkeley

Some Forecasts • Academic-based digital library developers may take an interest • Vendors might explore MBC as a meta-search tool • Repositories may get a boost • The practice of reference librarianship would benefit from this kind of tool Tech Trends | A105 | IRLE | University of California, Berkeley

Conclusions • We need to be aware of Meaning Based Computing • We should analyze its as-yet-unknown potential for search and discovery within our digital libraries • Social media are growing • Be prepared to make the case for library-based information analysis and counsel Tech Trends | A105 | IRLE | University of California, Berkeley

References ACM Digital Library: http://dl.acm.org Autonomy: http://www.autonomy.com Bayse, Thomas: http://en.wikipedia.org/wiki/Thomas_Bayes BertschMcGrayne, Sharon. The Theory That Would Not Die: How Bayse’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines and Emerged Triumphant from Two Centuries of controversy. Yale, 2011 Blackstone Discovery: http://blackstonedisocvery.com Markoff, John. “Armies of expensive lawyers, replaced by software.” The New York Times, March 4, 2011 Tech Trends | A105 | IRLE | University of California, Berkeley

Track A | Tech Developments & Trends | A105Meaning-Based Computing:New Functionalities from the World of Enterprise Search Terence K. Huwe Director of Library & Information Resources Institute for Research on Labor & Employment University of California, Berkeley thuw@library.berkeley.edu

Terence K. Huwe Director of Library & Information Resources