New Directions in Social Media

1. New Directions in Social Media Tom ReamyChief Knowledge Architect KAPS Group http://www.kapsgroup.com

2. Agenda • Introduction – Social Media and Text Analytics • Deeper than Positive-Negative • Building a Foundation for Social Media • Adding intelligence to BI, CI, and Sentiment • New Dimensions for Social Media • New taxonomies • Cognitive Science of Emotion • Applications • Expertise Analysis, Behavior Prediction, Crowd Sourcing • Integration with Predictive Analytics, Social Analytics • Conclusions

3. KAPS Group: General • Knowledge Architecture Professional Services – Network of Consultants • Partners – SAS, SAP, IBM, FAST, Smart Logic, Concept Searching • Attensity, Clarabridge, Lexalytics, • Strategy– IM & KM - Text Analytics, Social Media, Integration • Services: • Taxonomy/Text Analytics development, consulting, customization • Text Analytics Fast Start – Audit, Evaluation, Pilot • Social Media: Text based applications – design & development • Clients: • Genentech, Novartis, Northwestern Mutual Life, Financial Times, Hyatt, Home Depot, Harvard Business Library, British Parliament, Battelle, Amdocs, FDA, GAO, World Bank, etc. • Applied Theory – Faceted taxonomies, complexity theory, natural categories, emotion taxonomies Presentations, Articles, White Papers – http://www.kapsgroup.com

4. Introduction – Social Media & Text AnalyticsBeyond Simple Sentiment • Beyond Good and Evil (positive and negative) • Social Media is approaching next stage (growing up) • Where is the value? How get better results? • Importance of Context – around positive and negative words • Rhetorical reversals – “I was expecting to love it” • Issues of sarcasm, (“Really Great Product”), slanguage • Granularity of Application • Early Categorization – Politics or Sports • Limited value of Positive and Negative • Degrees of intensity, complexity of emotions and documents • Addition of focus on behaviors – why someone calls a support center – and likely outcomes

5. Introduction – Social Media & Text AnalyticsBeyond Simple Sentiment • Two basic approaches: • Statistical Signature of Bag of Words • Dictionary of positive & negative words • Beware automatic solutions – Accuracy, Depth • Essential – need full categorization and concept extraction to get full value from social media • Categorization - Adds intelligence to all other components – extraction, sentiment, and beyond • Categorization/extraction rules – not just topical or sentiment • Combination with advanced social media analysis • Opens up whole new worlds of applications

6. Text Analytics Foundation for Social MediaText Analytics Features • Noun Phrase Extraction • Catalogs with variants, rule based dynamic • Sentiment Analysis • Objects and phrases – statistics & rules – Positive and Negative • Auto-categorization • Training sets, Terms, Semantic Networks • Rules: Boolean - AND, OR, NOT • Advanced – DIST(#), ORDDIST#, PARAGRAPH, SENTENCE • Text Analytics as Foundation • Disambiguation - Identification of objects, events, context • Build rules based, not simply Bag of Individual Words

7. Case Study – Categorization & Sentiment

8. Case Study – Categorization & Sentiment

10. New Dimensions for Social Media • New Taxonomies – Appraisal • Appraisal Groups – Adjective and modifiers – “not very good” • Four types – Attitude, Orientation, Graduation, Polarity • Supports more subtle distinctions than positive or negative • Emotion taxonomies • Joy, Sadness, Fear, Anger, Surprise, Disgust • New Complex – pride, shame, embarrassment, love, awe • New situational/transient – confusion, concentration, skepticism • Beyond Keywords • Analysis of phrases, multiple contexts – conditionals, oblique • Analysis of conversations – dynamic of exchange, private language

11. New Applications in Social Media • Expertise Analysis • Experts think & write differently – process, chunks • Categorization rules for documents, authors, communities • Applications: • Business & Customer intelligence, Voice of the Customer • Deeper understanding of communities, customers – better models • Security, threat detection – behavior prediction, Are they experts? • Expertise location- Generate automatic expertise characterization • Behavior Prediction–TA and Predictive Analytics, Social Analytics • Crowd Sourcing – technical support to Wiki’s • Political – conservative and liberal minds/texts • Disgust, shame, cooperation, openness

12. New Applications in Social Media • Analysis of Conversations • Techniques: self-revelation, humor, sharing of secrets, establishment of informal agreements, private language • Detect relationships among speakers and changes over time • Strength of social ties, informal hierarchies • Combination with other techniques • Expertise Analysis – plus Influencers • Quality of communication (strength of social ties, extent of private language, amount and nature of epistemic emotions – confusion+) • Experiments - Pronoun Analysis – personality types • Essay Evaluation Software - Apply to expertise characterization • Model levels of chunking, procedure words over content

13. New Applications in Social MediaBehavior Prediction – Telecom Customer Service • Problem – distinguish customers likely to cancel from mere threats • Analyze customer support notes • General issues – creative spelling, second hand reports • Develop categorization rules • First – distinguish cancellation calls – not simple • Second - distinguish cancel what – one line or all • Third – distinguish real threats

14. New Applications in Social MediaBehavior Prediction – Telecom Customer Service • Basic Rule • (START_20, (AND, • (DIST_7,"[cancel]", "[cancel-what-cust]"), • (NOT,(DIST_10, "[cancel]", (OR, "[one-line]", "[restore]", “[if]”))))) • Examples: • customer called to say he will cancell his account if the does not stop receiving a call from the ad agency. • cci and is upset that he has the asl charge and wants it offor her is going to cancel his act • ask about the contract expiration date as she wanted to cxltehacct • Combine sophisticated rules with sentiment statistical training and Predictive Analytics and behavior monitoring

15. New Applications: Wisdom of CrowdsCrowd Sourcing Technical Support • Example – Android User Forum • Develop a taxonomy of products, features, problem areas • Develop Categorization Rules: • “I use the SDK method and it isn't to bad a all. I'll get some pics up later, I am still trying to get the time to update from fresh 1.0 to 1.1.” • Find product & feature – forum structure • Find problem areas in response, nearby text for solution • Automatic – simply expose lists of “solutions” • Search Based application • Human mediated – experts scan and clean up solutions

16. New Directions in Social MediaText Analytics, Text Mining, Semantic Technology • Two Systems of the Brain • Fast, System 1, Immediate patterns (TM) • Slow, System 2, Conceptual, reasoning (TA) • Text Analytics – pre-processing for TM, ST • Discover additional structure in unstructured text • Behavior Prediction – adding depth in individual documents • New variables for Predictive Analytics, Social Media Analytics • New dimensions – 90% of information – converting text to data • Text Mining for TA– Semi-automated taxonomy development • Bottom Up- terms in documents – frequency, date, clustering • Improve speed and quality – semi-automatic

17. New Directions in Social MediaConclusions • Social Media Analysis requires a hybrid approach • Software, text analytics, human judgment • Contexts are essential • Text Analytics needs new techniques and structures • Smaller, more dynamic taxonomies • Focus – verbs, adjectives, broader contexts, activity • Value from Social Media Analysis requires new structures • Appraisal taxonomies, emotion taxonomies Plus • Better models of documents and authors – multi-dimensional • Result: • Enhanced sentiment analysis / social media applications • Develop whole range of new applications • Combine with semantic technology, predictive analytics

18. Questions? Tom Reamytomr@kapsgroup.com KAPS Group http://www.kapsgroup.com Upcoming: Text Analytics World – Boston, October 3-4 SAS A2012 – Las Vegas, Oct 8-9 Taxonomy Boot Camp – Washington DC, Oct 16-17 Gilbane – Boston, November 27-29

19. New Applications in Social Media • Two tasks – emotion detection (distinguish emotion words ), and emotion classification • Get references from this article • Primary and complex emotions • Many states that people call emotion are not in standard taxonomy – ex. Love, confusion, concentration, worry, flirtatiousness, skepticism, indifference • Analyzing political trends – augment opinion polls • Identifying political bias

20. New Applications in Social Media • IDEA – TA needs to grow too – not just nouns, but verbs, adjectives and adverbs, phrases, etc. • Text to Emotion Engine • Uses basic emotion taxonomy – happy, sad, fear, surprise, anger, disgust • Identify conditional sentences – “I’m happy when he is here” • Emotion intensity – adjectives – very, eetc.

21. New Applications in Social Media • “Most expressions of feeling can be more properly situated within a context of exchange,” • People use a variety of techniques to establish intimacy in epistolary exchanges: • Self-revelation, humor, sharing of secrets, establishment of informal agreements, private language • Can be applied to bodies of documents like corporate email exchanges to determine such features: • De facto hierarchy of an organization (Me – add expertise?), strength or weakness of social ties, changing relationships among co-workers over time