Key-Sun Choi and Yeun-Bae Kim* KAIST Korterm, NHK STRL* kschoi@cs.kaist.ac.kr

1. Terminology, Interactive Media and Standards - A Scenario toward Personalized Interactive Knowledge Services - Key-Sun Choi and Yeun-Bae Kim* KAIST Korterm, NHK STRL* kschoi@cs.kaist.ac.kr http://www.korterm.org/

2. Outline • Integrating the multiple number of lexical knowledge base • Question-answering for what-, and why-type question • By causality probing • Integration of Video clipping of answer segments

3. (4) Acquisition (5) Template (5) template : Since it was identified Since it was identified Term Term : BSE : BSE “ “ ” ” term term BSE BSE in the mid in the mid - - 1980s in 1980s in BT BT : disease : disease consists of consists of Britain, mad cow Britain, mad cow symptom symptom : Sponge : Sponge - - characteristics characteristics disease, or BSE, has disease, or BSE, has like change of a like change of a for BT, symptom for BT, symptom resulted in the resulted in the (1) Query brain part brain part and and casuse casuse . . slaughter of millions of slaughter of millions of cause cause : infectious : infectious - - cattle cattle -- -- and the deaths and the deaths prion prion (6) Integration of dozens of people of dozens of people synonym synonym : mad cow disease : mad cow (3) Wrapping BSE BSE ? ? from the related brain from the related brain - - (3) wasting disease known wasting disease known dis (7) Representation (7) representation (8) Presentation eas as... as... (8) e (2) Unit of Database no item no item BT (2) unit of BSE mad dis dis mad in dictionary when in dictionary when synony cow eas eas cow related the first BSE the first BSE m dise diseas e e (9)Definition for Semi-expert BT BT (9) definition for semi - expert e BSE BSE PO ase mad mad mad mad S cause nou synony synony cow cow infectiou cow cow n … … a fatal disease of cattle a fatal disease of cattle sympto s related related m m dise dise diseas diseas Prion affecting the nervous system, affecting the nervous system, (9’) Definition for Child m e e (9 ’ ) definition for child nervou PO PO ase ase S S cause cause nou nou resembling or identical with resembling or identical with s .. infectiou infectiou n n sympto sympto s s a brain disease of cows that a brain disease of cows that scrapie scrapie of sheep and goats, of sheep and goats, Prion Prion m m causes death, and some causes death, and some nervou nervou and probably caused by a and probably caused by a s .. s .. people who eat them people who eat them … … prion prion transmitted by infected transmitted by infected tissue tissue -- -- an infectious protein an infectious protein particle similar to particle similar to virus virus … … Steps to Give Appropriate Answering to EachCustomer Who Asks about BSE

4. (b1) cow disease relates to human disease knowledge space (a2) (b) hypotheses world space (a1) (a3) justification Cow disease database (b3) human disease causes cow disease (b2) cow disease causeshuman disease D1 D2 (b4) mad cow disease causes human brain disease (d1) justificationinstance (c1)ontology for disease diseasefood Human disease database disease justificationcause(d2) D3 diseasemeat diseasevegetable D4 human disease cow disease diseasebeef human brain disease BSE edible(d2) diseasecow diseasehuman Referent mapping (a4) diseaseanimal diseaseplant infectedReferent mapping(d2) Diseaseliving thing (c3) ontology for living thing Justification process goes from the world space to the knowledge space

5. universal hypothesis hypothesis H1: relate(cow disease,human disease) specification opposition justification Does mad cow disease cause human brain disease?an enlarged information space Knowledgespace … generalization H2: cause(mad cow disease,human brain disease) specification H3: cause(mad cow disease,human brain and neural disease) H4: cause(mad cow disease,human brain and non-neural disease) … referent absurd hypothesis Worldspace article on H2 media on H1 media on H3 … article on H4

6. Construction of knowledge space • How to construct • the knowledge space and • its linkage to the associated justification world space • By • Experienced humans or • (Semi-)automatic machines • Knowledge Discovery from Text

7. Tell me whether a mad cow disease will cause a human brain disease. • Where is relevant resources in response to a given knowledge request? • How to meta-data catalogs by categorizing the information content in each repository • Information Seeking Problem

8. Does the “mad cow disease” cause the “human brain disease”? • What is the correlation between “mad cow disease” and “human disease”? • Where is the related repository? • the human disease repository, and • The mad cow disease repository. • Knowledge Seeking Problem

9. Tell me whether a mad cow disease will cause a human brain disease. • How to correlate concepts • Find possible relationships between animal disease and human disease • Link a animal disease record with the corresponding a human disease report. • What is shared ontology? • Disease, virus, … • What is related contextual ontology? • Food chain, time period • How to standardize that? • ISO/TC37 • MPEG7

10. Ontology understand Cold pneumonia feel OBJ High fever SUBJ cause symptom remedy suffer ADV lip appear SUBJ blister concrete/abstract concrete abstract object work<abstract> animate Human activity Natural phenomena inanimate action sense physiology condition/abnormality animal bacteria blood/secretion/waste matter imperfect drug life disease blood secretion Waste matter drink/eat Physical disability injury rest nutrition Health care Knowledge Seeking Model Knowledge Seeking Would you please tell me an effective remedy? I feel cold and suffer a high fever. What is the cause of SARS? The new corona virus is the leading hypothesis for the cause of SARS . The primary way that SARS appears to spread is by close person-to-person contact. No specific treatment recommendations can be made at this time. Empiric therapy should include coverage for organisms associated with any community-acquired pneumonia of unclear etiology, ….. When you have a pneumonia, you should be rest and take an antifebrile if you have a high fever ….

11. Knowledge Seeking • Necessity of Knowledge Seeking • Questions requiring -- "why", "what" , "how“ type questions • Problems of Knowledge Seeking • Justification Probing on knowledge • How to utilize various Knowledge resources • Goal • Building an algorithm for • searching some topical paths • in order to find causal explanations for questions • E.g., “Why do patients pay money to doctors?”

12. Knowledge Representation • Two types • Tree（Hierarchical） • Graph

13. Knowledge Representation • Implication • doctor  human, *cure, #occupation, medical • Instantiation • *cure  {doctor, medical worker, …} • Event Relation • cause(cure,suffer-from) • cure.patient = suffer-from.experiencer • cure.content = cure.content • Converse/Antonymy • Converse(take,give)

14. dictionary Concept facets 人間 agent=possession=target= agent=possession=source= 支払い converse pay give take entity human occupation occupation doctor money #occupation #occupation why *cure *cure earn $earn $earn role event patient cause Alter-possession question give give take cure pay earn Knowledge Structure • Knowledge structure : Multi-facet Structure • Knowledge space for each knowledge feature • Knowledge feature : e.g. isa(x,y)

15. Justification by Knowledge Seeking (1) Doctor cures the patient and earn the money as occupation (2) ‘Pay’ is ‘give’. Converse of ‘give’ is ‘take’. ‘Patient’ is the agent (*) of ‘give’ and the source ($) of ‘take’.  ‘Take’ is the hypernym of ‘earn’. “The patient pay X to the doctor.” is “The doctor earn X.” (3) The hypernym of ‘pay’ is ‘take’. From “pay money”, the possession of ‘take’ is ‘money’. entity (3) occupation patient doctor money (1) #occupation $cure *cure earn $earn *pay $pay (2) converse give take agent=patientpossession=target= agent=possession=source=patient doctor converse crossing money money doctor alter-position event

16. “doctor cures patient” T1 T1 Video Material Attaching description segmentation parsing “doctor earns” T2 T2 T3 T3 descriptivecomponents Script-type index SDS-type index doctor cure patient earn subject (from T1 to T2) patient cure object doctor (from T2 to T3) subject earn T1 T2 time Video indexing scheme based on natural language content description T3

17. Gaps in possible syllogism • Syllogism • If pay(human2,money,to-human1) is earn(human1,money,from-human2), and earn(doctor,money,from-human2), • Then pay(human2,money,to-doctor). • Axiom-1 • Converse (antonymy) event role inter-relation • pay(agent=human2,content=X,target=human1) iff earn(agent=human1,content=X, source=human2) • Axiom-2 • If cure(doctor, human2) and occupation(doctor), then earn(doctor,money,source=human2) • Extended syllogism • If Axiom-2 and cure(doctor,human2) and occupation(doctor), then earn(doctor,money,from-human2).

18. Gaps in Linguistic Knowledge Base • Query: • Why doctor cure patient? • Why doctor earn money? • Why patient pay money to doctor? • Extended syllogism • Axiom-2 • If cure(doctor, human2) and occupation(doctor), then earn(doctor,money,source=human2) • If Axiom-2 and cure(doctor,human2) and occupation(doctor), then earn(doctor,money,from-human2). • Gaps in Linguistic Knowledge Base • No axiom • Motivation of Crossover algorithm: causal linking • to link the gaps in linguistic knowledge base • to find possible axioms

19. Motivation of Crossover algorithm • Motivation of Crossover algorithm: causal linking • to link the gaps in linguistic knowledge base • to find possible axioms

20. Similarity: mission • Mission • for two nodes, • check the connectability between two nodes • if sense ambiguities for a word, • select the best sense. • if there are multiple expansion possibilities in the next step, • select the best node. • Final goal • to find a causal linkage

21. Similarity: motivated example • For query: “Why does doctor cures patient?” • similarity guides to find all kinds of possible situation between doctor and patient before/during/after “doctor cures patient”. • for example, • Patient suffers from a disease. • Doctor cures the patient. • Doctor is an occupation. • Occupation is to earn money for living. • All properties of doctor (e.g., cure) is relevant to the occupation. • Curing is to earn the money. • Doctor earns the money. • Patient pays the money to the doctor.

22. How to find the relevance from linguistic knowledge base incl. role relation for: patient ~ disease suffer from ~ cure earn ~ cure pay ~ patient patient ~ disease role relation suffer from ~ cure cause interrelation earn ~ cure subject of event relevance of occupation pay ~ patient converse event relation pay ~ earn Similarity for causal connectability

23. patient ~ disease role relation suffer from ~ cure cause interrelation earn ~ cure subject of event relevance of occupation pay ~ patient path up and hypernym’s general properties’ inheritance pay < act patient < human < *act patient ~ disease to find the same event feature. patient  human $cure  *sufferFrom disease  medical  $cure  undesired suffer from ~ cure to expand thru event interrelation. sufferFrom(exp=X,cont=Y) causes cure(agent=A,patient=X,cont=Y) earn ~ cure *earn ~ *cure, *earn ~ $cure to compare of role entities of events. doctor  *cure  #occupation *cure  ~#occupation if they are inside of the definition of ‘doctor’. occupation  affairs  earn  alive pay ~ patient pay << act (hypernym) human  *act patient << human path-up, to find role entity of event, path-down Similarity Measure

24. feature similar similar(1) expand similar search “rolerelation” for event interrelation inverse and sister similar expand inverse similar similar if their role entities share these events. (expanded version) similar if their inverse concepts share these two concepts. expand sister and feature similar If they are in sisters, they are similar, and plus if the sister (e.g., occupation) contains it as a feature. (crossover(1)) Algorithm: path-up + inverse + path-down path similar Similarity between two hypernyms patient ~ disease to find the same event feature. patient  human $cure  *sufferFrom disease  medica  $cure  undesired suffer from ~ cure to expand thru event interrelation. sufferFrom(exp=X,cont=Y) causes cure(agent=A,patient=X,cont=Y) earn ~ cure *earn ~ *cure, *earn ~ $cure to compare of role entities of events. doctor  *cure  #occupation *cure  ~#occupation if they are inside of the definition of ‘doctor’. occupation  affairs  earn  alive pay ~ patient pay << act (hypernym) human  *act patient << human path-up, to find role entity of event, path-down Rationale of Similar, but exhausted list must be checked. Link

25. similar(Level) • Goal • to merge all of similarity measures • path similar • feature similar • crossover similar (not yet fixed) • to give Level of Similarity • Notation: similar(Level)

26. Level of Similarity • Motivation: • to be able to compare the similarity levels • to be dynamically adaptable to the knowledge levels • Levels of details • Level 0: based on itself and its top path • = path similar • Level 1: to expand to the features • =feature similar • Level 2:

27. similar level 0 medicine*disease&medical# cure>cause>sufferFrom {patient=experiencer,content=content}cure>possible consequence>beRecovered {patient=experiencer} why doctor cure patient human#occupation*curemedical agentpatientcontentmedical human*sufferFrom$cure

28. payer*money advanced$ give>consequence>lose {agent=possessor,possession=possession}give<implication<receive {possessor=target,possession=possession} converse give take hypernym hypernym why patient pay money human*sufferFrom$cure agentcontentsource commercial$earn*buy#sell$setAside doctor human#occupation*curemedical occupation affirsearn Virtual Knowledge Base: causal linking

29. A Snapshot

30. Experimentation Q1: Why do patients pay money to doctors? Path: patient → $cure → doctor → #occupation ← $earn ← money Interpretation: Patients cured by doctor. The doctor is related to the occupation and the occupation is that to earn the money. So patients pay money to doctors. Connected concepts Q2: Why does researcher read textbook? Path: researcher → #knowledge → #information ← readings ← textbook Interpretation: Researcher is related to the knowledge and the knowledge is related to the information. Textbook is the object of readings and readings is related to information. So researcher read textbook.

31. Future Works • Resources • EDR, WordNet, Cyc • HowNet based • Algorithm and Representation • Lexical Chain • Interpretation as Abduction • Bayesian Belief Net • Causality • Stopping Condition • What is “Causality” and “Explanation”? • Automatic Video Synchronization with Causal Justification Path

32. Conclusion • How to link the already existing linguistic knowledge base • To be tested for the definition and the causality link. • To be adapted for the user knowledge level • to find more causality link. • How to link the video archive to linguistic causal path • Dependency structure • MPEG7