1 / 21

How Useful are Natural Language Interfaces to the Semantic Web for Casual End-users?

How Useful are Natural Language Interfaces to the Semantic Web for Casual End-users?. Esther Kaufmann and Abraham Bernstein Presented By Stephen Lynn. Overview. Natural Language Interfaces Goals/Objectives Introduce 4 Interfaces Experiment Evaluation Results Future Work.

tameka
Download Presentation

How Useful are Natural Language Interfaces to the Semantic Web for Casual End-users?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. How Useful are Natural Language Interfaces tothe Semantic Web for Casual End-users? Esther Kaufmann and Abraham Bernstein Presented By Stephen Lynn

  2. Overview • Natural Language Interfaces • Goals/Objectives • Introduce 4 Interfaces • Experiment • Evaluation Results • Future Work

  3. Natural Language Interfaces • Plain text queries • Phrases • Full Sentences • Challenges • Linguistic Variability (ambiguous meaning) • Domain Independence • Retrieval Performance (linked to portability) • Usefulness of NLIs

  4. Goals/Objectives Usability of NLIs Usefulness of NLIs

  5. Evaluation Interfaces • Portable • Domain-Independent • Good Performance • 4 Interfaces • Least to Most Restrictive

  6. NLP-Reduce • Free-form text query • Remove Stop Words/Puncuation • Word Stemming • Identify Triple Structures (no details) • Enhanced Triple Store (WordNet) • Generate SPARQL • Return Results

  7. NLP-Reduce

  8. Querix • Parse Query • Extract Query Skeleton from Syntax Tree • Identifies Triple Patterns • Match Triples to Knowledge Base Resources • Generate SPARQL • Enhanced with WordNet Synonyms • Return Results

  9. Querix

  10. Querix – Ambiguity Resolution • What is the biggest state in the US?

  11. Ginseng • UI based on a grammar • Built dynamically from target knowledgebases • Incremental Parser • Offer possible completions (code completion) • Only accepts entries in list • No invalid queries • Convert to SPARQL • Return Results

  12. Ginseng

  13. Symantic Crystal • Graphical Display of Ontology • Select Elements in Ontology • No Invalid Queries • Specify Constraints • Incrementally Build Query • Generate SPARQL • Return Results

  14. Semantic Crystal

  15. Usability Study • How usable and useful are NLI applications? • Setup • 48 subjects • 4 interfaces • Same 4 questions for each interface (minor changes) • Area of Alaska? • Number of lakes in Florida? • States that have city named Springfield? • Rivers run through state that has largest city in US? • Change sequence of interfaces

  16. Experiment • Read Introduction Notes • Instructions on Interface #1 • Answer 4 questions with interface • Fill out Usability survey about Interface • Repeat 2-4 for other Interfaces • Fill out Comparison Questionnaire

  17. Evaluation Results

  18. Evaluation Results

  19. Strengths • Good General Points • Automation is good (not Sematic Crystal) • Result format affects user trust • Balance between freedom and restriction • User Evaluation • Analysis

  20. Weaknesses • Completion time not a deciding factor in satisfaction • Still pushing Semantic Crystal • Personal Attachment • Unclear distinction between QL and Interface

  21. Future Work • Compare with more NLIs • Multiple Domains • Single Infrastructure w/Different Uis • Evaluate Usability/Usefulness

More Related