460 likes | 622 Views
Ontology-based Open-Corpus Personalization for e -Learning. Sergey Sosnovsky Committee : Peter Brusilovsky Darina Dicheva Daqing He Chad Lane Michael Spring. Definition: Open-corpus vs. Closed-corpus.
E N D
Ontology-based Open-Corpus Personalization for e-Learning Sergey Sosnovsky Committee: Peter Brusilovsky DarinaDicheva Daqing He Chad Lane Michael Spring School of Information Sciences, University of Pittsburgh
Definition: Open-corpus vs. Closed-corpus • Definition 1 (Closed-corpus adaptive system):“…operates on a closed corpus of documents, where documents, relations between the documents, and relations between the documents and the knowledge models of the system are defined at design time” • Definition 2 (Open-corpus adaptive system):“…operates on a open corpus of documents, where documents, relations between the documents, and relations between the documents and the knowledge models of the system are not known at design time and, moreover, can constantly change and expand” School of Information Sciences, University of Pittsburgh
A Closed-Corpus Adaptive System School of Information Sciences, University of Pittsburgh
A Closed-Corpus Adaptive System in the Open-Corpus Settings Missing link School of Information Sciences, University of Pittsburgh
Open-corpus Personalization: Step by Step • discovery of open-corpus content • identification of individual learning objects in open-corpus content • modeling of open-corpus content • connection of open-corpus learning objects to each other and to the preexisting content • providing adaptive access to open-corpus content • tracing of users’ interaction with the open-corpus content • user modeling based on interactions with the open-corpus content School of Information Sciences, University of Pittsburgh
The Proposed Approach Compared to the Existing Solutions School of Information Sciences, University of Pittsburgh
Structured e-Learning Content on WWW Textbooks Encyclopedias • High-quality, domain-oriented • Their content has been chosen, structured and formatted for the purpose of explaining the domain knowledge by the experts in the domain • The structure of the domain knowledge have been used as a mean for organizing these resources • If extracted, this structure represents the model of the resource and the model of the domain as the author understands it Tutorials Dictionaries Digital Libraries Links between Pages Headers of Sections Table of Contents Content Formatting School of Information Sciences, University of Pittsburgh
Structured content on WWW School of Information Sciences, University of Pittsburgh
Structured content on WWW School of Information Sciences, University of Pittsburgh
Structured content on WWW School of Information Sciences, University of Pittsburgh
Structured content on WWW School of Information Sciences, University of Pittsburgh
Proposed Approach • Extraction of a topic-based model from an open-corpus resource • Mapping a topic-based model into the central reference ontology • Maintaining open-corpus personalization School of Information Sciences, University of Pittsburgh
Step1: Extraction of Open-Corpus Models School of Information Sciences, University of Pittsburgh
Step 2: Mapping Topic-based Models into a Reference Ontology School of Information Sciences, University of Pittsburgh
Step 3: Open-Corpus Personalization School of Information Sciences, University of Pittsburgh
Ontology-based Open-corpus Personalization Service • Recommends supplementary reading as a list of links to the topics’ content • Works as a value-added service foran exercise system (QuizJET) • Implements two adaptation technologies: • Context-based adaptive link recommendations/ordering • Knowledge-based adaptive link annotation • Domain: Java Programming School of Information Sciences, University of Pittsburgh
OOPS Interface: Recommendation Phase A QuizJET Exercise List of open-corpus topics recommended and annotated by OOPS School of Information Sciences, University of Pittsburgh
OOPS Interface: Reading Phase Feedback/exitbuttons Navigation links to the next and the previous topics content of the chosen topic School of Information Sciences, University of Pittsburgh
Personalization in OOPS School of Information Sciences, University of Pittsburgh
Evaluation: Basic Info • Subjects: • 40 undergraduate and graduate Pitt students with little experience in Java programming • Material: • Two sets of introductory Java topics: • Easy: objects, classes, conditionals • Difficult: loops, arrays, ArrayLists • Task: • “Solve QuizJet exercises, use supplementary reading, when necessary” School of Information Sciences, University of Pittsburgh
Evaluation: Schedule Easy topics Difficult topics (17 exercises) (16 exercises) School of Information Sciences, University of Pittsburgh
Evaluation: Three Versions of the System Adaptive recommendation of Open-corpus content The original textbook instead of recommendation Adaptive recommendation of Closed-corpus content--------------------------------------Based on manual indexing of the textbook topics with the concepts of the ontology School of Information Sciences, University of Pittsburgh
Evaluation: Experiment Design 40 students School of Information Sciences, University of Pittsburgh
Evaluation: Data Collection • Pre-test & Post-test => • Questionnaire • Transaction logs from QuizJET • Transaction logs from OOPS • A topic recommended • A recommendation accepted • Browsing occurred • A recommendation exited (by using which button) School of Information Sciences, University of Pittsburgh
General Learning Effect • H1: Scrorepre-test< Scorepost-test across all groups and conditions p = 0.089 School of Information Sciences, University of Pittsburgh
Effect on Learning Difficult Material • H2: KGopen-corpus = KGclosed-corpus for easy material • H3: KGopen-corpus = KGclosed-corpus for difficult material • H4: KGopen-corpus > KGtextbook for easy material • H5: KGopen-corpus >Kgtextbook for difficult material p = 0.043 School of Information Sciences, University of Pittsburgh
Effect on Learning Conceptual Material ≈ • H6,7: KGopen-corpus > KGtextbook for conceptual learning material There are two values of type boolean: ______________ and ______________ • What is the output of the following code segment? • inta = 3 + 3; • int b = 2 + 2; • if (a != b) System.out.println(“ Not equal ”); • if (a == b) System.out.println(“ Equal ”); p = 0.023 p = 0.089 School of Information Sciences, University of Pittsburgh
Recommendation Acceptance Rate • H8.9: RecAccRateopen-corpus = RecAccRateclosed-corpus School of Information Sciences, University of Pittsburgh
Perceived Value of Recommendation • H12,13: PersRecValueopen-corpus = PersRecValueclosed-corpus School of Information Sciences, University of Pittsburgh
Questionnaire: Open-Corpus vs. Closed-Corpus Recommendation • I benefited from the materials recommended by the OOPS service • The recommendations of the OOPS service did not help me to solve the questions • Most often, when I clicked on a recommended link, I had to browse to find a helpful page • Most often, I was not able to find a helpful page even by browsing • Most often, I did not have to browse, one of the recommended pages was helpful for the question • Most often, the helpful page was the first or second in the recommended list Open-corpus Closed-corpus School of Information Sciences, University of Pittsburgh
Questionnaire: Open-Corpus vs. Textbook • The usual textbook was more helpful for solving questions than the recommendations of the OOPS service • It was easier to find a helpful page among the recommendations of the OOPS service than in a textbook • OOPS recommendations were better only because they were accessible within the same system, which was easier to use than the book • OOPS recommendations were better because OOPS brought the helpful pages much closer, and I did not have to look for them in the entire book Open-corpus School of Information Sciences, University of Pittsburgh
Contributions of this dissertation • A fully-automated approach towards open-corpus personalization and semantic open-corpus content modeling for e-Learning • An adaptive e-Learning service implementing the approach • A new algorithm for mapping ontologies and topic models • Evaluation experiment demonstrating: • several significant learning effects of the developed service compared to a regular textbook • no significant difference between the open-corpus and the closed-corpus recommendation from point of learning • quality of open-corpus recommendation comparable to the closed-corpus one (based on both transactional logs and subjective evaluation) • subjects strongly prefer OOPS over a textbook for the “right” reason School of Information Sciences, University of Pittsburgh
Future Work Directions • Further evaluation of the proposed approach: • A larger-scale control study • Evaluation in a course settings • Evaluation with multiple collections of open-corpus content • Further Development of the service: • Implementation in other domains • Model extraction component • Mapping algorithm • Cross-mapping with multiple collections • Use the social feedback to refine the recommendation School of Information Sciences, University of Pittsburgh
Acknowledgements School of Information Sciences, University of Pittsburgh
Thank you Questions? School of Information Sciences, University of Pittsburgh
Characteristics of the Approach • it supports a full cycle of open-corpus personalization • it allows for creation of semantic models of open-corpus content • it requires little to no involvement of a human author • it is domain-independent • it is compatible with a wide range of adaptation technologies • it is potentially applicable in a wide range of application fields beyond e-Learning School of Information Sciences, University of Pittsburgh
OOPS: Architecture School of Information Sciences, University of Pittsburgh
Modeling open-corpus content School of Information Sciences, University of Pittsburgh
Knowledge-based Adaptive Navigation School of Information Sciences, University of Pittsburgh
Indexing of QuizJET questions School of Information Sciences, University of Pittsburgh
Recommendation statistics School of Information Sciences, University of Pittsburgh
Exercises statistics School of Information Sciences, University of Pittsburgh
Questionnaire: General Questions + Questions about Adaptive Annotation • The adaptive annotations of the recommended links helped me to choose appropriate recommendations • I did not use the adaptive annotations of the recommended when choosing the recommendations • I think the idea of recommending reading material to help solve the question is good, but implementation could be better Open-corpus • I like the interface of the OOPS service • I tried to use the help of the OOPS service School of Information Sciences, University of Pittsburgh
Algorithm for Topics to Ontology Mapping (ATOM) • Input: the central ontology and a topic-based model • Output: matrix of similarities between concepts and topics School of Information Sciences, University of Pittsburgh
Average Accepted Recommendation Rank • H10,11: AvgAccRecRankopen-corpus = AvgAccRecRankclosed-corpus School of Information Sciences, University of Pittsburgh