1 / 18

Visualizing Topic Flow in Students’ Essays

Visualizing Topic Flow in Students’ Essays. Presenter : Wu, Min-Cong Authors : Stephen T. O’Rourke , Rafael A. Calvo and Danielle S. McNamara 2011, EST. Outlines. Motivation Objectives Methodology Experiments Conclusions Comments. Motivation.

arlo
Download Presentation

Visualizing Topic Flow in Students’ Essays

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Visualizing Topic Flow in Students’ Essays Presenter : Wu, Min-Cong Authors : Stephen T. O’Rourke , Rafael A. Calvo and Danielle S. McNamara2011, EST

  2. Outlines • Motivation • Objectives • Methodology • Experiments • Conclusions • Comments

  3. Motivation • Writing is an important learning activity, essays Visualizing is important that can help people assess and improve the quality of essays.

  4. Objectives • This paper presents a novel document visualization technique and a measure of quality based on the average semantic distance between parts of a document.

  5. Methodology-Mathematical Framework In order to Visualization, so need to reduce dimension : Visualizing Topic Flow term-by- paragraphs matrix topic model is created topic model is projected visualization of the document’s paragraphs stop-words low frequency words stemming is applied 2-dimensional space Use NMF Quantifying Topic Flow identify features in the topic model of the document. term-by- sentence matrix topic model is created

  6. Methodology-Visualizing Topic Flow(term-by- paragraphs matrix) j(paragraphs) i(term) Term’s frequency In paragraphs If Log-Entropy is large, this word is more import Term’s Entropy in document

  7. Methodology-Visualizing Topic Flow(NMF dimensionality reduction technique) ≈ Term-by-paragraphs martix (m*n) Term-by-topic martix(m*r) Topic-by-paragraphs martix(r*n) Ex.X(6,2)=w(6,3)*H(3,2) number of latent topics which can be approximated by minimizing the squared error of the Frobenius norm of X−WH.

  8. Methodology-Visualizing Topic Flow(2-dimensional representation) paragraph-paragraph triangular distance table iterative majorization algorithm (least-squares) minimize a loss function (Stress) Multidimensional Scaling use in Similarity comparison between the vector dissimilarities approximated distances in the low dimensional

  9. Methodology-Visualizing Topic Flow(Visualizing Flow ) Paragraphs the diameter of the grid equal to the maximum possible distance between any two paragraphs node-link Next paragraphs conclusion The degree of deviate from a circle introduction High grade, Why? Because: 1. paragraphs appear close, 2. ‘introduction’ and‘conclusion’ is similar Low grade

  10. Methodology-Quantifying Topic Flow Semantic distances between consecutive pairs of sentences or paragraphs Double average over all the pairs of sentences or paragraphs DI <=0, indicates a random topic flow DI> 0, indicates the presence of topic flow.

  11. Experiment-Evaluation 1: Flow and Grades(Experiment Dataset) Dataset:120 essays written for assignments by undergraduate students at Mississippi State University Essay grades :1-6 level Subset:High:67(1-3)Low:53(3.2-6) k(number of topic):5

  12. Experiment-Evaluation 1: Flow and Grades(Measuring Topic Flow ) less present using either of the dimensionality reduction techniques P<0.05 P>0.05

  13. Experiment-Evaluation 1: Flow and Grades(Measuring Topic Flow ) Measure the correlation

  14. Experiment-Evaluation 2: Supporting Assessment(Methodology) 1.inter-rater agreement that the tutors had with two expert raters. 2. The two tutors independently marked assignments with map and no map hypothesized : Essay’s agreement can be subjectively assessed faster, more accurately, and more consistently with map. answer

  15. Experiment-Evaluation 2: Supporting Assessment(Essay Subset Preparation ) The 40 essays remaining were divided into two subsets of 20 essays each according to the MASUS procedure to assess subest1 subest2

  16. Experiment-Evaluation 2: Supporting Assessment(Results) Rater1:native English speaker Rater2: non-native English speaker In order to eliminate the effect of essay length

  17. Conclusions • Tutors assess the essays faster and more accurately and consistently with the aid of topic flow visualization.

  18. Comments • Advantages • effectively discover market intelligence (MI) for supporting decision-makers. • Applications • Document visualizations.

More Related