Computer-assisted essay assessment

1. Computer-assisted essay assessment • Similarity scores by Latent Semantic Analysis • Comparison material based on relevant passages from textbook • Defining threshold values for grade categories • Grading the essays

2. Results

3. Latent Semantic Analysis (LSA)aka Latent Semantic Indexing (LSI) • Several Applications • Information Retrieval • Information Filtering • Essay Assessment • Documents are presented as a matrix in which each row stands for a unique word and each column stands for a text passage (word-by-document matrix) • Truncated singular value decomposition is used to model latent semantic structure • Resulting semantic space is used for retrieval • Can retrieve documents that share no words with query .

4. Latent Semantic Analysis (LSA) • Singular Value Decomposition • Reduces the dimensionality of word-by-document matrix • Using a reduced dimension new relationships between words and contexts are induced when reconstructing a close approximation to the original matrix • Reduces irrelevant data and “noise”

5. Word-by-document matrix Latent Semantic Analysis (LSA)Document comparison • Semantic space is constructed from the training material • To grade an essay, a matrix for the essay document is built • Document vector of essay is compared to the semantic space

6. A B Latent Semantic Analysis (LSA) • Document comparison • Euclidean distance • Dot product • Cosine measure • Cosine between document vectors • Dot product of vector divided by their lengths

7. Latent Semantic Analysis (LSA) • Pros • Doesn’t just match on terms, tries to match on concepts • Cons • Computationally expensive, its not cheap to compute singular values • Choice of dimensionalityis somewhat arbitrary, done by experimentation

8. Latent Semantic Analysis (LSA) • Word-by-document matrix

9. Latent Semantic Analysis (LSA) • Singular value decomposition

10. Latent Semantic Analysis (LSA) • Two dimensional reconstruction of word-by-document matrix

11. Latent Semantic Analysis (LSA)