Using Version Spaces to Learn Facet Evidence in INFACT

1. Using Version Spaces to Learn Facet Evidence in INFACT PI: Steve Tanimoto Dave Akers Nick Benson Adam Carlson

2. INFACT Features • Threaded discussion groups • Manageable group membership • Visibility control • Text and Sketching Learning Facet Assessment Rules for INFACT

3. Without INFACT Read student work Circle text Write in margins With INFACT Read student posts Highlight text Add annotation Linked to Facet DB “Canned” comments Shows student progress Annotation in INFACT Learning Facet Assessment Rules for INFACT

4. INFACT Markup Tool Example Learning Facet Assessment Rules for INFACT

5. The Many Faces of INFACT • A Tool for Teachers and Students • Platform for student discussions • Learn more about student (mis)understanding • A Curriculum Developer’s Tool • Create discussion-oriented curriculum content • An Education/Software Researcher’s Tool • Identify Facets • Test teaching methodology • Test educational software ideas Learning Facet Assessment Rules for INFACT

6. Markup Rules • Target Audience:Educational Researchers and Curriculum Developers • Package rules with content to automate markup • Rules can inform teachers or possibly fire off interventions bowling <0 ball < falls < faster < tennis <0 ball Learning Facet Assessment Rules for INFACT

7. Outline • Overview of INFACT/Markup tool • Version Space Learning • Application of version spaces to facet assessment • Issues with learning facet assessments Learning Facet Assessment Rules for INFACT

8. Version Space Learning Algorithm • Maintain all hypotheses consistent with data • Hypotheses are organized into a generality lattice • Use General and Specific Boundary Sets to efficiently represent all hypotheses Learning Facet Assessment Rules for INFACT

9. Concept Description Language • AKA Hypothesis language • Example: conjunction of binary features3 featuresRed/YellowCircle/TriangleHollow/Filled H G T F G ? F R C Learning Facet Assessment Rules for INFACT

10. Generality Lattice • Partial ordering over hypotheses • Organized by generality U ?R? ?G? ??T ??C H?? F?? ?RC ?RT ?GC ?GT HR? FR? HG? FG? H?C H?T F?C F?T HRC HRT HGC HGT FRC FRT FGC FGT Ø Most general Least general Learning Facet Assessment Rules for INFACT

11. Version Space Algorithm • Positive example: Generalize specific boundary set to cover example • Negative example: Specialize general boundary set to exclude example • If boundary sets ever cross VS collapses – examples are inconsistent Learning Facet Assessment Rules for INFACT

12. Example of VS Algorithm 1 • Example: FGT + Concept: ?GT U … … FGT Ø Learning Facet Assessment Rules for INFACT

13. Example of VS Algorithm 2 • Example: HGC - Concept: ?GT U F?? ?R? ??T ... FGT Ø Learning Facet Assessment Rules for INFACT

14. Example of VS Algorithm 3 • Example: HGT + Concept: ?GT U F?? ??T ?GT FGT Ø ... Learning Facet Assessment Rules for INFACT

15. Outline • Overview of INFACT/Markup tool • Version Space Learning • Application of version spaces to facet assessment • Issues with learning facet assessments Learning Facet Assessment Rules for INFACT

16. Learning Facet Assessment Rules • Given • “The big block will fall faster” • “Of the two blocks the faster falling one will be the bigger one” • Learn • block < faster ^ fall ^ big • Must specify • Rule language • Generalization hierarchy Learning Facet Assessment Rules for INFACT

17. Facet Assessment Rule Language • Terms / Term Abstractions • Big, Block, (Block/Ball/Anvil) • Ordering • Optional Distance constraint • Fall <3 Faster • Conjunction • (Big < Block) ^ (Fall <3 Faster) ^ Gravity Learning Facet Assessment Rules for INFACT

18. Generalization Hierarchy • Most General • Most Specific U TermAbstraction Term1 Term1 ^ Term2 Term1 < Term2 Term1 <0 Term2 Learning Facet Assessment Rules for INFACT

19. INFACT Rule Learner Examples • Facet • Understands formula for reversing an image • Evidence • “at the left side, the original mona lisa but on the right we would see the mona lisa flipped backwards” • “flipped backwards” • Rule • flip <0 backward Learning Facet Assessment Rules for INFACT

20. More examples • Facet • Brightness increases with number • Evidence • “to scan the lightest looking areas on the picture and look for the highest value of R + G + B value.” • “The lightest is the highest RGB” • Rule • lightest <9 highest Learning Facet Assessment Rules for INFACT

21. Yet more examples • Examples • “we would see at the left side, the original mona lisa but on the right we would see the mona lisa flipped backwards.” • “I think this will show a picture of two Mona Lisa's horizontally next to each other. The Mona Lisa on the right will be inverted horizontally, so the face will be pointed a different way than the second one” • Rule • mona <0 lisa • Want: flipped < backwards OR inverted < horizontally Learning Facet Assessment Rules for INFACT

22. Issues with Infact Rule Learner • Common terms and order dependence • Common terms tend to dominate • Results depend on order examples are seen in • Disjunctive concepts • Not represented explicitly • Hard to learn red or green vs. triangle or square Learning Facet Assessment Rules for INFACT

23. Issues with Infact Rule Learner • Common terms and order dependence • Disjunctive concepts • Solution: Use negative examples • Sources for negative examples • How to update version space given infinite rule language red or green vs. triangle or square Learning Facet Assessment Rules for INFACT

24. Future Work • How to incorporate domain-specific knowledge • Recognize formulae • Color specifications (e.g. R: 255 G: 128 B: 0) • Recognize (pixelmath) programs • Can we generalize over these things too? Learning Facet Assessment Rules for INFACT

25. More Future Work • INFACT allows students to draw sketches in addition to making textual posts • Learn rules for facet evidence in sketches • Primitives: circle, rectangle, textbox, etc. • Spatial predicates: left/right, above/below, in • e.g. circle left of triangle above text(“before”) AND triangle left of circle above text(“after”) Learning Facet Assessment Rules for INFACT