1 / 33

EPSY 625 LECTURE 3

EPSY 625 LECTURE 3. COGNITIVE ASSESSMENT. AFFECT. TASK DEMANDS: STRUCTURING COGNITIVE TESTS. TYPES ARTIFICIAL ANALOG ACTUAL TESTS: 1. ACHIEVEMENT 2. INTELLIGENCE. TASK TYPES. ARTIFICIAL- intended to assess response to novel conditions not encountered before

marshall-garrett
Download Presentation

EPSY 625 LECTURE 3

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EPSY 625LECTURE 3 COGNITIVE ASSESSMENT

  2. AFFECT

  3. TASK DEMANDS:STRUCTURING COGNITIVE TESTS • TYPES • ARTIFICIAL • ANALOG • ACTUAL • TESTS: • 1. ACHIEVEMENT • 2. INTELLIGENCE

  4. TASK TYPES • ARTIFICIAL- intended to assess response to novel conditions not encountered before • ANALOG- intended to assess response to conditions not ethically or economically establishable • ACTUAL- intended to assess response in “real” setting

  5. I. ACHIEVEMENT TESTS • A. Curriculum Guides/ Mandates: TEXAS ESSENTIAL KNOWLEDGE AND SKILLS (TEKS) TAKS • B. Textbook Surveys • C. Political/Philosophical Selection (e.g.. Hirsch, “CULTURAL LITERACY) • D. Myth/Nostalgic/History “all _______ should know this

  6. ACHIEVEMENT TESTS • D. Myth/Nostalgic/History “all _______ should know this” • E. Issue: content/pedagogical validity • F. Content selection • Random/Ordered • Importance: how determined? • Taxonomies

  7. TAXONOMIES • a) Bloom et al. • Evaluation • Synthesis • Analysis • Application • Comprehension • Knowledge

  8. b) Table of specifications • Topics Taxonomy • K C Ap An • A 6 9 9 6 30% • B 10 15 15 10 50% • C 4 6 6 4 20% • 20% 30% 30% 20% 100%

  9. SAMPLING OF BEHAVIORS • All tests sample a Universe, defined by a combination of all possible tasks, occasions, raters, and measurement methods • Domain refers to a content area to which the tasks refer Sampling Variability of Performance Assessments Richard J. Shavelson, Gail P. Baxter, and Xiaohong Gao Journal of Educational Measurement, Fall 1993, Vol. 30, No. 3, pp. 215-232

  10. COGNITIVE RESEARCH • 1. Memory effects - STM/LTM or Level: working-long term • 2. Processing effects - spatial • - analogical • - reasoning • - integrative/simultaneous

  11. COGNITIVE RESEARCH • 3. hot cognition/affect/ motivation - Paivio’s “dual coding” theory • 4. Task structure - VISUAL/SPATIAL - SEMANTIC/VERBAL - PROCEDURAL/ORDER

  12. COGNITIVE RESEARCH 5. Knowledge structure - Declarative (what) - Procedural (how-strategy) - Conditional (when-strategy)

  13. CONTENT SELECTIONIMPLICATIONS • Situational nature of performance • Complexity in development • Limitations in generalizability

  14. II. INTELLIGENCE TESTS • A. THEORY BASIS: 1. “g” Construct- single factor 2. Limited # (2 or 3 factors) 3. Multiple intelligences 4. Limited scope for assessment (school)

  15. 1. “g” FACTOR • BINET-TERMAN: children’s mental functioning: STANFORD-BINET IQ • British psychology: • SPEARMAN: factor analysis • RAVEN: Progressive Matrices

  16. 2. Limited # of factors • WECHSLER and adults: Verbal and non-verbal IQ: • WAIS • WAIS-R • Developmental downward extension: • WISC • WISC-R • WISC-III

  17. 2. Limited # of factors • KAUFMAN & KAUFMAN: KAB-C: • children’s IQ • simultaneous & sequential (from Luria’s cognitive theory)

  18. 3. MULTIPLE INTELLIGENCES • MERCER’S SOMPA (System of Multicultural Pluralistic Assessment) • GARDNER’s Multiple Intelligences • STERNBERG’s Analogical Reasoning Theory • Subtest use of SB, WISC-III, and KAB-C

  19. 4. LIMITED SCOPE • School intelligence (Publisher developed): • Differential Aptitude Tests • Scholastic Achievement Tests: SAT, GRE, GMAT • COGAT (Cognitive Abilities Test), etc.

  20. B. SAMPLING OF PROCESSES • SPECIFIC ABILITIES/PROCESSES: • ANALOGICAL REASONING • SPATIAL ABILITY • MEMORY • NONVERBAL: BLOCK DESIGN

  21. TYPES OF RESPONSE • I. SUPPLY • II. SELECT

  22. I. SUPPLY RESPONSES • A. WRITTEN (ESSAY, SHORT ANSWER) • B. ORAL • C. DRAWING/SKETCHING • D. COMPUTATION • E. PERFORMANCE

  23. A. WRITTEN- ESSAY • Extended response allows greater sampling of knowledge domain • Dependent on writing (computer vs. pen) speed, legibility, strategic knowledge • Greater time to score- need for rubric, fatigue in scoring • Need to constrain topic, task

  24. A. WRITTEN- SHORT ANSWER • Restrict topics: • Definitions or concepts • Quick computations • Limit response length • Establish protocol for scoring • Establish scoring system

  25. B. ORAL RESPONSE • Historical precedence (Greek, Roman, European Middle Ages, University system) • Performance aspect: knowledge and personal interaction • Typically faster response required- “thinking on one’s feet”

  26. C. SKETCHING/DRAWING • Less commonly required • Often task-specific (e.g.. Knowledge maps) • Incorporated into broader assessments or tasks (e.g.. Part of physics or math problem)

  27. D. COMPUTATION • Most common to mathematics and science fields • Mental or written requirement • Use of calculators or computers • Verbal component may be important (word problems) • Spatial component may be important (imageability)

  28. PERFORMANCE • COMPLEX CONSTELLATION OF ACTIVITIES • SIMULATION • REAL SITUATION • ISSUES

  29. PERFORMANCE ISSUES • COST • TIME TO SET UP • TIME TO SCORE- RELIABILITY • # OF TASKS SAMPLED • AUTHENTICITY VS. VALIDITY

  30. II. SELECTION RESPONSE • MULTIPLE CHOICE (INCLUDING T-F) • MATCHING

  31. MULTIPLE CHOICE • EFFICIENT SAMPLING- Time, cost • RELIABILITY- produces reliable measures • VALIDITY ISSUE: Does selection represent same knowledge as supply? • Limitations overstated by critics • current development does not take advantage of information available, new cognitive theory

  32. MULTIPLE CHOICE • Livingston, Reynolds & Willson (2005) list for good item writing • # options- depends on good alternatives • Options generated from incorrect cognitive processes, become clues to knowledge structure (e.g.. BUGGY arithmetic program)

  33. MATCHING • VARIANT ON MULTIPLE CHOICE • LIMIT # OF MATCHES < 10 • REQUIRE SINGLE CONCEPT • REQUIRE MORE OPTIONS THAN QUESTIONS

More Related