1 / 22

Creating A Multiple Measures Placement System

Creating A Multiple Measures Placement System. An Exercise With Ron Gordon & Armand Brunhoeber. The Concept. Although test scores may predict failure, they do not necessarily predict success.

teness
Download Presentation

Creating A Multiple Measures Placement System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Creating A Multiple Measures Placement System An Exercise With Ron Gordon & Armand Brunhoeber

  2. The Concept • Although test scores may predict failure, they do not necessarily predict success. • Research shows that students’ backgrounds, environments, and personal habits may have more influence on their potential academic success than their residual academic skills. • Find a way to factor that information into the placement decision at testing time.

  3. Decision Process • Involve Faculty • Select Questions • How many • What variety • Select Weight Values • Use negative weights? • How much possible total weight

  4. Select Questions • Use enough questions that any one does not have undue influence • Must be manageable • Each weighted answer choice requires an additional line in the placement rule • Weights must be repeated in each rule segment • Questions must, in some way, relate to student success.

  5. Limits • Limit total weight so that background information does not allow students to skip a course level • Limit number of questions to a manageable number • More questions adds to testing time • Placement rules can become unmanageable • Answer choices must be mutually exclusive and all inclusive

  6. Example • (If Arithmetic, plus all weighted choices is >= 75 OR Algebra, plus all weighted choices is >= 48)AND (Algebra, plus all weighted choices is < 65 OR Algebra Not Taken) AND (CLM, plus all weighted choices is <62 OR CLM Not Taken) Then Placement is Elementary Algebra

  7. Example, Continued • If this rule had 5 questions with 4 weighted answer choices each, there would be 80 lines just for weights. • With too many questions, or too many choices per question, rules can become unmanageable

  8. Conditional Weights • High school accomplishments have limited shelf life • How much does it matter that a 25-year-old student had 2 years of high school algebra? • Does it matter that the same 25-year-old student works for a surveyor and uses algebra daily?

  9. Conditional Weights • Young students have not had time to build skill usage experience, but their high school accomplishments are relevant • What older students accomplished in high school is less relevant than what they do now.

  10. Example • How long has it been since you were enrolled in high school or other formal educational process? • Less than 2 years or still enrolled • 2 to 5 years • More than 5 but less than 7 years • 7 years or more • Use high school data for up to 5 years, experience for more than 5 years.

  11. Assigning Weights • Total possible weight should not move student more than one level in either direction • Set maximum possible weight so a student who scores near or above the midpoint of a placement range could move up, but one who scores below the midpoint could not. • Use faculty to select BGQ and assign weight • Guide them

  12. Multiple Measures Movement Model

  13. Sample Question With Weights • Which choice below best describes you when you read textbooks or other complex information? • I usually need to read material several times before I understand it well -.01 • Sometimes I can understand what I read the first time, but often I must reread it .00 • I usually understand what I read if I take notes or highlight passages. +.01 • I always understand what I read the first time through +.02

  14. Preparing to Build the System • Assign numeric codes to course names • Determine which tests will be used for each course in each discipline • Create cut score Table • Create a BGQ weight Matrix

  15. Building the System • Create Background questions • Assign BGQ to groups • Create branching profiles • Create course groups • Create courses and assign to groups List • Create majors if used • Create placement rules Edit1Edit2

  16. Verify • Write most complex rule first • Run verify function in branching profile • Use several BGQ and score combinations to test the placement rule • Compute weighted score for each run • Try to hit cut scores to test for bad weight or answer choice selections

  17. Common Errors • Unequal weights between rules • E.G. A response has .01 weight in one rule and -.01 weight in the next rule • Misplaced Parentheses • The multiple measures weights make the rule larger and more difficult to visualize • Misuse of AND/OR • Misuse of arithmetic operators • Wrong answer choice in rule line

  18. Troubleshooting • From the score report, determine what the student’s weight should be from the BGQ responses • Using the weight, compute the weighted score • Determine what the placement should be • Examine the appropriate rule for errors

  19. Computing the Weight 1

  20. Computing the Weight 2

  21. Computation • Score is multiplied by 1 plus the accumulated weight. • 85 * (1+.04) = 88.4 • Placement will be based on a score of 88. • Example 2 • 85 * (1+ [-.03]) = 82.45 • Placement will be based on a score of 82

  22. Creating a Multiple Measures Placement System An exercise with Ron Gordon & Armand Brunhoeber Thank you for not throwing things at the presenters

More Related