NH-DI Scoring Training

1. NH-DI Scoring Training Session IScore Room Liaisons & Score Room Staff scoring@nh-di.org or Al Simons - al.simons@nh-di.org Jim Heedles - jim.heedles@nh-di.org Beth Clark - clarknh@mcttelecom.com Jim Zawisza - zawisza@post.harvard.edu

2. Overview: Liaisons & Score Room Scoring Goals Score room liaison’s role Appraiser Assignment Sheet Correct Scores Uniform Subjective Scoring Processing and Procedures

3. Scoring Goals Ensure that every team gets the full credit they deserve for their challenge solution And only the credit they deserve Turn scores around promptly Allows team managers to review them Maintain the confidentiality of the scores Never leave your paperwork unattended Let everyone get home at a reasonable hour Have fun in the process

4. Score Room Liaison Role Help the score room run efficiently! By catching problems early Gets scores back to Team Managers quickly A round trip to the score room costs at least 20 minutes Ensure scores are correct before they come to the score room Catch unusual items Ensure all the scores are recorded And only the scores they are assigned!!! (no extras) Order the paperwork for ease of use in the score room

5. Checking Scores Never have two team’s paperwork out in your work area at once If your HA needs you to look at something, put the current team’s paper away! Frequently Head Appraiser will assign Appraisers in groups. If so, order score sheets to group similar scoring patterns next to each other. Makes it easier to see missing scores. After checking, please put the score sheets in alphabetical order by Appraiser name before sending to score room.

6. Who scores what??? Initially filled in by Head Appraiser Please make sure HA uses the names that the appraisers will use on the score sheets Bring it to the score room as soon as it is complete We MUST have it well before the first team competes! We’ll enter it into the computer and return a printed copy Have the HA check it, then you keep it and use it! If it is wrong, bring it back & we’ll do it over

7. Appraiser Assignment Sheet

8. Correct Scores Make sure that each appraiser has scored all (and only) the items they are assigned Make sure that the writing is legible If you see something “unusual”, bring to attention of the Head Appraiser Two appraisers that had been tracking each other and are now widely different Joe has been scoring high all day, and scores an item as a 2. You’re the HA’s right hand, focused on scores

9. Objective versus Subjective Scores (1) Objective scores are a statement of fact. “Did the dinosaur blink its eye?” All appraisers MUST have same score. Score sheet has checkboxes for objective scores. Subjective scores reflect an Appraiser’s opinion or judgement. “Creativity of …” Always at least two appraisers Score range is 1 - 30 Scores are averaged

10. Objective versus Subjective Scores (2) If a zero is assigned for a subjective score, then all assigned appraisers must agree They are making an objective (statement of fact) decision the item isn’t eligible to be scored This becomes an objective score Thus it needs to follow objective scoring rules (all Appraisers must agree)

11. Documenting Reasons The Appraisers must give a written reason anytime there is A zero subjective score A deduction If it isn’t there, we have to send the papers back The Head Appraiser needs the reason when talking to the Team manager This is easy to catch in the challenge room

12. The “First Three” Syndrome In first level of the day, Appraisers like to keep the first three teams’ paperwork They are calibrating their judgment Sometimes want to go back and adjust the first scores they awarded In practice adjustments are relatively rare… Collect the scores from the Appraisers Do your processing If they want it back to adjust that’s fine But you’re ready to send the scores on, right after the third team

13. Deductions Intended to make the competition fairer Most deductions are assessed against specific items “Other Deductions” is a catchall category Allows appraisers to remove the unfair benefit gained by an infraction Deduction may not be for more than the score received. Two ways to do this: By points - Need to average appraisers’ scores for this Preferred: By percentage – No need to average. Worksheets provided Looks similar to score sheets; printed on a special color To be used when needed. Must give a reason for every deduction.

14. Deductions (cont) By Points: Enter the deduction in the 1-30 range … We’ll scale it in the score room, just like scores No need to scale or total By Percentage of the awarded score Enter deduction as a percentage (up to 100%) No need to actually know the score MUST indicate percent or points for each deduction! Deductions for objective scores Used with caution. Objectivity is… Use scoring units (not 1-30) e.g. pounds for weight held

15. Deductions Worksheet Don’t scale deductions. The score room will do that. For each deduction follow the number with “pt” or “%”. For example: 55% 3.2pt

16. Challenge Specific: A-Assembly Required Item 3 - Order Completion Has its own worksheet Worksheet has some calculator work All scores are objective NO Deductions allowed per PTBs(powers that be) Number of parts in Product A and B have unusual objective range, either zero, or three through nine

17. Challenge Specific: A worksheet From Tournament Data Form The order appraiser should do this. These lines need to be summed across. Order appraiser will fill in the individual boxes but may not compute the three totals. These are determined after the performance is over The order appraiser should do this These are computed by the DI scoring program.

18. Challenge Specific: B, C, and D Challenge B – Solar Stage No objective scores Challenge site may be very dark most of the time. Challenge C – Coming Attractions No objective scores Challenge D – News to me Two objective scores

19. Challenge Specific: E - Hold It! Item 1 – Three worksheets Structure weight / Legal Structure worksheet Different maximum weights based on level Elementary 75 grams, Middle 50 grams, Secondary/University 25 grams No calculator work All work done by check-in appraiser Weight Held worksheet Total weight held must be calculated Golf Balls Held worksheet Requires calculating successful attempts Requires validating successful attempts against weight COUNT Contains a subjective scoring item While deductions are allowed here, there can be unexpected consequences due to the multiplier Check with your ACM/RCM or the score room in the event this happens

20. Challenge Specific: E Golf Ball worksheet Successful Hold attempt Golf Ball Appraiser (GBA) does this. Subjective scores Make sure these are filled in Successful hold total This should be done by GBAs. Verify the total. Weight COUNT, not pounds GBAs will copy this from weight held Verify the count Useable Golf ball count GBAs do this Verify # is the smaller of two prio #’s Not used

21. Making up the Score Packet Team label on the upper left corner of envelope Please put the papers in this order—it really helps Deductions worksheet Only if any deduction was assessed Prep area sheet Order Completion sheet (Challenge A only) Weigh-in sheet (Structure only) Weight held sheet (Structure only) Golf Ball sheet (Structure only) Regular Appraiser sheets In alphabetical order by Appraiser name Everything else last No score-relevant information Stickies on the special stickie sheet at the very back of the packet ONLY positive comments! Trash inappropriate ones… Label

22. Normal Packet Return After scores are computed, we print two forms Team Manager (TM) score sheet on top Head Appraiser (HA) score sheet We return all the team’s paperwork in the envelope A green sticker means everything is OK HA can give scores to Team Manager

23. Handling Exceptions If we find a problem with the scores… We write up the problem and put it in the packet We send the whole packet back with a RED sticker Treat this as a priority; there is a Team Manager waiting! Review the problem with the Head Appraiser Leave it to HA to determine how to fix Return the package to the score room ASAP

24. To Inform the Score Room of a Change • You / your HA will have a sheet of RED hot-dots • If your HA wants to correct a score after presenting to the TM • Put a note in the packet in front of the other papers describing the desired change; HA signs it • Cover the GREEN dot with a RED one • Return the whole packet to the score room • We’ll make the change, and return the whole packet again to the challenge room • Complete with new TM and HA score sheets • We will remove and destroy the previous sheets

25. Summary Never leave your paperwork unattended! Liaisons are a vital part of the score room team If you need help, we’re happy to come by Tell a runner you want to see us Any questions on what we’ve covered? Let’s take a quick look at your score sheets.