Data Director will only be as good as the data it contains …

1. Data Director will only be as good as the data it contains… Key Questions: (Purpose) How do we know what data points to put in Data Director? How do we know if a data point in Data Director is “good”?

2. Activity 1: Brainstorm Data Points Create a list of all the possible Data Points that you or your district would think to put into data director…

3. Activity 1: Brainstorm Data Points Continued… As a group, list 10 on a poster and label rank them In terms of “most valuable” or “Best (#1)” data point to “worst (#10)”…

5. Criteria for Quality Data: Reliability Definition • Reliability is the extent to which a test is repeatable and yields consistent scores -- also note Measurement Error) Questions • Is the assessment administered under specific, consistent, and appropriate guidelines? • Has the same assessment been administered to all students, in multiple classes, multiple years? • Was the assessment scored consistently with appropriate practices? • What is the likelihood of error in the scoring of the assessment?

6. Criteria for Quality Data: Validity Definition • Validity is the extent to which a test measures what it is supposed to measure.  • In order to be valid, a test must be reliable first; but reliability does not guarantee validity. • Validity is a subjective judgment made on the basis of experience and empirical indicators. Questions • Is the assessment measuring what you think it is measuring? • Can you identify the specific skills/expectations the assessment indicates it is measuring? • Can you provide evidence that the assessment items appropriately align with the expectation’s performance verbs and content nouns?

7. Criteria for Quality Data: Context Definition • The definitions necessary to give the assessment meaning such as score range, high score, scale score/performance level, etc. Questions • What is the range for the assessment score? • Is the data raw data or has it been manipulated to be scale score or performance level? What other variables impacted the manipulation between raw score and adjusted scores? • What benchmarks indicate success on the assessment?

8. Criteria for Quality Data: Purpose Definition • The reason this data point is important for use in Data Director. Questions • Is the data point meant to be aggregated with other “similar” points at the classroom, building or district level? • What is appropriate to compare these data to? (unrelated, correlational, causal effects)

9. Criteria for Quality Data: Specificity Definition • The individual qualities that allow a distinction to be made or precise details that support the purpose. Questions • If the data point is not assessment based, does the data point indicate one point of information or is it trying to represent a multitude of concepts? • If the data point is not assessment based, was the data point consistently created by one or multiple person(s) using a reliable process? (inappropriate practices like average of averages)

10. Criteria for Quality Data: Source Definition • Origin of data. Questions • Is the data source internal (original) or did it get “moved” from another source into data director?

12. Activity 2: Application of Criteria Rubric As a group, Go through as many of your Data points on your list and break them down in terms of the criteria (as we Just practiced)

13. Activity 2: Application of Criteria Rubric Continued… As a group, Revisit your list and initial rankings, then Decide if you want to keep the order or change it (and be able to articulate why) **Be Prepared to share out with the rest of the groups your best and worst data point and why?

14. Activity 3: Reflection & Extension Reflection: How does what you just learned connect with your role as a data coach? Extension: What is one idea, concept or learning from today that you will share with another person? Who is that other person and why share it with them? What questions are still circling in your head regarding quality data?