CRESST Quality School Portfolio System Reporting on School Goals and Student Achievement

1. CRESST Quality School Portfolio System Reporting on School Goals and Student Achievement

2. The What And Why of School Information Systems: Why QSP?

3. Catchwords of the Day • Standards • Accountability • Total Quality Management • School Inquiry • Learning Communities • Continuous Improvement

4. Common Perspectives • Student performance matters • Status quo is insufficient • Students’ future success requires rigorous new standards and expectations • Local school communities hold the keys to students’ success • Continuous school improvement requires on-going assessment and self evaluation.

5. Why Else? • Federal, state and local programs mandate evaluation • School accreditation requires it • Funders like it

6. A Model of Data Based Decision Making

7. Using Data to Address Important Questions: Some Examples • How are students doing? How well are they accomplishing standards? • What are the relative strengths and weaknesses of student performance? • Why are results as they are? What reasons help to students’ progress? • Are our special programs working?

8. Using Data to Address Important Questions (cont): • Are we equitably helping all students to achieve? Who are we well serving? Who not? • What previously hidden challenges do we face? • Who needs special help? On what?

9. Using Data to Address Important Questions (cont): • Are specific program components accomplishing their objectives? e.g. professional development? • Are new strategies making a difference in student learning?– new curriculum materials being tried by some teachers, tutoring services, new mathematics course? • Are our day to day standards consistent? Do our grades mean anything?

10. REASONS TO USE QSP • Understand how things are working so they can be improved • Raise awareness and support for action • Communicate with the school community • But don’t: • Use to assess blame • Use to evaluate teachers

11. Assessing Information Systems: Not All Data Are Good • Aligned • Technically Accurate • Fair • Credible • Useful

12. 1. System Is Aligned with Goals for Student Performance • Tests reflect: • standards for student performance • valued aspects of learning • what’s being taught • Multiple measures required

13. Standards-Based Assessment Framework

14. 2. Assessments Are Technically Accurate • Evidence of validity: test measures what is intended, not something else; gives accurate information for intended purposes • Evidence of reliability: measure is stable and consistent • All scores are fallible and imperfect

15. 3. Assessments Are Fair • Includes all students -- including students with disabilities and ELL students • Enables all students to show what they know and can do • All students have the opportunity to learn

16. 4. Assessments Are Credible. • Valued and believable to important stakeholder groups • Teachers • Students • Parents and Community • Funders • Different interests mean multiple measures

17. 5. Assessment System is Useful • Diagnostic value • Focus on malleable variables • Action-oriented • Dynamic

18. Ultimately, There Are NO Easy Answers • Data provides evidence • School professionals must find answers: • Use expertise to make sense of the data • Make decisions on what to do next

19. Background Information: Your QSP Database

20. Includes • Norm Referenced Tests • SAT9 administered in 1998 and 1999 • At least Math, Reading, and Spelling are represented. • Descriptive Variables • Ethnicity, Gender, Grade etc... • Score Types • Percentile Rankings • Long. data for the most recent 2 years for your school (1997 - 1999)

21. Does NOT Include • Most Program Specific Data • Course Specific Grading Variables • Last Year’s English GPA for example • Non Cognitive Outcome Data • Student Affect measures... Or Survey Data • Long. SCHOOL Data beyond two years back

22. Longitudinal Data Issues • Database is missing key data. • Dropouts, Graduates, transfers • No current variable that provides number of years at school.

23. LIMITATIONS • Very difficult to accomplish school program evaluation with only two years of data. • Difficult to accomplish deep inquiry of subject matter strengths and weaknesses without subscale data. • Difficult to Make recommendations. • Often depend on contextual information • Should have multiple measures to verify

24. So, What CAN we do with the data provided?

25. Prospects • Identify apparent general strengths and weaknesses. • Focus goals on apparent deficits. • Students at risk • Identify our apparent stars. • Inform our decisions

26. IMPORTING DATAINTO QSP

27. IMPORTING DATA 1. Choose the correct file type first 2. Choose the file you want to import. 3. Open will begin the import process. 2 3 1

28. IMPORTING DATA 1. If the variables in your data file are separated by commas (and each student is on a separate line), click OK. 2. Otherwise, adjust the item separator (and student separator, if necessary). Then press OK. 2 1

29. IMPORTING DATA 1. If the first row of the data file holds variable names, indicate that here. (The contents of the first row are shown here for your convenience.) 2. Click OK. 1 2

30. IMPORTING DATA Next, match the variables from your data to existing QSP Slots. Doing so carefully is critical. Validity of future analysis may be compromised by mistakes made here. TIPS The Objective is to match a column in the file being imported with an appropriate place for that data within your QSP Data file. It is best if you know the file being imported very well before you attempt to import it.

31. IMPORTING DATA 1. If you are not sure what information is contained by variable in your database, the Column Items box can help by showing you a snapshot of the data. 1

32. IMPORTING DATA Some variables will not have a preset QSP variable appropriate for matching. For these, you may choose from a set of “optional” variables. 1 2 3 For Example you would: 1. Select “Language Proficiency” in the first box 2. Select “Opt Backgnd 1” in the second box 3. Press Select.

33. IMPORTING DATA 1. When you have matched all the variables you want to import, you can choose Ignore Rest to quickly dispense with the others. They will appear IGNORED! in the variable pairs window to the right. 2. Finally, choose the correct year for the incoming data. (You must import ONE year at a time!) 3. Click on the Next button to proceed with the import. 3 1 2

34. IMPORTING DATA You will then see this import confirmation window. If all pairs appear correctly and are preceded by blue check marks, press OK to proceed with the import. Otherwise, you may click the Back button to make necessary changes.

35. UNDERSTANDING GROUPS

36. Disaggregations Within QSP To create new groups in QSP, first choose the Groups option under the Database menu. Or select the Button indicated here.

37. Once Groups has been chosen, the box below will appear. You would then click the New button. Creating Groups The Current Group category represents the group that is presently represented. The Custom Groups category represents any groups that have been created. The System Groups category represents the already existing groups within the Data. The Combo Groups represents any combination of groups. Click the New button to create a new group.

38. Once you click the New button, the following dialog box will appear. You then have the choice of Custom Group or Combo Group. Creating Groups First choose Custom or Combo group, then click OK.

39. If you choose the Custom Group category, this dialog box will appear. Creating Groups

40. Depending upon the variable you select, the operator options will change. Creating Groups Choosing a Categorical variable will provide the Booleans (Is or Is not). Choosing aNumerical Variable will provide the (=, <>, <, >) as possible selections.

41. To create a Custom Group, you must name the group, then provide the criteria for the group. Creating Groups For instance, in this example, we’d create a group that represents all students with a GPA below 2.0. This Group can then be added to reports or used to filter Graphic representations like Pie charts and Histograms. Type in the name of the group. Choose a variable. Select the Operator Then indicate the comparison value. Click OK when done.

42. You can also create Combo Groups, by choosing the Combo option. Creating Groups Click on Combo Group. Click OK when done.

43. The Combo Group, allows you to combine up to eight groups to in order to focus your reporting on a particular population of interest. Creating Groups Let’s say you want report on the Hispanic male Students who have Spanish as a Primary Language. You can select those groups, then combine these with other outcome related groups you have defined yourself like our Low GPA Group. Name the Combo Group. Choose your Groups. You may combine up to eight. Click OK when done.

44. Creating Groups Any Group can also be viewed in the QSP Database View. Select the Group in the Dialog and hit “OK” then return to the database View. Also indicates the number of students who fit that criteria.

45. Tabular Reports In QSP

46. QSP Tabular Report Options (5 Types) Crosstabulations And Frequencies Answers only Count and % Not Longitudinal Only Categorical Variables

47. QSP Tabular Report Options cont. Tables - Mean, Median, and % at or above Groups By Variable on Multiple Outcomes Longitudinal Option Questions Table - Mean, Median, # & % Tested Groups by Variable on One Outcome Multiple Operators Simultaneously Advanced Table (Mean, Median, Number) Groups by Groups on One Outcome

48. Key Issues with Tables Often Difficult to Read Formatting Difficulties Information Overload Require More effort on the part of reader. Provide more Information, especially Longitudinal options.

49. Crosstabs frequency tables for one categorical variable, sub-divided by another We Will Make a Crosstab that includes Gender & Grade

50. Frequency Tables List the numbers of times each value occurs in a set of values We will create a Frequency Table of the Language Fluency Variable