Mathematics and Science Partnership (MSP) Programs

1. Mathematics and Science Partnership (MSP) Programs U.S. Department of Education: San Diego Regional Meeting February 22, 2010

2. The Annual Progress Report (APR) • SEC. 2202.(f): Each eligible partnership receiving a grant or sub-grant shall report to the Secretary… • Partnerships/projects must send their APRs to their State Educational Agency (SEA), the SEA must review and approve the APR, and submit it to the U.S. Department of Education (USED) within 60 days from the end of the performance period.

3. APR (continued…) • Typical Example: • Project Achieve is awarded a 3 year MSP sub-grant that has a start date of 3/1/2009 and an end date of 2/28/2012 (performance cycle); with “performance period 1” starting on 3/1/2009 to 2/28/2010. • The project has 12 months to complete their activities for that first year (as indicated in their application to the state). • During the first 12 months the project would have completed their first year’s activities, collected student data, collected teacher data and would be ready to start the report by 2/28/2010.

4. APR (continued…) • The project finishes the APR and sends it to their SEA (at a time determined by the SEA; before 60 days). • The SEA will review the APR and determine if • The APR is complete and accurate; or • The APR needs revision. • If the APR is complete and accurate the SEA will submit the report to USED by 4/30/2010 (60 days after the performance period). • If the SEA has determined that the APR needs revisions, the SEA will comment on the report, the project will make the needed changes, resubmit to the SEA, the SEA will review again, and submit to USED by 4/30/2010 (60 days after the performance period).

5. APR (continued…) • This process will continue until… • The project has completed a cycle (in this case 3 years) and submits their final APR (projects are encouraged to also submit their Final Evaluation Report that summarizes all 3 years of activities); or if • The project is discontinued (a decision made by the SEA). • At the end of the performance cycle projects must re-compete to receive an additional cycle of funding. (Note: not all states have 3 year cycles)

6. Overview of the APR Submission Process (Norm) SEA creates an APR for the project to complete The project completes the APR and submits it to the SEA by the indicated date. SEA sends the APR to the USED. The USED reports MSP results to Congress. The SEA reviews the APR and determines if the APR is acceptableOR if revisions are needed.

7. OR…Partial Submission (extraordinary) SEA creates an APR for the project to complete The project completes the APR and performs a partial submission and informs the SEA of when the APR will be completed by the indicated date. (Only with SEA approval). The SEA reviews the partially submitted APR and determines if the APR is acceptableOR if revisions are needed & accepts or negotiates the final submission date with USED input. SEA sends the APR to the USED. The USED reports MSP results to Congress.

8. Ed-MSP.NET

9. APR Due Dates

10. Accessing your APR • Post-award, your SEA coordinator should create a new APR for newly funded projects and continuing projects. • You should receive an email from Ed-MSP.net (check your spam box). • If you have forgotten your login or password please access the forgotten password option on the login page located on http://apr.ed-msp.net.

11. Logging In After accessing this link, it will ask you for your email (contact your SEA if you are not sure which email is listed). The system will send you an email to re-establish your new login and password. If that does not work contact msp@wasoftware.com or your state’s contact at USED

12. Levels of Access There are three levels of access for projects • Project Directors can read, write, and submit reports to their state coordinators. (2 Max) • Other Contacts can read and edit reports. (5 Max) • Read-only access. (5 Max) *It is the project’s responsibility to tell the SEA which individuals require access and their levels.

13. Completing the APR Most of your answers can be found in the User's Guide…please refer to this document.

14. User’s Guide A copy of the User’s Guide can be found as a part of your handouts!

15. User’s Guide Overview • Step-by-step instructions on how to complete the report. • Includes definitions that describe what each section should include. • Includes FAQ’s • Includes technical information on the Mathematics and Science Partnerships Teacher Content Knowledge (MSPTCK) • User’s Guide for the MSPTCK

16. Completing the APR (cont…) Change APR Status…allows the PD (only) to submit the report to the SEA coordinator. Edit New APR…allows the project to enter data into the APR

17. Completing the APR (cont…) The Status column indicates where the project is in the APR submission process. -Incomplete…not submitted to SEA -Approval Pending…sent to SEA; SEA has yet to review. -Revisions Needed…reviewed by the SEA; project must re-submit to the SEA after changes have been made. -Partial Approval…project is missing core data (teacher, student, and/or evaluation); project must re-submit when data is complete by the approved date. -Approval Granted…SEA has approved the APR and submitted to USED.

18. APR Contents

19. APR Section I- MSP Project Information This section provides a summary of basic project information: • Project: Name, APR Performance Period, funding amount, etc. • Lead Organization: the fiscal agent. • Partner Organizations: includes the fiscal agent, the High-Need Local Education Agency (LEA), the Institute of Higher Education (IHE), partner schools (public and private), other LEAs (charters, etc.), businesses, non-profits, or other.

20. Section II- Abstract This section asks the project to briefly describe their project. The abstract should include a description of: -the participants (teachers, student, etc); -the professional development models and/or interventions; -any other information the project would like to share. *The abstract will be viewed as a public document on the ed-msp web-site and must be cut & pasted into this section. It may not be saved as an attachment.

21. Section II- Abstract You must cut and paste or type into the text box below.

22. Section III- Responsibilities This section is designed to identify the various partners’ roles and responsibilities. • Administer the Overall Partnership: (logistical management, budgeting, recruitment, etc). • Designing Professional Development: (Creating PD curriculum, designing learning tools, assessments, etc). • Deliver Professional Development: (Leading the PD sessions, technical assistance, etc). • Evaluate MSP: (Collecting and/or analyzing data, designing the evaluation plan, etc.).

23. Section III- Responsibilities Projects should make their best estimate in this matter; however it should be noted that the percentages should add up to 100%

24. Section IV- PD Participants This section asks about information on participants in professional development activities • Number of Higher Ed Faculty involved in the MSP Project (math, science, engineering, education, and other) • 1. Primary Goal (improving teachers’ content knowledge, training teachers leaders, both, or other)

25. Section IV- PD Participants • 2. Primary Target (individual teacher or schools) If you select schools, you will then enter information on each school and tell whether the selection of schools focused on high-need schools.

26. Section IV- PD Participants The remaining sections identify the number of participants in an MSP project. • Total teachers receiving MSP PD in Math or Science (don’t double-count!) • Elementary School Teachers • Middle School Teachers • High School Teachers • Administrators* • Participant Students *The focus of PD must be on training teachers, and teachers must make up the majority of participants. Administrators may attend PD, but can not receive a stand-alone offering.

27. Section IV- PD Participants If a teacher teaches multiple groups of students (e.g. general education and AP) count the teacher only once – under the teacher’s primary/dominant role. If a teacher teaches multiple groups of students (e.g. general education and AP) count the teacher only once – under the teacher’s primary/dominant role.

28. Section V- PD Models This section asks the project to identify the number of contact hours, the structural model, and a detailed account of activities in the professional development models. • Contact Hours: the average number of PD hours provided to each teacher (e.g. summer institute hours, on-line follow-up, etc…notindependent study, etc.) • Type of PD Activities: this section asks the project to identify the structural model and primary focus of PD: 1. Summer institute: PD that is conducted for at least 2 weeks and 60 hours 2. Summer institute with follow-up: a summer institute plus at least 3 days of follow-up training conducted in the classroom during the academic year. 3. Other: Any structure that is not listed above. • Description of Professional Development Model -Provide a description about the person that provided the PD and a more detailed account of the activities.

29. Section V- PD Models Choosing a PD type will open a specific series of windows that allow the project to provide further detail.

30. Section VI- Professional Development Content This section identifies the Professional Development Content. • Mathematics Content • Science Content • Both

31. Section VI- Professional Development Content

32. Section VII- Program Evaluation • Type of Evaluator • Evaluation Design: Identify the primary design (Random Assignment, Matched Comparison, Non-matched comparison, One-group, Qualitative/descriptive, or other) and provide a brief description • Did your project administer pre/post tests? When did the tests take place (immediately before the PD, immediately after, 3 weeks after, etc.) & how often? • Did you assess student content knowledge? • Did the project use other measures of success, if so please describe? • Include any relevant information about sampling, data collection procedures, and analysis . • Phase of Implementation

33. Section VII- Program Evaluation Choose the Phase of Implementation

34. Section VII- Program Evaluation D) Assessment Measures: • Provide information on each assessment used to evaluate teachers & information on each assessment used to evaluate students. • Indicate whether each assessment has been tested for validity and reliability and whether it is used for GPRA.

35. Section VII- Program Evaluation If the project uses multiple assessments (as many projects do)…the project must access this page and option multiple times to report on each assessment or evaluation tool.

36. Section VII- Program Evaluation E) Analysis of Changes in Teacher Practice: - How did your project measure success in the classroom? F) Teacher Findings G) Student Findings H) Impact on the Partnership I) Other Impacts *In these narrative sections, please describe anecdotes or describe evidence of the program’s effectiveness.

37. Section VII- Program Evaluation J) Upload Report: Evaluation report In the Evaluation Report, we are looking for specific information on how the evaluation was designed and implemented. • How did the project determine if teachers experienced statistically significant gains in content knowledge? • Were there any threats of contamination? If so, how were they handled? • Provide a timeline and description of each of the evaluation activities. • The Final reports should address the Data Quality Initiative (DQI) rubric criteria (if projects are reporting experimental or quasi-experimental designs).

38. Section VII- Program Evaluation If projects do not have the evaluation report ready by the date of submission, projects may (in consultation with the SEA and USED) partially submit the APR until the evaluation report is completed.

39. Section VII- Program Evaluation Please note…if this is your project’s final year (and will have to re-compete to acquire additional funds), please check this box & attach an evaluation that evaluated the project across its entire performance cycle. This should include (among other things) the aggregate student and teacher information and evaluation across the life of the project.

40. Section VIII- Government Performance & Results Act • Teachers: • Use the MSPTCK spreadsheet to indicate the number of teachers who have made statistically significant gains in all professional development opportunities. *Remember to save the spreadsheet to your personal computer as it will not save on the site. • Students: • Include student data using only the most important assessment for mathematics or science. *For Both Sections: -If the results are unavailable, projects may (with consultation of SEA and the USED) perform a partial submission.

41. Section VIII- Government Performance & Results Act

42. Section VIII- Government Performance & Results Act Click the spreadsheet… Note: if the project has administered more than one content knowledge assessment to different groups of teachers, run the MSPTCK for each group and aggregate the results. If you have difficulty please access the users guide.

43. Section VIII- Government Performance & Results Act Select Enable Macros

44. Section VIII- Government Performance & Results Act

45. Section VIII- Government Performance & Results Act If you need detailed instructions please access this link for step by step directions.

46. Section VIII- Government Performance & Results Act States may have changed their assessment structure, but the concept of application remains the same

47. Section IX- Lessons Learned In this section we are asking the project to provide an overview of the challenges and successes encountered in the project over the past 12 months. A) MSP Implementation B) MSP Evaluation

48. Section X- State Review This section is to be filled out by the state coordinator. • 1. MSP initial award date 2. Length of MSP award 3. Current year of implementation 4. Indicate if it is a final report

49. Section XI- Supplementary Docs.

50. Questions? • Pat O’Connell Johnson at Patricia.Johnson@ed.gov • Miriam Lund at Miriam.Lund@ed.gov