The VAM Roadshow Note: This presentation will be provided electronically.

1. The VAM RoadshowNote: This presentation will be provided electronically.

2. Current Legislative Session • The current Legislative Session may make changes to SB736 and approve DOE provided rules required by SB736 • Decisions made will impact some of the information in this PowerPoint. Could possibly be affected: • Explanations of cut points (slide 12) • FAQ (21, 26, 28-31) • Will most likely not be affected: • Definitions/Explanations of VAM, Direct, Indirect, characteristics of VAM (4-11, 13-15) • FAQ information (17-20, 21-22, 25, 27)

3. What will you learn? • Review of the Value Added Model (characteristics, standard error, confidence interval, rating information) • How to access student VAM information (only for those with direct VAM) • General FAQs and Info you need to know • Rubric changes • Assessments being used or not used • Plan for adding assessments (K-3) • VAM is not CAG friendly, but it can be useful information • What if an evaluation is below Effective?

4. What is VAM? • It is different than School Grades and the way we’ve always used FCAT for things such as learning gains • Covariate adjustment model • Uses student-level prior test scores and other measured characteristics to predict student achievement • Compares a student’s success to other student’s “like” them to create a predicted score • Comparisons are created based on Value Added Characteristics • Students predicted score is then compared to their actual score • Currently VAM used for FCAT and Alg I

5. Mount FCAT Value Added: Where are students on the mountain in relation to other similar students? School grade: Where are students on the mountain? Mount FCAT

6. Direct Versus Indirect • Direct state assessment- An FCAT Course and students took FCAT • Indirect- Not an FCAT Course but there are students who took FCAT • School FCAT-Reading VAM • District FCAT-Reading VAM

8. What characteristics are used to determine student comparisons and a student’s predicted score?

9. Value Added Characteristics Other predicting characteristics Characteristics with greatest ability to predict score

10. Other characteristics

11. Other characteristics

12. Value Added Information Continued • VAM scores are not singular (example- your score is a 1) • VAM scores are actually ranges • Initial range • Standard of Error • Confidence Interval

13. VAM Information Highly Effective Effective Effective Needs Improvement Effective Unsatisfactory Effective -.24 0 .11 Unsatisfactory Needs Improvement Effective Highly Effective

14. Accessing Direct FCAT VAM Information Do you know your High Score? Low score? Do you know your school’s high score and low score? Did any part of your range go into the negative?

15. Scale Score versus Predicted Score

16. Accessing the Direct FCAT VAM Information • Go to the AIMS Companion site at https://rttt.bayschools.net • Log in using your usual sign in credentials • Go to My Appraisals and select the + sign to the far left of the brown box. • If you have a direct VAM, you have a section titled something like, “VAM Mathematics/Reading.” • Select the + sign. • Down at the bottom there is a link titled, “Student Details” • You will only have this link if you have a DIRECT VAM (you teach an FCAT course and students took FCAT)

17. FAQs and other things YOU need to know!

18. Important Rubric Adjustment • During 10/11 the TASC decided to have different expectations for category 1, 2, 3, and 4 teachers. • This was approved by the TASC, the Board and DOE. • “When you know better, you do better.” • Scenario: • Category 4 teachers with an HE Instructional Practice and HE IPDP and an E in Student Growth were rated Effective (2.5 points). A Category 3 or 2 teacher would have had a HE rating with the same amount of points. • Solution: • For the 12/13 school year, all category of teachers will be rated the same. • The point range for HE for everyone is 2.40 and above.

19. Will we be able to use Discovery Education Scores? • At this time there is no plan to use Discovery Education scores for Student Growth • Discovery Education is used to progress monitor student learning and predict student achievement on FCAT • Cannot mix Progress Monitoring and Evaluative Data • Varying administration at schools impacts data reliability

20. When will all courses be given direct assessments? • Senate Bill 736 states that by 2014-15 school year, all courses offered by the district must have a direct assessment to assess student growth • Where do these direct assessments come from? • State Assessments: FCAT, End Of Course Assessments, FAA • District Approved Assessments: Brigance, IB, AICE, Dual Enrollment, AP • Approximately 50% of courses remaining not covered. • DOE provided Item Bank • Margaret Gamble is working with DOE and CFAC (Central Florida Assessment Coalition) to provide item writers • Concerns • Due to be complete 2014-15, the first year the performance pay is mandated by law • Who will maintain it, who will pull assessments, who will assess fidelity, who will input? • Resolution • We will maintain current Student Growth measurements and add state assessments and corresponding VAM as we have done with Alg I • Hope: The Legislature will give us more time.

21. K-3 Specifically • Issues we are grappling with- • Must include Student Growth measure (SB736) • DEA: Cannot be used (Progress Monitoring) • FAIR: Currently not aligned to Common Core and will not be aligned to Common Core until 2014 (Would it be evaluative or progress monitoring) • SAT 10: Not Aligned to Common Core and expensive • Iowa Assessment: Aligned to Common Core and VERY expensive (70K per year) • Item Bank: Not available until 14/15 • Current solution: Maintain School Level VAM decision • The Oversight Committee plans to revisit this very soon.

22. VAM isn’t CAG Friendly • We do not get a list that neatly describes all of the variables and how a teacher’s students do against all the students in the state • We aren’t given a neat matrix that says, “Teacher A- your ESE students did this, your 4th period did this, etc.” • We only know if your students met their predicted score or not and which students counted into your VAM • We are asking DOE if they can provide us the kinds of data people want to know so that we can use it to get better • But right now, we are not getting that information

23. So how can we use VAM information?

24. It is useful! For the 1st time ever, like students are being compared to like students.

25. A more complete picture of student learning

26. Bay District Schools*Excludes schools with no school grade Achievement Value Added High Achievement= A-B School Grade Low Achievement- C-D School Grade High Growth- Highly Effective- Effective VAM Low Growth- Needs Improvement VAM

27. What if my overall evaluation falls below effective? • Breathe. • Don’t panic. • Your first few steps should involve checking all of your data in AIMS. Sometimes there’s a computer glitch or something was overlooked etc. • THE BEST TIME TO DO THIS IS BEFORE YOU LEAVE FOR SUMMER BREAK. Go through your instructional practice rating and know what it is and know that it is accurate before you go!!! • What do Option B people check? (IPDP only)

28. Okay, it’s accurate and below effective. Now what happens? • This will affect your pay. According to Florida Statute, only teachers with an overall summative rating of effective or higher are eligible for either a step increase or the longevity bonus. You will receive a letter from me explaining the change in your pay (or lack of change as the case may be).

29. And then? • We are required to remediate teachers with overall ratings of needs improvement and unsatisfactory. • You will receive a self-assessment that you and your principal will complete together indicating mutually-agreed upon areas of need. • The professional development team and I will then review those assessment and match needs with existing professional development (at our expense).

30. What does a plan look like? • Each plan is organized by the domains and contains the identified needs, and PD required, to address those needs. • Plans include a combination of Beacon online classes and TDY days (paid for by the district) with the services of Staff Training Specialists who work with teachers on an individual basis. • Where needs are common, we have after-school workshops to work with small, focus groups.

31. This is embarrassing.Who will find out? • Obviously your principal will know as will the Director of HR. The Staff Training Specialists assigned to work with you will also “know” but must keep the information confidential as part of their job descriptions. STSs work with teachers across the district … no one will know why you are working with one other than you have chosen to do so. • Note: It is not public record for that school year. However, according to SB 736, the 11/12 evaluation is public record at the end of the following year (one year). • What can be provided is still up for debate and we are working on clarification (just the summative rating OR the summative pieces– IPDP, Instructional and Student Growth)

32. Want more information? • VAM presentation • Project 8 link at http://www.bay.k12.fl.us/Project8/tabid/2335/Default.aspx • Go to Florida Value Added and Student Growth • DOE VAM information • http://www.fldoe.org/committees/pdf/sghandout.pdf • http://www.fldoe.org/committees/pdf/PresentationValue-addedModel.pdf • www.fldoe.org/committees/doc/Value-Added-Model-White-Paper.doc

33. Contact Information • Sharon Michalik • Executive Director for Human Resources • michasj@bay.k12.fl.us • Dawn Capes • Instructional Specialist for Race to the Top • capesda@bay.k12.fl.us • Leon Faircloth • Project Manager for LIIS • fairccl@bay.k12.fl.us