1 / 27

A. Effect Size Can Tell a Lot! B. Future Ready Core NC Accountability Conference February 2009

A. Effect Size Can Tell a Lot! B. Future Ready Core NC Accountability Conference February 2009. Mike Gallagher Accountability Services Dept. NCDPI. Purposes – Two Parts.

dagan
Download Presentation

A. Effect Size Can Tell a Lot! B. Future Ready Core NC Accountability Conference February 2009

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A. Effect Size Can Tell a Lot! B. Future Ready Core NCAccountability ConferenceFebruary 2009 • Mike Gallagher • Accountability Services Dept. • NCDPI

  2. Purposes – Two Parts • Part A. Show how effect size can give important information to compare test results across EOG, EOC and SAT and other tests. • Part B. Review the background and give sources of information about the Future Ready Core.

  3. Part A: Effect Size Lens • Usefulness: Enables one to compare across tests and test forms. • Usefulness: Provides solid information about performance. • Concept: What effect size means. • Calculation: How to figure it.

  4. How does North Carolina improvement on SAT math compare with its improvement on NAEP math?

  5. SAT Math Score Trend for N.C. Between 1990 and 2008, the Math gap between NC and the Nation narrowed from 31 pts. to 4 pts. The annual SAT reports on the DPI Accountability web site have lots more information: www.ncpublicschools.org/accountability

  6. SAT Math Score Trend

  7. NC progress on NAEP math • NAEP results from 1992 to 2005 • Compared with the U.S. average and with other states, North Carolina went from back of the pack to ahead of the pack. NC was somewhat sluggish 2003 to 2007. • For the NAEP data in the following slides, go to: • http://nces.ed.gov/nationsreportcard/mathematics/

  8. NAEP Grade 4 Math Cut for Proficient = 249 Cut for Basic = 214

  9. NAEP Grade 8 Math Cut for Proficient = 299 Cut for Basic = 262

  10. Finding the Effect Size aka, “Standardized Difference” • Effect size compares two groups starting with a scale score. • For calculation of effect size, the average scale score of each group is needed, as well as the standard deviation (SD) of the scores. • Find the difference of the averages and divide by the SD.

  11. Comparing trends in tests: NAEP and SAT • SAT math, comparing NC in 1990 and 2008. • …..Eff.Size = (511-470)/100 = 0.41 • NAEP Grade 4 math, comparing NC in 1992 and 2007. • …..Eff.Size = (242-213)/30 = 0.97

  12. Is that a “Big-un?” • General use Effect Size Indexes • 0.20 … Small • 0.50 … Medium • 0.80 … Large • From A Power Primer, Jacob Cohen, Psychological Bulletin, 1992, Vol 112, No. 1, 155-159.

  13. While we’re talking SAT, • SAT puts an “official” question of the day on its web site. • The math questions are great thinking questions and they are mostly within our 8th grade territory. Why not have 8th graders use them as formative assessment for problem solving? • www.collegeboard.com

  14. Effect Size and NC Tests • Effect size can provide a comparison across test editions or subjects. • See the attached table for some examples. • …. The 3rd Edition Math EOG had much more rigor than the 2nd. • …. Effect size enables continuous comparison.

  15. Obtaining Av. Scale Scores • The “disag” report is on-line. • Go to the Accountability web page. • www.ncpublicschools.org/accountability • then scroll way down to Data and Reports, chose “State, School System (LEA) and school performance disaggregated data report.” • Now choose location, test, grade, subj.

  16. Green Book • Find out about getting data from the Green Book and its online brother: • both in its standard form … • and its interactive, electronic form. • http://www.ncpublicschools.org/accountability/testing/ • Scroll down to “State Testing Results”

  17. Questions to pose .. • In your LEA (or school) which grades, courses, SAT results are stronger? • In your LEA how do schools compare on average EOG, EOC, SAT? • How does your LEA (school) compare with similar LEAs (schools)? • What is the effect of what you’ve been trying to improve?

  18. Current Research • The Problem With “Proficiency”: Limitations of Statistics and Policy Under No Child Left Behind. By Andrew Dean Ho, Educational Researcher, 2008; 37, 351 • “The limitations [of the Percentage of Proficient Students (PPS) statistic] are unpredictable, dramatic, and difficult to correct.” …. From abstract.

  19. Part BFuture Ready Core

  20. ACT Study • … found that “whether planning to enter college or workforce training programs after graduation, high school students need to be educated in a comparable level in reading and mathematics.” • Ready for College and Ready for Work: Same or Different? 2006. Available on the web.

  21. Future Ready Core - Goals • On September 7, 2006, the NC Board of Ed approved goals: Future Ready Students for the 21st Century. • Note the first goal and the 1st item in that goal. • “Goal: NC public schools will produce globally competitive students. • Every student excels in rigorous and relevant core curriculum that reflects what students need to know and demonstrate in a global 21st Century environment … ”

  22. Future-Ready Core Course of Study • The State Board of Education in June 2007 approved the Future-Ready Core Course of Study with the goal that all students would graduate prepared for post-secondary education and careers.

  23. Future Ready Core Timeline • Beginning with the 2009-2010 ninth grade class, all students will be expected to meet the requirements outlined in the Future Ready Core Course of Study.

  24. FRC: Primary Differences. • An additional math course for those students who previously did not pursue the College/University Prep Course of Study. • An increase from 20 to 21 for the total number of credits required for graduation. • A recommended 4 elective credits in a concentrated area.

  25. FRC: Mathematics • 4 Credits(Algebra I, Geometry, Algebra II) OR (Integrated Math I, II, III) 4th Math Course to be aligned with the student’s post high school plans A student, in rare instances, may be able to take an alternative math course sequence as outlined under State Board of Education policy. Please see your school counselor for more details.

  26. FRC - Information • Information resources are linked from: • www.ncpublicschools.org/gradrequirements • click “Resources.”

  27. RESOURCES (on dpi web site) • Course & Credit Requirements Chart • Course and Credit Requirements Checklists for Parents • Graduating Future Ready • Making the Grade – Future Ready Graduates • Math Sequence • Questions and Answers • Student Accountability Standards Brochure

More Related