1 / 26

TEST

TEST. SLIDE. Align. Display. The Concept and Development of an NCTA Professional Journal. NCTA National Conference San Diego California September 23, 2011. The Concept and Development of an NCTA Professional Journal. The Journal Concept Project Status. Journal Publication.

ornice
Download Presentation

TEST

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TEST SLIDE Align Display

  2. The Concept and Development of an NCTA Professional Journal NCTA National Conference San Diego California September 23, 2011

  3. The Concept and Development of an NCTA Professional Journal The Journal Concept Project Status

  4. Journal Publication • Internet-based publishing • Easier Access • Lower costs • Publication frequency • Annually at first • Later, semi-annually

  5. Issue Composition • Articles • Featured pieces written by subject experts • Subject to peer review before publication • Ideally, three to six each issue • “Short Items” • Shorter articles • Used to address subject briefly • Between 0 and 2 per issue

  6. Issue Composition • Book Reviews • Review of recent publications related to Testing • Not peer reviewed, but edited • Varied articles per issue, about 1500 words in length • Notes on Contributors • Announcements • About the Journal • About NCTA

  7. Interface with the Research Community

  8. Sample of Psychometric Topics Placement Test Development/Validation

  9. Test Security Test Development and Classroom Intervention Strategies to Reduce Cheating

  10. Voluntary System for Accountability • Psychometric Considerations: • sample size • random vs. systematic samples • longitudinal vs. cross-sectional designs • floor & ceiling effects • validity of measures

  11. The effect of high school socioeconomic status on the predictive validity of SAT scores and high school grade-point average Zwick, R. & Himelfarb, I. (2011). Journal of Educational Measurement, 48, 101-121.

  12. Attempts to predict College FY GPA from HS GPA have been shown to result in over-prediction for African-Americans and Latinos. • They are predicted to do better than they actually do • Including SAT (or presumably ACT) scores into formula helps, but there is still noticeable bias. • FY GPA is slightly under-predicted for other ethnic groups Hypothesis is that over-prediction stems from HS's having different resources, and correlation between the availability of those resources and ethnic composition of HS.

  13. Created a measure of HS SES = sum of three variables • Aggregated father education • Aggregated mother education • Aggregated family income Many other variables they wanted to consider, but access to the data was difficult. These HS SES variables were all captured as part of the SAT student survey. They are all self-report measures.

  14. Predicting FY GPA from • Model 1: HS GPA • Model 2: HS GPA + SAT subscores • Model 3: HS GPA + SAT subscores + HS SES

  15. Prediction Errors for Different Models

  16. Prediction Errors for Low-SES HSs

  17. Prediction Errors for High-SES HSs

  18. Journal Leadership • Editor / Co-Editors • Point of contact for Journal • Selects reviewers • Decides whether to publish/not publish • Editorial Advisory/Review Committee • Helps set direction for the Journal • Edits manuscripts • Participates in decisions to publish/not publish

  19. Role of Outside Experts • Non-NCTA members • As authors • As peer reviewers and subject matter experts • As contributors to Journal leadership • Why they are valuable • Build on our knowledge base • Bring insights based on varied expertise • Contribute to the Journal’s credibility

  20. How do Articles get Published? • Manuscript is submitted to the editor • Reviewers Evaluate the Manuscript • Comment on content • Make recommendation to publish/not publish • Editor Assigns Peer Reviewers • Double Blind Review • Each manuscript gets at least two reviewers

  21. How do Articles get Published? • Editor Decides on Publication • Publish “as-is” • Publish subject to changes • Revise and Re-Submit • Not to publish • Article is assigned to an issue of the Journal • Article is formatted and copy-edited

  22. The Concept and Development of an NCTA Professional Journal Discussion Q & A

  23. Dr. Jack Turner University of South Carolina jeturner@mailbox.sc.edu Dr. James Wollack University of Wisconsin - Madison jwollack@wisc.edu

  24. Presentation Blurb: As NCTA has grown there have been increased opportunities to benefit the membership and influence the testing field. A professional journal hosted by NCTA is an opportunity to highlight testing-related research, technology advances, and business trends in a way that is relevant to our membership and beneficial to their professional development. This workshop will describe the partnership formed between NCTA and the Measurement Services Group within the American Educational Research Association (AERA) to develop a formal proposal for "The Journal of the National College Testing Association" to be submitted to the NCTA Board. Workshop attendees will be invited to comment on the concepts and plans being discussed. Opportunities for ongoing involvement in the project will be available.

More Related