Publishing in Measurement Journals

1. Publishing in Measurement Journals Journals as People (Not Just Outlets), Publishing as a Process (Not Just an Event) Presentation for EDMS MSMS Steve Ferrara American Institutes for Research December 4, 2006

2. Overview for this talk • A story about an article in EMIP • Some requested themes • What some editors look for • Review process • Responding to review feedback • Discerning the real meaning of an editorial decision letter Measurement Journals

3. Educational Measurement: Issues and Practice (EMIP) • Aimed at practitioners and users of tests, as well as professional educators, legislators, school personnel, and interested citizens. Its primary purpose is to promote a better understanding of educational measurement and to encourage reasoned debate on current issues of practical importance to educators and the public. EM:IP also provides one means of communication among NCME members and between NCME members and others concerned with educational measurement issues and practices. • http://www.ncme.org/ Measurement Journals

4. Measurement journals and other excellent outlets • EMIP • Journal of Educational Measurement • Applied Measurement in Education • Educational Assessment • Educational and Psychological Measurement • Applied Psychological Measurement • Objective Measurement • Measurement: Interdisciplinary Research and Perspectives • International Journal of Testing • Measurement and Evaluation in Counseling and Development • Language Testing • Educational Leadership (e.g., theme issue on assessment November 2005) • Phi Delta Kappan • NEA Today • Journal of Applied Testing Technology • Practical Assessment, Research and Evaluation • Content area journals (e.g., JRST, Reading Teacher) Measurement Journals

5. History of this article: late 2003 • “Findings…are quite important…study seems to be well designed and executed.” • “However, my position is that the current version of this paper is not suitable for publication in EMIP. In general, the paper…For that reason, I will not send out the paper for review.” • “You may want to consider revising the current paper and resubmitting it to EMIP. If you do, please include a discussion of how you have addressed each of these concerns. I will be pleased to send out a revised paper for review.” Measurement Journals

6. History: early 2004 • “The reviewers and I concur that this paper is appropriate for publication in EMIP, once you respond to a small number of issues. These issues focus on providing clarity and explicitness for the wide range of EMIP readers.” • “If you choose to revise and resubmit the current paper, please include a discussion of how you have addressed each of the reviewers’ comments and mine.” Measurement Journals

7. History: summer 2005 • Appeared in summer issue of EMIP: 24 (2) • An excellent paper Measurement Journals

8. Review and decision process • Decisions by people who follow guidelines and make judgments using explicit and implicit standards • Decide whether to have it reviewed • Reviewer comments to author and recommendation to the editor • Weigh the recommendations, make a decision • Write the editorial decision letter Measurement Journals

9. Responding to review feedback • Respond to every single comment that is made • Treat every single comment as advice • You’ve already put yourself on the line by submitting • Get past defensiveness, outrage at the unfairness of it all, etc. • Treat the process as an opportunity • Your professional development • Improve your paper • Improve your future work Measurement Journals

10. Discerning the real meaning of an editorial decision letter • I have read your paper, * (SF-), submitted for consideration for publication in EMIP. I also have read and considered comments on the paper from three reviewers. You have written an interesting/thought-provoking/compelling paper, and I appreciated the opportunity to read it. • The reviewers and I concur that the current version of the paper is not appropriate for publication in EMIP. • I offer the following comments for your consideration. • I have enclosed for your consideration comments from the three reviewers. If you choose to revise and resubmit the current paper, please include a discussion of how you have addressed each of their comments and mine. • Or: • You may want to consider other outlets for your work. • I have chosen not to send your paper out for review. Measurement Journals

11. Discerning the real meaning (cont.) • The editor does not want to discourage submissions (or may want to) • EMIP: 25-30 unsolicited submissions a year to fill ~16 slots • JEM: 100+ unsolicited submissions a year • Editors want to treat colleagues fairly and not make enemies • Editors want to fill out the journal with good and compelling articles Measurement Journals

12. What editors may look for • Want to influence the field • Need to feel confident about decisions • Want to be collegial and professional • Need to uphold the standards of the journal and the field • Don’t want to be embarrassed or make an error • Care and caution • Fairness, quality • Issues, practices, methods to target • Hot topics in the field • Recent papers • New ideas • Editor’s publication record • Editorial Advisory Board Measurement Journals

13. Feel free to contact me: Steve Ferrara Managing Research Director Educational Assessment Program American Institutes for Research 202-403-5431 sferrara@air.org Measurement Journals