Reviewing articles: Some guidelines for reviewers

Reviewing articles:Some guidelines for reviewers Duan van der Westhuizen University of Johannesburg 25 May 2012

Academic Fraud • Fabrication (making up results) • Falsification (manipulating research materials) • Plagiarism • Self-plagiarism – or multiple publication of the same content with different titles and/or in different journals "salami" This includes publishing the same article in a different language • The violation of ethical standards regarding human and animal experiments • Ghostwriting–someone other than the named author(s) makes a major contribution. • Conferring authorship on those that have not made substantial contributions to the research • Suppression of “bad” results

Academic Fraud • Scott S. Reuben (born 1958) was Professor of Anesthesiology and Pain Medicine at Tufts University • Sentenced to prison for health care fraud. • Considered influential researcher in pain management • His purported findings altered the way millions of patients are treated for pain during and after orthopedic surgeries. • Admitted that he never conducted any of the clinical trials on which his conclusions were based "in what may be considered the longest-running and widest-ranging cases of academic fraud.

What is your mission? “A reviewer’s job is to criticize the manuscript. Find any faults in the logic, method, results, and conclusions that the paper might have and then communicate these to the editor.” The editor is seeking your advice on whether to publish the article, right? Therefore ….

Double-blind peer-reviewed • Critical to the scientific process • Humans do not display their best behaviour when they are cloaked behind the mask of anonymity • “Critical” does not mean criticise • It means evaluate or judge – strengths or weaknesses, contribution • You have to an expert in the field

Good and bad reviews • A good review is supportive, constructive, thoughtful, and fair. It identifies both strengths and weaknesses, and offers concrete suggestions for improvements. It acknowledges the reviewer's biases where appropriate, and justifies the reviewer's conclusions. • A bad review is superficial, nasty, petty, self-serving, or arrogant. It indulges the reviewer's biases with no justification. It focuses exclusively on weaknesses and offers no specific suggestions for improvement.

Your decision • State your bottom line! • Accept (with revision) (between 5% – 25%) • Reject • Not original • Not new • Data/method is invalid • Not ready for publication

What is the “big” picture? • Is the problem that is addressed an important one? • Does it advance knowledge in the field in a substantive way? • What is the intellectual plot-line?

The Title • Read carefully • What are the main ideas? • Who were the participants? • What is special?

Abstract • Read carefully • Context • Theoretical Motivation • Method • What variables? • Major results • Cause and effect?

Introduction • Context • Rationale • Review of previous research • Context • Purpose (objectives)

Criterion 1: Significance and contribution to the field • What is the aim? • Was the aim achieved? • What is added to body of knowledge • Theory • Data • Practice • Method • Does it fit with the field? • What is missing? • Is this a problem?

Criterion 2: Method and approach • What approach/method was used? • What is the nature of the data? • Are there issues of bias/objectivity? • Are the methods appropriate? • Have the methods been applied correctly? • Have variables been overlooked that could influence the results? • Are the results valid and reliable? • Are ethical issues addressed sufficiently?

Criterion 3: Argument and use of evidence • Is the problem or hypothesis clear? • What claims are made? • Is the argument consistent? • Is evidence provided consistently? • How valid and reliable is the evidence? • Does the evidence adequately support the claims? • Are the conclusions valid?

Criterion 4: Findings and discussion • Do the findings cohere with hypotheses/research questions? • Have the results been interpreted appropriately? • What are the implications of the findings? • What recommendations for further research is made? • Are the conclusions valid?

Criterion 5: Writing style and text structure • Is the style sufficiently “scientific”?? • Was care taken with presentation of the text? • Language, grammar, spelling? • Is writing clear? • Are citations accurate? • Is the discourse appropriate? • Is the articled arranged or organised appropriately? • Tables/figures clear?

Criterion 6: Journal criteria • Editor should do this • Journal prescribes style • Guidelines published in Journal

Structure of your report • It may be prescribed, otherwise: • Short summary: what the article is about • Critique: • More positive than negative, negative first • More negative than positive, positive first • Advice on overcoming shortfalls • Recommendation/conclusion: Briefly! • Think carefully about recommending that further research is required before publishable

Report: Insufficient Literature “…the study is not well embedded in the literature. Although it is easy to find studies on …the authors .. do not cite any of these works. Thus, the contribution of the paper is difficult to judge. …the authors do not carefully compare their results with the findings outside South Africaor in South Africa.”

Some common good habits • Be speedy! • Read carefully! • Take notes • Try find something positive to say • Negative does not have to be nasty – avoid hostility • Keep it brief – 3 – 4 pages • Don’t nit-pick – keep the issue in mind

Reviewing articles: Some guidelines for reviewers