ABDC Journal Quality List Review 2013

1. ABDC Journal Quality List Review 2013 1503 & 1599 Ingrid Nielsen Department of Management Faculty of Business & Economics Monash University

2. How were panel members chosen? • A call for self-nomination to panels 1501, 1502, 1503, 1504, 1505, 1506 and 1507 was made on 8th April by Stephen Taylor, Chair of BARDsNet • Applications closed on 19th April • Applications were reviewed by the BARDsNet Executive, the overall Review Chair, Robert Faff, and the already appointed Panel Chairs • Due consideration was given to size, disciplinary representation, gender balance, availability and demonstrated expertise to contribute to the process • I was advised on 3rd May that my application to be a panel member had been successful • Initially, the panel comprised Neal Ashkanasy, Bob Cavana, Gavin Jack, Ingrid Nielsen and Vikas Kumar, ABDC Journal Quality List Review 2013

3. What happened specifically with 1503? • For personal reasons, the appointed 1503 Panel Chair, Neal Ashkanasy, decided to step aside from the Chair role • I was appointed to assume the Chair role on 21st May • Following feedback about a perceived expertise gap on the 1503 panel in the HR/ER areas, Adrian Wilkinson was added to the panel, also on 21st May • The 1503 panel was larger than any other panel, because of the size and disciplinary breadth of 1503 (741 journals in 2010; 821 in 2013) • There may have been people in the community who remained unhappy with the panel’s gender balance, but I was not unhappy with it • The particular mix of panel members worked really well together over the coming months on what was an enormously complex task ABDC Journal Quality List Review 2013

4. How did the process unfold: submissions • Calls for submissions were made on the ABDC website on May 1, 2013, with a deadline of May 31, 2013 • Submissions could be made in four (4) separate categories: • Request to add a new journal (Form A) • Request to downgrade a journal’s ranking (Form B) • Request to upgrade a journal’s ranking (Form C) • Request to transfer a journal’s FOR code (Form D) • All submissions were managed through the ABDC Office, where each was tagged to reflect the panel (MAN), the Form and the journal – eg., MAN_FC_F_096 • A ‘short form’ of each submission was also created in the ABDC Office, tagged with an ‘S’ – eg., MAN_FC_S_096 ABDC Journal Quality List Review 2013

5. How did the process unfold: submissions • Dropbox files were created in the ABDC Office for each Panel containing all submissions for a given Panel • An Excel spreadsheet summarizing each Panel’s full complement of submissions was also contained within the Dropbox and each submission number corresponded to the Excel line number • Links to the completed Dropbox files were sent to Panel Chairs on 24th June • Submissions with missing information and/or made on an incorrect form were initially highlighted in the Excel sheet in yellow until all submitters had been contacted and asked to update missing information and/or use the correct form ABDC Journal Quality List Review 2013

6. How did the process unfold: framework of the panel deliberations • In respect of logistics, the ABDC allowed each panel to devise its own internal processes as there was much heterogeneity in terms of numbers and locations of panel members as well as submission numbers • In respect of referents, panels were guided to refer to a reasonably homogenous set of indicators, however there was appropriate leeway given to differences in the ways certain disciplines viewed particular indicators • It would be reasonable to say that common referents across the panels included recourse to other well-established rankings, SSCI impact factors and immediacy indices, citation metrics, rejection rates, editorial board composition and in some cases, actual peer review was undertaken and further views were sought from the community ABDC Journal Quality List Review 2013

7. How did the process unfold: 1503 panel deliberation principles • A decision was taken early on that no panel member would be eligible to deliberate on a submission for any journal for which he or she was a past, current or incoming editor or for which he or she was currently on the editorial board or had held editorial board membership within the past five years • In respect of the above, a decision was taken that in the event that less than four panel members remained to deliberate any given submission, the Chair would to seek the view(s) of other community members to take the total number of decision points to at least five • A decision was taken early on that all panel members must disclose during deliberation on any given journal that he or she had published in that journal during the past five years ABDC Journal Quality List Review 2013

8. How did the process unfold: submissions to 1503 • A total of 205 submissions were made to 1503 as follows: • Form A – 68 • Form B – 1 • Form C – 136 • The panel endorsed removal of forty journals, where the content was deemed out of scope, the journal had insufficient English language content or the journal was a duplicate in error on the original list • Panels were required to arrive at ranking distributions in the following approximate ranges: • A*: 5 – 7% • A: 15 – 25% • B: 35 – 40% • C: remainder ABDC Journal Quality List Review 2013

9. How did the process unfold: 1503 panel deliberation process • A first cut was achieved via responses to a multiple choice Google doc, eg., • I agree that journal X should be upgraded from B to A • I disagree. Journal X should remain a B • I disagree. Journal X should be double upgraded to A* • I disagree. Journal X should be downgraded to C • Where all members were in agreement, the submission outcome was pencilled as resolved • Where majority plus 1 (5) was achieved, the submission outcome was pencilled as resolved except for cases where the dissenter was considered the primary expert in that area ABDC Journal Quality List Review 2013

10. How did the process unfold: 1503 panel deliberation process • A teleconference was held to attempt to resolve the ‘expert dissenter’ cases • Each unresolved submission was assigned to two panel members whose tasks were to make a written case for or against the submission • These cases were submitted in writing to the Chair and pulled together into a second round Google doc for voting • A face to face meeting of the panel followed, where all unresolved submissions were discussed and resolved, with two exceptions… ABDC Journal Quality List Review 2013

11. How did the process unfold: APJHR & JMO • Almost as much time and consideration was given to these two decisions as to the rest of the review task combined • For each journal, multiple confidential views were sought in light of the ‘local relevance’ factor • Several people declined to provide a view because they could not arrive at a decision themselves • We were required by the ABDC to retain the status quo unless the case for change was clearcut and in neither case could we achieve a majority view supporting change • In both cases though, the panel and its ad hoc members strongly encouraged resubmission to the next review ABDC Journal Quality List Review 2013

12. How did the process unfold: between panel relationships • Where submissions existed that proposed FOR code change, these discussions took place between the implicated Panel Chairs • Where erroneous double coding existed, decisions as to final FOR codes took place between implicated Panel Chairs and the higher ranking was adopted • Discussions about any other between panel issues took place at two face-to-face meetings of the Panel Chairs, held at UTS • The major between panel issue arose as a result of the Economics panel taking a decision to wholesale delete 236 journals from their portion of the list, without advising any of the other Panel Chairs • A request came to me on September 17 to review the titles (not the rankings) of the deleted Economics journals with a view to developing a solution to reinstating them in the list • My review of these titles precipitated the creation of a 1599 coded section ABDC Journal Quality List Review 2013

13. How did the process unfold: Economics delisted journals in 1503 and 1599 • Given time and availability constraints, the tasks of the 1599 panel were undertaken only by myself and Gavin Jack • Of the delisted Economics journals, 65 were brought into 1599 and 82 we brought into 1503 • Taken together, the journals brought into 1599 formed a fairly internally consistent group of journals covering environmental science, political science and international relations • The journals taken in to 1503 were those where a disciplinary presence of the journal type already existed in 1503 (eg., public administration) ABDC Journal Quality List Review 2013

14. How did the process unfold: feedback • The draft list was released for feedback on 11 September, 2013 with a deadline for responses of 30 September, 2013 • The criteria for feedback were the identification of errors in the draft list (ranking, spelling, FoR code, duplication etc); and did not extend to reiteration of previous arguments • 406 responses were received, 142 of these related to 80 journals in 1503 • Responses of the following type were acted upon: • Duplications • Reports of predatory journals • Errors in the title or ISSN/ISBN • Responses of the following type were not acted upon: • Reiteration of a previous submission with no further evidence ABDC Journal Quality List Review 2013

15. Final outcome • The final rankings distributions for 1503 and 1599 were: • 1503 • A*: 6.8% (56) • A: 23.4% (192) • B: 24.7% (203) • C: 45.1% (370) • 1599 • A*: 10.6% (7) • A: 28.8% (19) • B: 33.3% (22) • C: 27.3% (18) ABDC Journal Quality List Review 2013