Evaluating Journals and the Institutions that Publish in Them

Cabell’s Adds a New Dimension These ranking systems will assist users when: Evaluating the importance of a journal in furthering future research Determining the degree of difficulty an author encounters when seeking to publish in a journal

Classifications of Methods to Evaluate the Importance of Journals

Classifications of Methods to Evaluate the Importance of Journals Journal Citation Count - number of times an article has been cited in a publication Online Usage– citations, tweets, blog posts, news stories that mention the article Peer Review- panel members make judgments about research published in journals

Criteria for Evaluating these Approaches Reliability Validity Usefulness

Why Peer Review is Not Enough Inherent subjectivity lessens reliability Reviewers may have limited knowledge May lack framework for comparing the journals

Why Online Usage is Not Enough Does not meet the validity criteria Tweets, blog posts and news stories only indicate awareness Number of downloads does not indicate usefulness

Methodological Criticisms of Citation Counts Emphasis on the journal cited not the article Approach does not apply to more recent digital scholarship sources Citation counts slow to accumulate and update

Further Methodological Criticisms Reasons to cite other than develop a framework: Refute previous research Political reasons Curry favor with editors/reviewers Vary among disciplines Algorithms and statistical methods are difficult to understand

Why is the Cabell’s Approach Better? Classification Inclusion Simplicity

Cabell’s Classification Index (CCI)

Cabell's Classification Index (CCI) Used to determine the influence of a journal Frequently cited journals are considered to have more influence on future research Journals that have been published for five years or more are considered to influence future research

Cabell's Classification Index (CCI) Using citation counts from the most recent 3 years: Average citation count for each journal was calculated Reduces the importance any one article had on the journal’s citation count Gives more importance to recent published research relative to articles published more than 3 years ago The total weighted score of each journal was normalized using its Z score

Cabell's Classification Index (CCI) Classifies the influence of a journal into 5 categories:

Number of Journals & Average Citation Count

Sample Journal Entry with CCI

Number of Journals Listed in Cabell’s and/or Scopus

Journal Longevity CCI calculation includes journals that have been published for at least 5 years Longevity of publication indicates the journals have influence on research Publication history makes the journals familiar to researchers Permits researchers to use previous issues to determine the journal’s usefulness

Benefits of the CCI Allows journal rankings to be compared across disciplines Accounts for some variation in the calculations without altering perceived influence Provides a more in-depth description of the journals

Cabell’s Institutional Publishing Activity Index (IPA)

Why Acceptance Rates Are Not Enough 3 Issues Methodology used to compute acceptance rates differ among journals Publishers are reluctant to provide acceptance rates Specialized focus of a journal may limit the number of people who can publish in that journal

Cabell’s Institutional Publishing Activity Index (IPA) Used to improve the process of evaluating institutionspublishing in journals Recognize the differences in research resource availability among institutions Such factors result in differences in the publishing effectiveness of institutions

Cabell’s Institutional Publishing Activity Index (IPA) Using citation counts from the most recent 3 years: The counts in each classification were weighted by the average citations per document (article) Compensates for smaller percentages of journals in Premiere (10%) and Significant (10%) groups relative to the number of journals in High Influence (80%) Recognizes increased importance due to frequency of citation of journals in Premiere and Significant groups The total weighted score of each institution was normalized using its Z score

Cabell’s Institutional Publishing Activity Index (IPA) Classifies the publishing activity of an institution into 4 categories:

Cabell’s Difficulty of Acceptance Index (DA)

Cabell’s Difficulty of Acceptance Index (DA) Used to determine the varying degrees of difficulty authors encounter in seeking to have their manuscripts accepted by journals Institutions whose faculties frequently publish are likely to have advantages in resources It can be more difficult for authors to publish in journals which frequently accept articles from “Recognized” institutions (IPA)

Cabell’s Difficulty of Acceptance Index (DA) Using citation counts from the most recent 3 years: Classifies the challenge of having manuscripts accepted by journals into three categories Each journal’s Z score is calculated, and rankings are based on Z score

Cabell’s Difficulty of Acceptance Index (DA)

CCI Summary Cabell’s Classification Index (CCI) Includes a larger number of journals than other databases that emphasize citation counts Makes citation counts user-friendly Allows comparison of journals across disciplines Recognizes the relative influence of journals

IPA Summary Cabell’s Institutional Publishing Activity Index (IPA) Recognizes that the institution plays a major role in the faculty’s publishing activity Accounts for disparities in the budgets available for research, equipment, facilities, faculty support, research assistance, the rewards for publishing, and the capabilities of individual faculty members

DA Summary Cabell’s Difficulty of Acceptance Index (DA) A measure of an individual’s capability to publish Those journals labeled as “Rigorous” accept more manuscripts submitted by authors at “Recognized” institutions than institutions not labeled as “Recognized”

Cabell’s Login http://www.cabells.com Members Page Login User ID: Singapore Password: Cabell’s

Evaluating Journals and the Institutions that Publish in Them