1 / 31

CitEc as a source for research assessment and evaluation

CitEc as a source for research assessment and evaluation. José Manuel Barrueco Universitat de València (SPAIN) May , 2016. 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций»,

mgalvez
Download Presentation

CitEc as a source for research assessment and evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CitEc as a source for research assessment and evaluation José Manuel Barrueco Universitat de València (SPAIN) May, 2016 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», 17-20 мая, 2016

  2. Citations and citation indexes • Citations are indicators of the quality of a publication and therefore a key piece in any execise of research assesment • Bibliometric indicators (impact factor, h-index) are used (and often misused) for research evaluation purposes • Citation-based measures should complement expert peer-review and never supplant it • Research evaluation requires citation indexes in order to provide current and accurate citation data • For many years the only source of citation data were the indexes published by ISI (now Thomson Reuters) • Fortunately the landscape has changed … 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», 17-20 мая, 2016

  3. Citations and citation indexes • Thebigplayers: Thomson’s Web of Science and ScopusbyElsevier • Scholarlydatabases: PubMed or those from Cambridge Scientific, EBSCO, … • Intermediate services: Mendeley, PlumX or KUDOS are using citation data extracted from Scopus, WoS or CrossRef in their products • Social research networks: ResearchGate • Repositories: arXiv in physics or SSRN in social sciences • Atonomous citation indexes: • CiteSeer in computer science • CitEc in economics • Google Scholar Citations 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», 17-20 мая, 2016

  4. A few words about RePEc • RePEc (ResearchPapers in Economics) is a digital library • RePEc describes thewhole discipline: • Research outputs (articles, papers, chapters, books) • Publicationchannels (journals, workingpapers series …) • Researchers • Institutions (departments, researchinstitutes …) • RePEcarchitecture: data providers vs. serviceproviders • RePEc in numbers: • +2,000,000  research outputs • Comingfrom more than 1,300 data providers • Universitydepartments, internationalinstitutions, banks, publishers … • +46,000  authorcontact and publicationlistings 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», 17-20 мая, 2016

  5. CitEc : an autonomous citation index • It is a RePEc service <http://citec.repec.org> • Processes, extracts and disseminates citation data for RePEc • A citation dataset of: • +22,000,000 references • +8,000,000 citations • +812,000 documents processed (about 40% of the available documents) • CitEc produces open citation data • As other RePEc services, CitEc is based on volunteer work • No permanent funding • Server sponsored by INOMICS • Funds raised since 2014-02-04: 525 EUR 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», 17-20 мая, 2016

  6. CitEc : an autonomous citation index • Data generation: • All RePEc documents could become cited documents • 40% of the available documents are citing documents: • Full text not available: not in electronic format, not open access, not at the url provided … • Unable to process full text to extract references (i.e. digitized documents) • Data input: • PDFs provided by data providers • Need to convert PDF files to text in order to extract references • Process prone to errors: references lost, incorrect interpretation of characters … • Data providers need to do nothing about it • XML structured metadata about references • Good quality results • Only large publishers (Elsevier or Springer) are able to cope with this format • Nonetheless, around 60% of the references available come this way 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», 17-20 мая, 2016

  7. CitEc : an autonomous citation index • Quality control: • It relies on the reserchers community • Involves detecting and reporting wrong or missing content • Transparency and publicity of the data • Reporting of errors as easy as posible • at reference and document levels • Registered authors may: • Add references lists. Submitter data is logged and shown in web pages • Identify missing citations to their works. Only applies to suspect citations identified by the system. Author may claim or refuse them • Any CitEc user may: • Add missing citations to any document. All data is checked for correctness 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», 17-20 мая, 2016

  8. reporting errors referencescontributedbyresearcher addcitation

  9. Research evaluation trougth CitEc • Citation data cleaned in order to • Remove self-citations • Aggregation of cites between different versions of the same document • Evaluation at three levels: • Documents • Researchers • Institutions • Two evaluation tools • Citations profiles • Rankings 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», 17-20 мая, 2016

  10. Research evaluation • Citation profiles • Authors (46K profiles) • Research production: number of works published and their distribution by year and document type; books and journals edited; journals or series where she publishes. • Citation counts: total number of citations, unbundled number of citations by work, recent citing documents, total and percentage of self-citations. • Relationship with other researchers: co-authors, who is citing him or to whom he cites. • Basic bibliometric indicators like h-index and i10-index. • Series (2K journals and 4K working papers series) • Production and citation count data in time series that could be back to 1990 • Bibliometric indicators: impact factor, immediacy index and h-index • Institutions • To come!! 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», 17-20 мая, 2016

  11. Research evaluation • The Martin Srholec’s case: 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», 17-20 мая, 2016

  12. main data research outputs research activity additional info

  13. details

  14. main data

  15. popularity citation data

  16. Research evaluation trougth CitEc • Rankings • Allow easy comparation of individuals and institutions based in objective indicators • Created by Christian Zimmermann • Based in citations and popularity (abstract views and paper downloads) • Rankings created according to 31 different criteria • Aggregation of criteria based in the harmonic mean of the rank orders • Additional rankings for: top women, young or deceased economists; top researchers by region, country or research field… 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», 17-20 мая, 2016

  17. That’s all!Thanks for your attentionbarrueco@uv.es 5-й Международной научно-практической конференции «Научное издание международного уровня – 2016: проблем издательской этики, рецензирования и подготовки публикаций», 17-20 мая, 2016

More Related