1 / 14

Peer-Reviewer Guidance Data Seal of Approval

Peer-Reviewer Guidance Data Seal of Approval. Hervé L’Hours DSA Conference 2014. Amsterdam. 24 September, 2014. Overview. The Seal is awarded through the independent adjudication of a Peer Reviewer appointed by the DSA Board.

siobhan-crofton
Download Presentation

Peer-Reviewer Guidance Data Seal of Approval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Peer-Reviewer GuidanceData Seal of Approval Hervé L’Hours DSA Conference 2014. Amsterdam. 24 September, 2014

  2. Overview • The Seal is awarded through the independent adjudication of a Peer Reviewer appointed by the DSA Board. • Assembly Members commit to peer-review a maximum of three repositories per year per DSA repository. • Currently primarily by Board members • Peer-reviewer expansion acknowledged as critical for scaling • Peer-reviewer training acknowledged as critical for community confidence

  3. “Peer-Review”? • Never a good sign if Wikipedia has a disambiguation page for the term…. • “the terminology has poor standardization and specificity, particularly as a database search term.” • Trust-based review at a distance, based on evidence statements and supporting (ideally publically available) documentation • But not a full ‘Audit’ in ISO17021 terms

  4. ISO17021 • ISO Standards are guided by ISO17021 • “Requirements for Bodies Providing Audit and Certification of Management Systems” • “The overall aim of certification is to give confidence to all parties that a management system fulfils specified requirements. The value of certification is the degree of public confidence and trust that is established by an impartial and competent assessment by a third-party.”

  5. ISO17021 • Where we differ • Community-based collaboration, not legal entity • No site visits • Language • Further changes to the DSA approach will be guided by advice from within the community and from wider collaboration such as that with WDS through RDA.

  6. Who Peer-Reviews You? • “A Peer Reviewer must have completed at least one self-assessment, which resulted in the award of the DSA without outsourcing compliance to any guidelines, and be a member of staff of a General Assembly member“ • “The Board will attempt to match peer reviewers with the appropriate subject matter of the repository, but this is not a requirement” • “The Board will attempt to match peer reviewers with the relevant language of the repository, but this is not a requirement.”

  7. Guidance for Peer Reviewers • • Does the self-assessment statement correspond to the Guideline in question? (Also refer to the supporting text for applicants below each Guideline.) • • Are the links to supporting documentation available publically? • • Do you agree with the self-assessed compliance level and is the overall evidence sufficient to award the DSA? • • Have abbreviations been explained? • • In responding to the self-assessment provide helpful comments rather than specific questions where possible.

  8. Process • “A peer reviewer appointed by the DSA Board will either confirm the self-assessment evidence and compliance levels for each Guideline or reject them and provide relevant comments to be addressed by the applicant. The comment/response process will continue until the reviewer can decide whether to award the DSA. In the event of a dispute the applicant may appeal to the Board.” • Impartial • Confidential until the DSA is awarded.

  9. E. Appeals • An organisation not awarded the Seal after peer review [] may appeal to the Board in writing. • The appeal must clearly state the rationale behind the appeal. • The Board will review the appeal rationale and take a decision on a course of action with an 85% majority vote of the Board required. The action may include, but not be limited to: (a) affirming the earlier decision or (b) restarting the application process, with a new peer reviewer assigned.

  10. Evidence • Transparency • Link to publicly available documentation • Or deadline for public release • English or short summary in English • Reviewers Guide: “Topics for discussion and inclusion are suggested but they are neither exhaustive nor prescriptive”

  11. Challenges to Come • How do we know what is • Appropriate? • Sufficient? • Impartiality • Consistency • Competence • Openness and responsiveness to criticism

  12. Challenges: Outsourcing • “To take account of the increasingly distributed and service-based nature of modern repositories, the DSA Board expanded the possibility of outsourcing to all Guidelines. This decision will be monitored over time and may be amended in future in cooperation with the DSA community. Applicant information relevant to outsourcing is requested in the ‘Repository Context’ section and must form part of the evidence for each applicable Guideline.”

  13. Solutions • Public Forum for evidence and discussion • Sector-specific guidance • Work on standardisation of evidence base from preservation plans to licencing and access criteria

  14. Peer-Reviewer GuidanceData Seal of Approval Hervé L’Hours DSA Conference 2014. Amsterdam. 24 September, 2014

More Related