1 / 19

Overview of Evaluation Methods

Overview of Evaluation Methods. Barbara M. Wildemuth Professor, SILS, UNC-CH. Questions that might be asked. Questions about the current users of the collection Purposes, methods, impact Questions about potential future audiences Purposes, methods, impact.

nlebouef
Download Presentation

Overview of Evaluation Methods

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Overview of Evaluation Methods Barbara M. Wildemuth Professor, SILS, UNC-CH

  2. Questions that might be asked • Questions about the current users of the collection • Purposes, methods, impact • Questions about potential future audiences • Purposes, methods, impact NC sea rescue crew, NC State Archives

  3. Focus of the study • Evaluation versus basic research • Object of study Univ. of Michigan football teams, 1884, 1994, Bentley Historical Library, Univ. of Michigan

  4. Evaluation criteria Appalachian Trail, Shenandoah National Park, National Park Service Archives • Effectiveness • Utility/ usefulness/ impact • Satisfaction Pack train, Keystone-Mast Collection, 1870-1963, Online Archive of California

  5. Using the criterion for evaluation • Criterion-referenced evaluation • Norm-referenced evaluation Ladies Hockey Team, ca. 1910, Collecting Toronto: Through the Eyes of Larry Becker, City Archives of Toronto

  6. Data collection • Watching the users • Asking the users questions Nazaret ‘Chick’ Cherkezian and NET camera, National Public Broadcasting Archives, Univ. of Maryland

  7. Methods for watching the users • Direct observation during use • Think-aloud protocols • Transaction logs Library of Congress Manuscript Division Archdiocese of Milwaukee Archives

  8. Methods for asking questions • Questionnaires • Surveys • Measures • Interviews • Focus groups Campaign poster for UNC Daily Tar Heel editor, 1954 Univ. of North Carolina Southern Historical Collection

  9. Sample satisfaction measure McKinney, Yoon, & Zahedi, 2002

  10. Study sample • Define the population of interest • Consider sample size • Develop a sampling plan • Probabilistic sampling • Nonprobabilistic sampling Caroline Virginia Still Anderson (1848-1919), Presbyterian Historical Society

  11. Research design • Controlling extraneous variables • Achieving internal validity • Achieving external validity Alexander’s Ragtime Band, Historic American sheet music, Rare Book, Manuscript & Special Collections Library, Duke University

  12. Data analysis: quantitative • Descriptive statistics • Inferential statistics Quarantine Station, D. A. McLaughlin, National Archives of Canada, Canadian Genealogy Centre

  13. From results to conclusions • Check for statistical significance • Check for meaningfulness of results • Draw conclusions University College, University of Toronto Archives & Records Mgmt. Services

  14. Data analysis: qualitative • Qualitative content analysis • Specifying a unit of analysis • Recognizing themes • Looking for negative evidence Boy’s hat, Documenting the Southeast Asian Refugee Experience, Univ. of California, Irvine, Special Collections and Archives

  15. From results to conclusions • Identify themes for further analysis • Weigh positive against negative • Draw conclusions Eleanor Roosevelt, 1937 Farm & Home Week, Cornell University Archives Richard Nixon & Elvis Presley, 1970, National Archives & Records Admin.

  16. Review • The research/ evaluation question • Evaluation criteria • Data collection • Study sample • Research design • Data analysis • Drawing conclusions Bank, bottle, bumper sticker, Coca-Cola Company Archives

  17. Photo/ image credits (1) • Archdiocese of Milwaukee Archives, www.archmil.org/aboutus/dept.asp?ID=469 • Bentley Historical Library, University of Michigan, www.umich.edu/~bhl/ • Canadian Genealogy Centre, www.genealogy.gc.ca • City Archives of Toronto, www.city.toronto.on.ca/archives/ • Coca-Cola Company Archives, www2.coca-cola.com/heritage/ourheritage.html • Cornell University Archives, rmc.library.cornell.edu/collections/subjects/archives.html • Duke University, Rare Book, Manuscript, and Special Collections Library, scriptorium.lib.duke.edu/ Cunard Passenger’s log book, Mystic Seaport Library

  18. Photo/ image credits (2) • Mystic Seaport Library, www.mysticseaport.org/library/home.cfm • Library of Congress Manuscript Division, www.loc.gov/rr/mss/ • National Archives and Records Administration, www.archives.gov • National Park Service Archives, www.cr.nps.gov/history/hisnps/NPSarchives.htm • National Public Broadcasting Archives, University of Maryland, www.lib.umd.edu/NPBA/ • North Carolina State Archives, www.ah.dcr.state.nc.us/sections/archives/arch/ • Online Archive of California, www.oac.cdlib.org Back cover of “the state of Connecticut vs. new haven 9” Yale University Manuscripts & Archives

  19. Photo/ image credits (3) • Presbyterian Historical Society, www.history.pcusa.org • University of California, Irvine, Special Collections and Archives, www.lib.uci.edu/libraries/collections/special/special.html • University of North Carolina Southern Historical Collection, www.lib.unc.edu/mss/shcabout.html • University of Toronto Archives & Records Management Services, www.library.utoronto.ca/utarms/archives/ • Yale University Manuscripts & Archives, www.library.yale.edu/mssa/ Winston Churchill to Lady Randolph Churchill, May 1882 Churchill Archives Centre, Cambridge, UK, Churchill and the Great Republic (exhibit), Library of Congress Manuscript Division

More Related