Download
1 / 11
110 likes | 122 Views
Reading Discussion – s519 by Peter Hall – 1/21/09.
E N D
Reading Discussion – s519 by Peter Hall – 1/21/09 Ryan, J., McClure, C. R., & Bertot, J. C. (2001). Choosing measures to evaluate networked information resources and services: Selected Issues. In C. R. McClure & J. C. Bertot (Eds.), Evaluating networked information services: Techniques, policy and issues (pp. 111-135). Medford, NJ: Information Today Inc.
Abstract • IT and information managers see the importance in measuring their organizations networks and information flow important • Two questions to consider – What are the key elements of a strategy that measures these networks? What are the key issues that may arise when employing such a strategy? • Research suggests successful managers develop a strategy and consider key issues prior to implementing these measures and continuing with data collection, analysis, and use procedures
Examples of Measurements • Quantitative (Statistics) • Performance measures (Statistical ratios) • Qualitative (Interviews, surveys, observations, shadowing, etc.) • Any others?
The Research • Sponsored by United States Institute for Museum and Library Services • Multi-method approach – Site visits, case studies, interviews, focus groups, surveys, observation, path analysis • Iterative approach – Study team adapted collection and analysis based on learning • Limited study scope – Study participants were all library information managers • Drawbacks and/or limitations?
Selection Issues • What were some potential concerns and issues that these library information managers identified prior to adopting data collection, measures, and analysis? • Why are you measuring something? • What are the potential uses? • Tradeoffs – New measures bring additional costs, time, and resources of an organization • What will you do with your findings? • May have different outcomes and effects for different stakeholders in organization (i.e. management, employees, shareholders)
Measure Types • Capacity – Ability of an organization to use or deliver information through network • Use – Utilization measure of these resources or services • Efficiency – Relates services or resources used to services or resources provided • Impact – Effect of resource or service on an activity or situation • Outcome – Are goals achieved? • Examples of measure types? • Other considerations • Defining measures, data collection, and analysis • Fair measures – All of an organizations operations
Data Collection • Sources – Are all relevant stakeholders’ input collected? • Can data be collected uniformly for accurate analysis? • Which data collection measures should be used? Qualitative, quantitative, both? Any new collection techniques that should be used? • Privacy – Are stakeholders free to give an accurate portrayal? • Balancing data collection across organization • Proper instruction and preparation is key • Any other potential issues?
Data Analysis and Use • Combine new data with existing information – Internal, external • Example – Your new data with national studies or internal performance or composite measures • Is this information compatible for comparison?’ • How do you analyze the results generated by a new measure? • Is further instruction necessary? • Who interprets and how? • How is this data presented? • How do you use it? • Does the end cost outweigh the benefit? • Table 6.11 - Examples
Conclusions / Recommendations • Information managers should choose network measures based on context of change, complexity, utility, and resources • Proper strategy required prior to implementation of network measures, data collection, and analysis for success • Reducing and balancing staff burden of collection and analysis • Be able to provide proper instruction/training to properly educate important stakeholders • Future and additional considerations and research
Discussion Topics • Has anyone been involved in this process in their organizations? • Success/failures, most important metrics, time consumption, cost/benefits? • Did anything come out of this data? • Any other analytics tools used? • i.e. External, web-based tools • Internal work flow documents • Thoughts on the reading – Agree/disagree, glaring deficiencies, techniques used, research methods?
