250 likes | 363 Views
EVALUATION OF LSTA FIVE-YEAR PLAN. Presented by: Ester Smith, Ph.D. EGS Research & Consulting. Objectives and Methodology. Evaluate overall effectiveness of LSTA eight grant programs (1998-2001) using secondary data Evaluate in-depth Texas Library Systems, TANG, and Special Projects Grants
E N D
EVALUATION OF LSTA FIVE-YEAR PLAN Presented by: Ester Smith, Ph.D. EGS Research & Consulting
Objectives and Methodology • Evaluate overall effectiveness of LSTA eight grant programs (1998-2001) using secondary data • Evaluate in-depth Texas Library Systems, TANG, and Special Projects Grants • Three-prong methodology: (1) analysis of 1998-2001 data; (2) surveys of Library Systems, member libraries, Special Projects grantees and their patrons; (3) case studies
Survey Highlights • Obtained data from 10 Library Systems, 422 member libraries (81.5%), 17 Special Projects Grantees, and 62 participants • The Library Systems developed a comprehensive assistance infrastructure • The Systems are highly responsive to needs of member libraries • The Library Systems provide a wide range of services to member libraries
Survey Highlights (Cont.) • Most common services: collection development (98%), continuing education (97%), training in use of electronic resources (88%), consulting (77%) • More than 2/3 of libraries regard assistance “very helpful” • Libraries expressed a high level of satisfaction with Systems’ services and assistance
Survey Highlights (Cont.) • As a result of assistance and services, libraries saw significant improvements in operations: • Library staff increased management knowledge (85%) • Collections are more current and broad (77%) • Better use of technology and resources (74%) • Offer enhanced access to variety of information (73%)
Urban, Suburban Rural Area of Service • 72% of libraries served primarily rural areas • Libraries serving rural areas were less technologically advanced, did less planning, fewer were members of consortia • Libraries serving rural areas were of greater need of service, considered services more beneficial, more satisfied with services, improved services to a greater extent
Legal Service Population Size • 58% libraries have small legal service populations (10,000), 31% medium (10,000-49,999), 11% large (50,000+) • Legal service population size was significantly associated with libraries’ level of automation; services from Systems (small: Internet connections, training advisory boards, grant writing); perceived helpfulness of Systems; degree of library improvement
Libraries Operating Expenditures • 31% had operating expenditures of less than $50K; 35% - $50K-$150K; 34% - $150K+ • Operating expenditures were significantly associated with level of automation; type of services; perceived helpfulness; degree of improvement • Libraries with small operating expenditures were more dependent on Systems
Technical Assistance Negotiated Grants (TANG) • TANG changed the technology map of Texas libraries • TANG staff provided assistance with security, networking, troubleshooting, servers, operating systems, grants • 86% found TANG services either “very helpful” or “helpful” • TANG increased libraries self-sufficiency
Special Projects Grants • TSLAC awarded 17 grants to 15 libraries • Grants involved: bilingual/ESL programs (9); expansion of non-English collections (7); job assistance (1) • Grants served early childhood, youth, homebound elderly, families, bilingual, low-income, low literacy populations • Participants were highly satisfied with services
Special Projects Grants (Cont.) • Impact of grants: • - Recruited new groups of patrons (88%) • - Increased number of patrons (82%) • - Increased number of pre-schoolers exposed to reading (59%) • - Increased parents’ recognition of reading importance (59%) • - Increased literacy rate in community (47%) • - Increased English proficiency (35%)
Key Conclusions • Systems infrastructure offers a comprehensive set of services and support in all areas of library operations • Libraries considered Systems to be very helpful • Rural libraries, libraries with small legal service populations, libraries with small budgets greatly depend on Systems and have higher appreciation and greater improvement
Key Conclusions (Cont.) • TANG has made a significant difference in libraries’ technological self-sufficiency • Special Projects Grants provided valuable services that had a direct impact on participants, their children and families and subsequently on the community
LSTA Programs • Texas Library Systems • TANG • Special Projects Grants • Library Cooperation Grants • Continuing Education and Consulting • Library Establishment Grants • TexNet Interlibrary Loan Program • Texas State Electronic Library Program
Evaluation Addressed: • Activities performed in 1998-2001 • Evaluate based on allocation of funds, libraries served, impact of program • How well programs promoted access to information resources • Major accomplishments and successful practices • Areas for improvement
Analysis Findings • Each grant program showed a high level of activity as measured by variables such as: materials acquired/provided, number of persons receiving services, number of librarians trained or assisted • Attempted to establish common performance measures across grant programs
Data Limitations and Implications • Mixing of funding sources limits ability to associate outcomes with funding source • Using Targets as a performance measure • Targets as a management tool • Longitudinal performance: random variance from year to year; lack of patterns or continuity
Data Limitations and Implications (Cont.) • No correlation between performance measures and portion of population served (counting events/occurrences) • Program effectiveness = productivity, efficiency, impact • Unclear if libraries use productivity and efficiency measures as management tools
Going Beyond Output Measures • Libraries and Systems generate a great volume of data, but not actionable data • Current output measures limit evaluation: show activity but not much else • LSTA needs to develop impact measures for the different programs • Create a model with standards again which libraries and Systems can evaluate their performance
Going Beyond Output Measures • Use performance data (targets, output, productivity, efficiency, impact) and longitudinal data as operations management and planning tools • Develop performance models using productivity and efficiency data and use these as performance targets