One Economy/Broadband Opportunity coalition Evaluation plan June 27, 2011

1. One Economy/Broadband Opportunity coalitionEvaluation planJune 27, 2011 Joint Center for Political and Economic Studies

2. Evaluation Team • Joint Center for Political and Economic Studies (lead) • National Urban League Policy Institute (NULPI) • Polling – Princeton Survey Research Associates International (PSRAI)

3. Goals Goal One: Assess and determine how well the One Economy/BBOC program achieves the following performance metrics: • 150,000 new households subscribe to broadband service between April 2, 2010–March 31, 2013 as a result of the BTOP program.

4. Goals Goal Two: Compare how well the One Economy/BBOC program extends broadband use among individuals and households in five select test cities relative to five control cities without program services. A list of questions includes: • How can technology meet individuals’ demand for information on a 24/7 basis in a way that aligns with their location, literacy and language? (measure: effectiveness of public purpose media) • What marketing strategies and messages are more useful to communities that have been slow to adopt broadband? (measure: effectiveness of national awareness campaign) • How does access in the home facilitate higher rates of broadband usage? (measure: effectiveness of affordable broadband access in affordable housing) • To what extent does digital literacy training, especially when conducted by local residents, facilitate broadband adoption among hard to reach communities? (measure: effectiveness of Digital Connectors program and mobile vans) • In what way do the cumulative program impacts or discrete elements of the BTOP program contribute to a “lift” in sustainable broadband adoption? (measure: impacts of individual versus cumulative program elements)

5. Goals Goal Three: Assess and document the shift in the culture of use, and relevant “success factors” in sustainable broadband adoption among hard to reach communities, especially low-income, minority, language-challenged households, as a result of the digital ecosystem created by BTOP program elements (i.e., Digital Connectors, Access Services, Public Purpose Media, and Mobile Training). Specific questions to be asked: • What is the social dividend of technology as shown in improvements in education, health, and employment? • How does easy access to broadband in the home facilitate greater opportunities, and supports for low-income households? • How effective are digital literacy programs that employ local people to promote new technology, and its benefits to low-income residents? • What is the length of subscription for low-income households once connected to broadband services?

6. Evaluation Design and Process • Comparative research design • Process: • Identification of Test and Control Cities, Data Collection Tools, and Completion of the Evaluation Plan • Deployment of Evaluation Resources and Implementation • Creation of the Interim and Final Reports

7. Program Logic Model

8. Criteria for Test Cities • Receive all three BTOP Services • Digital Connectors • Access Services • Computer Centers • Has one BBOC affiliate in the city • Have a poverty rate that is at least 10% • Have a diverse population that highlights one of the major racial or ethnic groups of the BBOC Coalition (African American, Asian American or Hispanic American) • Represent a distinct geographic region

9. Criteria for Control Cities • Receive one or no BTOP services • Have a poverty rate that is at least 10% • Have a diverse population that highlights one of the major racial or ethnic groups of the BBOC Coalition (African American, Asian American or Hispanic American) • Represent a distinct geographic region • Have minimal BTOP investments (under $5 million) from other national programs as indicated through NTIA public records on the BTOP program

10. Test Cities Control Cities • Chicago, IL • Los Angeles, CA • Washington, DC • San Diego, CA • Raleigh, NC • Columbus, OH • Austin, TX • Baltimore, MD • Omaha, NE • Charleston, SC

11. Demographic and Economic Characteristics Source: US Census

12. Summary of BTOP Services by Test and Control Cities

13. Coverage Areas of Major Carriers by Test and Control Cities

14. Performance Indicators • The number of broadband subscribers, new versus existing by demographics • The number of trainings completed and the level of consumer satisfaction • The number of households connected • The number of community computing centers engaged • The number of training hours completed by Digital Connectors – orientation and consumer digital literacy training • The number of web users of available online content • The number of connections made by consumers to relevant services (e.g., educational, employment, health care, etc) after broadband subscription (data will be gathered through pre- and post-assessment surveys)

15. Secondary Indicators • The number of program participants enrolling and completing workforce development programs post-program engagement • The number of program participants enrolling and completing educational programs post-program engagement • The number of program participants retaining viable employment • The number of program participants advancing their expertise in technology courses post-program engagement

16. Evaluation Challenges • Identifying a stable population to evaluate over the course of the grant will be one challenge faced in the program evaluation. • Ensuring consistent data collection at all levels of the program to ensure a robust inventory of participant responses. • Anticipating the potential lag times between participants’ program enrollment, and actual service connections. • Disentangling in both test and control cities the affects of existing market conditions, and other digital access resources (e.g., libraries, community technology centers, schools, etc.). • Capturing the racial, ethnic and language diversity of constituents within the Test and Control cities, especially when gathering information for the program evaluation.

17. Evaluation Team For more information, contact: Nicol Turner-Lee, Ph.D. – nturnerlee@jointcenter.org Alicia Durfee – adurfee@jointcenter.org