660 likes | 778 Views
National Consortium for leadership in sensory disabilities. 3+2 REVIEW JUNE 2011 WASHINGTON, DC Kathleen M. Huebner, Project Director Brooke Smith, Project Coordinator John Killoran, Deafblind Liaison Brenda Schick, Deaf/Hard of Hearing Liaison Michael Sharpe, External Consultant.
E N D
National Consortium for leadership in sensory disabilities 3+2 REVIEW JUNE 2011 WASHINGTON, DC Kathleen M. Huebner, Project Director Brooke Smith, Project Coordinator John Killoran, Deafblind Liaison Brenda Schick, Deaf/Hard of Hearing Liaison Michael Sharpe, External Consultant
Project Overview The Problem: • There is an immediate and critical need for doctoral-level personnel in the sensory disabilities areas of… • blindness/visual impairment (B/VI) • deafness/hard of hearing, (D/HH) and • deafblindness (DB) • To assist in ameliorating the ongoing shortage of… • appropriately and adequately trained personnel for providing early intervention and educational services to children and students with low incidence disabilities • leadership personnel who direct and teach in low incidence personnel programs and/or who serve in agency and public policy leadership positions • researchers trained in low incidence disabilities NLCSD 3+2 6.21.11 OSEP # H325V090001 2
Project Overview The mission of NLCSD is: “To increase, through specialized doctoral training, the number and quality of leadership personnel competent in the areas of higher education and research to improve education services for infants, children and youth who have sensory disabilities (deafness/hard of hearing, blindness/low vision, and deafblindness) including those with multiple disabilities.” NLCSD 3+2 6.21.11 H325V090001 3
Project Overview This will be accomplished by… • Increasing the number of doctoral leadership personnel in low incidence sensory disabilities • 28 NLCSD Fellows • Creating the NLCSD Consortium • Doctoral level programs in Special Education focusing in one of the identified sensory disabilities - All B/VI and DB; Those that meet criteria in D/HH • Developing and implementing a Research-Based Enrichment Program • Strengthen research abilitiesand confidence • Establishing an inclusive and collaborative Community of Learners • Enhance academic performance of students with sensory disabilities • Improve the process, collaboration, and efficiency with which research informs practice NLCSD 3+2 6.21.11 OSEP #H325V090001 4
Project Overview Methodology… • A participatory action approach for the conceptualization, development, and implementation of project activities • Involves others in identifying the needs and assisting in developing the policies, procedures, solutions and doctoral education • Initially more time consuming than typical approaches, participatory action • Increases buy-in from often disparate groups • Leads to efforts that are more sustainable • Formative and summative evaluation activities are the responsibility of the project’s independent, external evaluator • Evaluation results are shared among all stakeholders • Input, feedback and participation used by the Leadership Team for revising project activities, planning future activities and ongoing decision-making NLCSD 3+2 6.21.11 OSEP # H325V090001 5
Project Overview NLCSD is a: • Multidisciplinary, collaborative effort which includes FELLOWS NLCSD 3+2 6.21.11 OSEP # H325 V090001 6
Project Overview NLCSD 3+2 6.21.11 OSEP # H325V090001 7 Fellows Two Cohorts… • 12 in Cohort 1 • 2 DB • 4 B/VI • 6 D/HH • 16 in Cohort 2 • 1 DB • 9 B/VI (2 to join Cohort 1) • 6 D/HH 28 Total Anticipated • 3 DB • 13 B/VI • 12 D/HH
Project Overview NLCSD 3+2 6.21.11 OSEP #H325V090001 8 23 National Organizations including… • 8 Blind/Visually Impaired Consumer, Parent, Service Agencies • 8 Deaf/Hard of Hearing Consumer, Parent, Service Agencies • 4 Deafblind Consumer, Parent, Service Agencies and Projects • 3 General Special Education Organizations and Agencies Public Advisory Council
Project Overview NLCSD 3+2 6.21.11 OSEP # H325V090001 9 24 IHEs • On campus leadership programs leading to doctorates • 16 programs in B/VI • 13 in D/HH • 8 in DB *Several have more than 1 program representative 3 additional IHEs represented by Leadership Team members • Salus University • University of Colorado - Boulder • Western Oregon University University Consortium
Project Overview NLCSD 3+2 6.21.11 OSEP # H325V090001 10 Fiscal agent • Salus University/College of Education & Rehabilitation Project director • Dr. Kathy Huebner Project coordinator • Dr. Brooke Smith Fiscal and administrative leadership and support • Dr. Audrey Smith • Ms. Wendy Woodward • Ms. Tina Fitzpatrick Salus University
Project Overview NLCSD 3+2 6.21.11 OSEP # H325V090001 11 Funding agent • $999,500. annual average Initially funded for 4 years; now 5 years Federal Project Officer • Ms. Glinda Hill OSEP
Key Personnel Federal Project Officer • Ms. Glinda Hill, USDOE, OSEP Leadership Team • Dr. Kathleen Huebner, Director, Salus University • Dr. Brooke Smith, Coordinator, Salus University • Dr. Brenda Schick, University of Colorado - Boulder, Liaison-D/HH • Mr. John Killoran, Western Oregon University, Liaison-DB Senior Research Fellows • Dr. Kay A. Ferrell, University of Northern Colorado • Dr. ShirinAntia, University of Arizona • Dr. Roseanna Davidson, Texas Tech University Enrichment Committee Chairs • Dr. ShirinAntia, University of Arizona • Dr. Deborah Hatton, Vanderbilt University External Evaluator • Dr. Michael Sharpe, University of Minnesota NLCSD 3+2 6.21.11 OSEP #H325V090001 12
Objectives NLCSD has four major objectives which guide its work… • Produce 28 doctoral leaders in Blindness/Visual Impairments (B/VI), Deafness/Hard of Hearing (D/HH), & Deafblindness (DB) • Create and utilize a consortium to provide participatory guidance in the design and implementation of a multi-university, doctoral-level leadership preparation program in the special education sensory disabilities fields of Blindness/Visual Impairments (B/VI), Deafness/Hard of Hearing (D/HH), & Deafblindness (DB) NLCSD 3+2 6.21.11 OSEP #H325V090001 13
Objectives (cont.) • Develop and implement a Core Curriculum to be known as the “Research-Based Enrichment Program” using various modes of technology, deemed to be appropriate by the University Consortium, PAC, Enrichment Committee, Technology Committee and Leadership Team with input from the NLCSD Fellows, such as Blackboard, Wiki, Webcasts, Webinars, and Face-to-Face meetings to deliver research-based forums. • Create an inclusive online “Community of Learners” (CoL) composed of professionals, individuals with disabilities, and parents who are involved in the education of students who are D/HH, B/VI, and DB. NLCSD 3+2 6.21.11 OSEP #H325V090001 14
Objectives By meeting NLCSD’s objectives… • The quality and quantity of leadership in low incidence sensory disabilities will be greatly augmented • Collaboration and efficiency with which research informs practice in relation to teacher preparation will be increased, and • The academic performance of children who are D/HH, B/VI, and DB will be enhanced NLCSD 3+2 6.21.11 OSEP # H325V090001 15
Barriers Encountered Not necessarily barriers…but “Major Project Modifications” • Establishing a collaborative effort • Unanticipated changes in key personnel • The establishment of two Cohorts • Changes in the number of Fellows accepted • Changes and expansion of the Research Enrichment Program • Encouraging participation in the Enrichment Program • Expansion of communication systems and strategies • Evaluation activity changes NLCSD 3+2 6.21.11 OSEP #H325V090001 16
Activities & Major Accomplishments The Absolute Priority established by OSEP under which NLCSD was funded specifically delineated that • …to meet the requirements of this priority, the project, at a minimum, must conduct seven broad activities… These seven activities serve as the organizer for discussing the project’s accomplishments to date • Within the RFA, each activity was originally listed as a single narrative, in its entirety • For ease of analysis, reporting and review the larger activities have been organized into smaller, more discrete activities NLCSD 3+2 6.21.11 OSEP #H325V090001 17
Activity 1 • Establish a Consortium, comprised of IHEs that prepare students to work as doctoral leaders in D/HH, B/VI and DB • Obtain OSEP approval for its members Accomplishments • The Consortium was established and is comprised of • 24 IHEs with on campus leadership programs leading to doctorates • Members of the Leadership Team represent three additional IHEs bringing the total IHE representation the Consortium to 27. • 16 programs in B/VI, 13 in D/HH, and 8 in DB. • Some IHEs have doctoral programs in more than one area of sensory disability and therefore have more than one representative on the Consortium • OSEP approved recommendations and requested one additional member NLCSD 3+2 6.21.11 OSEP #H325V090001 18
Activity 2 • Establish policies and procedures for decision-making and the work of the Consortium • Create standard guidelines for awarding tuition and stipends to all students; submit proposed guidelines to the OSEP Project Officer for approval Accomplishments - Decision-making • Formal policies, procedures, and operating guidelines have been developed, reviewed and approved by the Consortium, including... • Decision making procedures • Fellow recruitment, application and selection processes • Measurement and reporting mechanisms for Fellows’ progress • Contingency planning in case of faculty loss. • All documents have been approved by the OSEP FPO • Additions or modifications continue to be made as needs are identified, or as informed by formative and summative evaluation activities NLCSD 3+2 6.21.11 OSEP #H325V090001 19
Activity 2 Accomplishments – Student Selection and Recruitment • Consortium committees established for • Recruitment/Marketing and Application Development • Application Review, Enrichment • Communication/Technology • Recruitment through announcements forwarded by Consortium and PAC members to various Listservs, websites, organizations, and colleagues • Application review was completed • Application Review Committee with balanced representation across all three disability areas • Scoring Rubric and point system developed and used • Conference call paneling with all committee members present • Leadership Team present on paneling calls for consultation and assurance that the process adhered to OSEP requirements NLCSD 3+2 6.21.11 OSEP # H325V090001 20
Activity 2 Accomplishments -Tuition and Stipends • Tuition paid based on respective university’s current tuition rate • Fees are not paid unless university includes fees in the cost of tuition • Some include all fees, some include partial fees, and others do not include any fees in their tuition • Tuition is paid directly to the university by NLCSD; Fellows do not receive a tuition bill • Living stipend provided to each Fellow • Amount of the stipend is individually determined based on local cost of living indices • Stipends are paid directly to the university for disbursement per their policy • Stipends awarded range from a $20,000 to $22,000 per year. • NLCSD reimburses all travel expenses incurred for NLCSD related travel based upon Salus University’s (the NLCSD fiscal agent) travel policies. NLCSD 3+2 6.21.11 OSEP #H325V090001 21
Activity 2 Accomplishments -Measurement and Reporting of Student Progress • University coursework and progress towards degree completion • Assessed by respective IHE • Research Enrichment Program • Participation is assessed for each topic area by the faculty member who leads that particular section using the NLCSD Enrichment Class Checklist • Other requirements • Completion of their university’s IRB training tracked by the Project Director • Participation and attendance in required F-2-F activities tracked by the Project Director • Each IHE reports to the Project Director on • The Fellows’ academic standing, GPA, each quarter/semester NLCSD 3+2 6.21.11 OSEP #H325V090001 22
Activity 2 Accomplishments - Contingency Planning in Case of Key Personnel or Consortium Faculty Losses • Leadership Team Loss • Confer immediately to discuss potential modifications and responsibilities • One team member, or committee member, attends to other matters that may be more critical than the project • Other team members step in and provide the time and assistance as needed. • Consortium Faculty Loss • Consortium agreement is between Salus and the Consortium member IHE, not the individual faculty • If unable to participate in a face-to-face meeting, required to identify a proxy • When more than one faculty member at the university has direct responsibility for the low incidence disability area, may send an alternate faculty member • If Consortium member leaves an IHE, the new faculty member for that program would be asked to participate NLCSD 3+2 6.21.11 OSEP # H325V090001 23
Activity 3 • Develop and implement a single, common research-based curriculum for all Consortium students that aligns with each IHE Consortium member’s course of study and that reflects scientifically based practices in course syllabi. Accomplishments • Developed and implemented a single, common research-based curriculum reflecting scientifically-based practices with low incidence disabilities • Made available for review, comment, and approval by the Consortium, PAC and Fellows on the NLCSD Wiki and revised as indicated. • Designed to enhance doctoral Fellows IHE training by including • Preparation for being a researcher and faculty member engaged in teacher-training • Establishing a Community of Learners • Providing opportunities to interact with each other, other researchers, and PAC members NLCSD 3+2 6.21.11 OSEP 3H325V090001 24
Activity 3 Accomplishments: Overview of Enrichment Years 1 - 4 • Year 1 • Research in Low Incidence Populations • Year 2 • Development of a Research Topic • Year 3 • Grant Writing • Year 4 • The Professoriate NLCSD 3+2 6.21.11 OSEP 3H325V090001 24
Activity 3 Accomplishments: Year 1 Topics • Topic 1: Why Engage in Research? • Topic 2: Ethical Practices • Topic 3: Quality Indicators of Research in Special Education • Topic 4: Quality Indicators in Qualitative Research • Topic 5: Quality Indicators in Single Subject Research • Topic 6: Quality Indicators of Group Experimental Research • Topic 7: Quality Indicators of Descriptive and Correlational Research • Beyond Content • A variety of faculty (2-3) lead different sections • Students began initiating their own discussions and conference calls • Summer Face to Face will have two Fellows’ initiated workshops/presentations • Students began becoming competent in technology NLCSD 3+2 6.21.11 OSEP 3H325V090001 24
Activity 3 Accomplishments: Year 2 • Fellows will develop research questions and a rationale based on published work • Fellows will establish an ‘outside’ committee, with other consortium members, PAC members, other Fellows, OSEP personnel, and practitioners • Fellows will have their own wiki • Fellows will present their project to all Fellows (webinar or F2F) NLCSD 3+2 6.21.11 OSEP 3H325V090001 24
Activity 4 • Maintain a web site that meets government or industry-recognized standards for accessibility and that links to the Web site operated by the Technical Assistance Coordination Center (TACC) Accomplishments • Created and maintained • a NLCSD web site housed at Salus University • Several NLCSD Wikis • Linked to the web site operated by Technical Assistance Coordination Center (TACC) NLCSD 3+2 6.21.11 OSEP #H325V090001 25
Activity 5 • Establish and maintain an advisory committee to review the activities and outcomes of the Consortium and to provide programmatic support and advice throughout the project period • Submit the names of proposed advisory committee members to OSEP for approval within eight weeks after receipt of the award Accomplishments - Establishment of Public Advisory Council (PAC) • Established and maintained PAC • To review the activities and outcomes of the Consortium • To provide programmatic support and advice throughout the project. • Initial meeting in February 2010; next meeting scheduled for July 2011 NLCSD 3+2 6.21.11 OSEP #H325V090001 26
Activity 5 Accomplishments- PAC representation • PAC membership recommended to and approved by OSEP • Organizations, agencies, and associations that represent the interests of persons with sensory disabilities • 8 representing blindness and vision impairments • 8 representing deafness and hard of hearing • 4 representing deafblindness • 3 additional (NASDE, CEC, HECSE) • 4 persons with sensory disabilities • 4 Parents of those with a sensory disability • Professional organizations representing the interests and work of faculty and administrators of IHEs • Fellows • Not selected when the first Consortium and PAC meetings were held • Fellows, since their selection, are involved in PAC activities NLCSD 3+2 6.21.11 OSEP #H325B090001 27
Activity 6 • Communicate and collaborate, on an ongoing basis, with other OSEP funded projects, and the Technical Assistance and Dissemination (TA&D) Network Centers • Access and use research based strategies and promising practices that are disseminated by personnel preparation centers and TA&D Network Centers Accomplishments • Ongoing collaboration • NLCSD is included in the TACC (TA&D Network Center) website • Established associations with NCLVI, NCDB and the HK Fellows project • Ongoing training activities occur with NCDB • TA&D network planning • Initial contact has been made with the Director of TACC and planning continues • No previous precedent upon to which frame these collaborative activities and discussions will occur during the next year. NLCSD 3+2 6.21.11 OSEP #H325V090001 28
Activity 7 • Maintain ongoing communication with the OSEP Project Officer through monthly phone conversations with the grantee and Consortium members and email communication Accomplishments • With FPO • FPO is active member of NLCSD’s Leadership • FPO involved in Leadership Team discussions and decisions • Monthly conference calls among the NLCSD Team Leadership were found to be insufficient • Weekly conference calls were initiated. • FPO included on all Wikis (except the Fellow’s Wiki which is private to them) • FPO copied on all critical e-mails • E-mails, briefings and other correspondence that are disseminated to all stakeholders are first reviewed and approved by the FPO prior to dissemination NLCSD 3+2 6.21.11 OSEP #H325V090001 29
Activity 7 Accomplishments • With Others • Four Wiki websites that contain documents and discussions • The primary Wiki is open for membership from all, including the public • Three private Wikis: 1) Fellows only; 2) Leadership only 3) Enrichment Wiki • Blackboard has been developed and houses • the Enrichment Program including all syllabi, project guidelines and requirements • research publications • separate discussion sections for different sections of the syllabi NLCSD 3+2 6.21.11 OSEP #H325V090001 30
Activity 7 Accomplishments • With Others • Public website, established within a week after funding, includes • NLCSD Home, Announcements & News • Committees, Staff and Fellows • Public Advisory Council and University Consortium lists • Recruitment, Application Process, Application, • Briefings and additional documents such as Mission, Objectives, Proposal, Continuation Reports, Meeting Agenda, Meeting Minutes, Documents, Enrichment Program, Enrichment Documents, NLCSD Wiki, Evaluations, Externship Opportunities, Meetings, Professional Opportunities, Remembering Leaders, Research Interests, and Resource Links NLCSD 3+2 6.21.11 OSEP #H325V090001 31
Highlights of Project Evaluation Activities Overview of the Evaluation Plan • Formative evaluation focuses on ways of improving and enhancing NLCSD. Asks: How can this program be improved? • Summative evaluation aimed at determining essential effectiveness of NLCSD. Asks: Should this program be funded? Data Sources • Evaluation designed to collect information from … • NLCSD Leadership Team • NLCSD Fellows • NLCSD Consortium Members Methods of Analysis NLCSD 3+2 6.21.11 OSEP #H325V090001 32
Major Findings: Objective 1 Objective 1: Produce 30 doctoral leaders in Blind/Visually Impaired (B/VI), Deaf/Hard of Hearing (D/HH), and Deafblind (DB) • Enrollment data obtained from the NLCSD Leadership Team shows that 28 individuals have been selected as NLCSD Fellows in the areas of B/VI, D/HH, and DB. • NLSCD is obligated to ensure a minimum of 24 Fellows complete their respective programs. Thus, if all current Fellows are retained in their respective programs, NLCSD would meet this objective. NLCSD 3+2 6.21.11 OSEP #H325V090001 33
Major Findings: Objective 1(cont.) • Evaluation Question: Are sufficient numbers of ethnically, linguistically, and experientially diverse students recruited and admitted to NLCSD doctoral programs? NLCSD 3+2 6.21.11 OSEP #H325V090001 34
Major Findings: Objective 2 Objective 2: Create and utilize a consortium to provide guidance in the design and implementation of a multi-university, doctoral-level leadership preparation program in the special education sensory disabilities fields of D/HH, B/VI, and DB. • The NLCSD Consortium currently consists of 24 colleges and universities with collaborative faculty representing 32 degree areas. NLCSD 3+2 6.21.11 OSEP #H325V090001 35
Major Findings: Objective 2 (cont.) • A Public Advisory Council (PAC) was established to review and provide feedback to the NLCSD Leadership Team. The PAC consists of 23 membership organizations. NLCSD 3+2 6.21.11 OSEP #H325V090001 36
Major Findings: Objective 3 Objective 3: Develop and implement a Core Curriculum to be known as the Research-Based Conceptual Model using an online Wiki-based research forum In the 2010-2011 academic year, 7 NLCSD Enrichment Course topics were conducted to supplement and complement research methodology courses taken by Fellows NLCSD 3+2 6.21.11 OSEP #H325V090001 37
Major Findings: Objective 3 (cont.) Findings of NLCSD Course Enrichment Topics Survey • Fellows consistently “Agreed” or “Strongly Agreed” that topic content promoted relevant and useful learning and reading materials matched learning objectives. • Similar findings were observed with regard to Fellows’ ratings regarding whether learning objectives were relevant and the extent to which information learned would be useful in other courses and/or work environments. • Fellows showed high levels of agreement that opportunities were provided to them to share learning resources and the content was available in a manner that met their accessibility needs. NLCSD 3+2 6.21.11 OSEP #H325V090001 38
Major Findings: Objective 3 (cont.) Findings of the NLCSD User Experience Survey • A high level of agreement among Fellows was observed on items which probed usability issues (whether typefaces were easy to read, visual contrast of text, etc.). • Fellows strongly agreed that the NLCSD Blackboard allowed for social networking opportunities. • A strong consensus among Fellows was found with regard to their ability to obtain needed technical assistance and support. • Less agreement was observed on items involving the assessment of learning outcomes and the extent to which the NLCSD Blackboard could be navigated with ease. NLCSD 3+2 6.21.21 OSEP # H325V090001 39
Major Findings: Objective 3 (cont.) Findings of End of Event Evaluation • Fellows consistently indicated various components of the 2010 NLCSD Fellows Meeting to be highly useful. • Fellows indicated they were able to identify a wide variety of “take home” ideas, ranging from increased opportunities for networking and collaborating as well as opportunities for obtaining new ideas in the pursuit of research topics. • Fellows perceived value in increasing time spent with members of the NLCSD Leadership Team to engage in interactive dialogue and an exchange of ideas. NLCSd 3+2 6.21.11 OSEP #H325V090001 40
Major Findings: Objective 4 Objective 4: Create an Inclusive Online Community of Learners (CoL) Findings of the NLCSD Enrichment Class Checklist • Results show that NLCSD Fellows meet performance criteria with regard to participation in discussion boards, responding to posts, and completion of assigned readings. • Most Fellows meet criteria with regard to their ability to demonstrate evidence of understanding and critical thinking skills, clear writing skills, and making a contribution to the CoL. • Data collection activities with regard to collaborative research activities will occur in project years 3-4. NLCSD 3+2 6.21.11 OSEP #H325V090001 41
Major Findings: Objective 5 Objective 5: Activities Conducted by the NLCSD Consortium are Reported to be of High Quality • Mean rating of 4.73 (on 5-point scale) was observed when Fellows were asked to rate the “Quality of the information provided” at the November 2010 NLCSD Fellows Meeting. • Based on aggregated results obtained on six NLCSD Course Enrichment Topics surveys, it was found that 39% of the Fellows selected “Strongly Agree” and 52% indicated “Agree” to the item “The readings and discussion for the topic(s) were of high quality.” Nine percent of Fellows (9%) selected “Disagree,” and none chose “Strongly Disagree.” NLCSD 3+2 6.21.11 OSEP #H325V090001 42
Major Findings: Objective 6 Objective 6: Activities Conducted by the NLCSD Consortium are Reported to be of High Relevance • Mean rating of 4.91 (on 5-point scale) was observed when Fellows were asked to rate the item “Relevance of the information provided” at the November 2010 NLCSD Fellows Meeting. • Based on aggregated NLCSD Course Enrichment Topics Survey, it was found that 60% of the Fellows selected “Strongly Agree” & 40% indicated “Agree” to “Topic objectives were relevant.” No Fellow selected either “Disagree” or “Strongly Disagree.” • When asked to rate whether “Topic discussions promoted relevant and useful learning,” 50% of the Fellows indicated “Strongly Agree,” while 40% selected “Agree,” and 10% indicated “Disagree.” NLCSD 3+2 6.21.11 OSEP #H325V090001 43
Major Findings: Objective 7 Objective 7: Activities Conducted by the NLCSD Consortium are Reported to be Highly Useful • Mean rating of 4.30 (on 5-point scale) was observed when Fellows were asked to rate the item “Usefulnessof the information provided” of November 2010 NLCSD Fellows Meeting. • Based on aggregated results obtained on surveys from 6 NLCSD Enrichment Course Topics: 44% of the Fellows selected “Strongly Agree” and 56% indicated “Agree” on “The objectives within this topic will be useful to me.” No Fellows selected “Disagree” or “Strongly Disagree” for this item. • In rating of the item, “Topic content promoted relevant and useful learning,” 55% Fellows indicated “Strongly Agree” and 45% indicated “Agree.” No Fellow selected either “Disagree” or “Strongly Disagree. NLCSD 3+2 6.21.11 OSEP #H325V090001 44
Why Project Director(s) Believe Project to be Significant and/or Effective Results of SWOT Analysis • Communication and commitment of project staff • Overall consortium structure • Enrichment coursework NLCSD 3+2 6.21.11 OSEP #H325V090001 45
Evidence of Project Effectiveness Results by Major Project Objective • Objective 1: 28 Fellowswereselected over 2 cohorts and the initial 12 members of Cohort I have been participating in project activities throughout the 2010-2011 academic year. • Objective 2: NLCSD has assembled an array of Consortium members representing some of the most highly regarded research IHEs in the nation. Also, the NLCSD Public Advisory Council provides input and feedback on project activities. • Objective 3: A series of NLCSD Enrichment Course topics have been developed and implemented. Evaluation evidence regarding effectiveness relative to learning outcomes has been obtained. • Objective 4: The NLCSD project has developed an infrastructure for the CoL through the establishment of Enrichment Course offerings via the Blackboard and Wiki. NLCSD 3+2 6.21.11 OSEP #H325V090001 46
Evidence of Target Audience Satisfaction • Evidence of audience satisfaction reflected on The End of Event Surveys, NLCSD Enrichment Class Checklist, NLCSD Research Enrichment Course Survey, and the NLCSD User Experience Survey. • Evidence of audience satisfaction reflected in GPRA measures of quality, relevance, and usefulness. NLCSD 3+2 6.21.11 OSEP #H325V090001 47