September 30, 2014 Dawn Cullity Bernadette Ferro Kent Phillippe

1. September 30, 2014 • Dawn Cullity • Bernadette Ferro • Kent Phillippe VFA Year Two: Show & Tell

2. VFA Overview Then The Voluntary Framework of Accountability is the first national system of accountability specifically for community colleges by community colleges. Now

3. AACC’s Objectives for the VFA • Improve the user experience • Increase utility (so we can meet all the goals listed here) • Value-added metrics for accountability (using data to measure performance and make better decisions) • National benchmarking for good practices • Opportunity to streamline reporting / relieve IR capacity • Ability to tell a better, more robust story • Better data to inform public policy and advocacy User Experience Utility

4. Objective: Improve Data Input • Feedback on the user set-up and data collection • process/data tool • Stability of, and access to, the online collection and display (dashboards) • Tool is clunky, especially the online forms, and needs better validation processes • Workflow is not intuitive and progress bars does not tell us where we are in data collection process feedback improvements

5. Workflow: Data Configuration Submit

7. Workflow: Managing Data Collection

8. VFA Manual Data Entry

9. VFA Cohorts Disaggregated

10. Objective: Increase Utility of VFA Exports • Feedback on the use of the VFA and its data • Dashboards need improvement to be useful and understandable • Colleges need to be able to download final values including data and calculated percentages • Benchmarking parameters and policies • Dashboards need a lot of interpreting before passing to senior leadership/board feedback improvements

11. VFA Export: Outcomes Dashboard

12. VFA Export: Tabular Data

13. VFA Benchmarking Dashboard: Benchmarking Selectors

14. VFA Benchmarking Dashboard: Six-Year Outcomes

15. VFA Export: Public Dashboard Report

16. Objective: Improve Data Collection Resources • Feedback on the resources (instructions) • Clarity: make the Metrics Manual user friendly, understandable • Consistency • Authority: be definitive with methodology and how metrics are to be reported feedback improvements

17. Reporting VFA Six Year Cohort Developmental Education Progress Measures Two-Year Progress Measures Six-Year Outcomes Measures Cohorts Two Year Cohort Main Credential Seeking First Time In College Measures CTE Cohort Career & Technical Education Measures Adult Basic Education Measures ABE Cohort

18. VFA Metrics Manual

19. VFA Metrics Manual (cont.)

20. VFA Overview & Training Webinars • http://vfa.aacc.nche.edu/webinars/Pages/default.aspx

21. Close-Out • What do you think should be our priorities going forward? • Questions & Feedback • Contact VFA at VFA@aacc.nche.edu Y2 , Y3, Y4… Reminders • Sign-ups open October 1st • Data collection opens November 3rd principle accountability system

22. Questions During the Webinar Data Dashboards • Have the dashboards been tested with IE, Firefox, and Chrome? • The VFA dashboards are supported on the following Internet browsers: • Internet Explorer (IE 8 – IE 10) • Firefox (16.x – 22.x) • Google Chrome (22.x – 28.x) • Please clarify public vs private dashboards. • Upon publishing your VFA data, the VFA makes a link available on your College Profile page that takes the public user to your Outcomes data. The Outcomes data are presented to the user in a packaged PDF report, again made available on a link from your College Profile page. All information that is contained in your Outcomes Dashboard is packaged into this public PDF report. The Outcomes data includes the “All Students” (not disaggregated) counts of the following: Developmental Education Progress, Two-Year Progress, and Six-Year Outcomes Measures; CTE Profile & Completer/Leaver Outcomes, and ABE Cohort Outcomes. Raw data, disaggregated data, and the benchmarking tool and data will not be available to the public. We will send the user group a sample in the next few weeks. • Will the new dashboards show data from previous years’ data collections? • Yes. The new Outcomes and Benchmarking dashboards will render all data from Year One (2013-14 data collection) onward. • The Public Dashboard Report – a PDF report showing the graphs or your published Outcomes data –is only available for VFA Year Two (2014-15 data collection) ONWARD. To the public, the link to this report will always be for the most recent year. • Can we view the names of benchmarking institutions selected? • Currently, the tool does not return a list of the colleges that are in the benchmarking group. AACC is working with the VFA Oversight Board and the dashboard developers on both the policy and technical changes needed to make this happen. We have receive this request and are working toward fulfilling it. • Please note that the benchmarking tool will always use selector criterion to establish the colleges in the benchmarking group; colleges will not be able to pick individual colleges against which to benchmark. Questions & Answers

23. Questions During the Webinar (cont.) Raw File Upload Process • More info about the "raw data" upload • The Raw Data upload process requires the college to upload 5 files: • Student demographics, • Student developmental education placement, • Student course data, • Completions at two years, and • Completions at six years. • The data will be extracted, translated and loaded into the VFA database. • After the ETL is completed, the data from the five files will be permanently deleted from the system. The VFA does not keep student-unit record data. • If the raw data upload method will become the "method of choice," will the bulk upload method eventually be abandoned? • The VFA has no plan to abandon any of the three data input methods at this time. We anticipate that many users will move toward the raw upload method because it is a process where data extraction is streamlined and methodology is preserved. Like all data initiatives, though, we understand that users work differently and have preferences for how to enter data. Questions & Answers

24. Questions During the Webinar (cont.) Misc. • What data can be downloaded to Excel? • All data that you entered for your college can be downloaded to an Excel document, so that the college may manipulate their own data. Downloading benchmarking data requires more development and is not available for Year Two. • How is the VFA handling the "English/Reading" combination topic going forward? • For Year One, the VFA allowed colleges who offered English and reading under a combined topic to report the progress and outcomes of these students under an English/Reading combination subject. There was confusion and errors in reporting, so for Year Two we ask all colleges who offer the combined subject to report it under the English subject. For these colleges, the reading subject area will be null. • When will the new versions of the VFA Metrics Manual and User Guide be released? • There are aspects of the Metrics Manual and User Guide that require all other data collection development to be completed. Therefore, the VFA Metrics Manual will be released on November 3rd, when data collection opens. The User Guide will be released soon after. • Is there an easy way to report SAM (Student Achievement Measure) using VFA? • The data reported for SAM’s associate/certificate model is a roll-up of the VFA’s Six-Year Outcomes for the full-time and part-time, credential-seeking cohort. If a college reports the VFA, it can easily report SAM. The SAM folks have been working with their developers on a file that helps with getting VFA data into SAM. We will update you on this post-October 20 when SAM launches. See the Resources section of the SAM website for more information: http://www.studentachievementmeasure.org/resources# Questions & Answers