Accountability and Grants Management: Connecting the Dots

1. Accountability and Grants Management: Connecting the Dots PROGRAM MANAGEMENT AND ACCOUNTABILITYRequirements and Considerations Across Discretionary Grantees Day One Plenary Session—Presentation U.S. Department of Labor, ETA, Region 4 Discretionary Grantee Training Conference March 1-2, 2011

2. Objectives Establish a foundation for program management and related accountability requirements, irrespective of the source of the requirements or the employment and training program funded Identify gaps in current plans, policies or operations and discuss ways to address

3. Approach (1)

4. Approach (2) 4

5. ConsiderationsAs You Develop Your Accountability Framework… • Definitions—Don’t believe everything you think (a/k/a Webster’s dictionary doesn’t always apply) • Make sure you’re following all required parameters • You need systems to capture, maintain, and report all the information • Policies, Procedures, Protocols • When there are partners, sub-awards, contractors . . .

6. Who you’re serving…General Grantee Requirements* All grantees need policies and procedures to ensure compliance with all federal parameters governing who can participate in the program *Although this section applies to grantees serving participants, policies and procedures to support operations is critical across grantees and organizations. 6

7. Who you’re serving…Federal Parameters Citizenship Selective Service Veteran Status Ethnicity, Race, Disability Status SSN Characteristics (grant-specific although many are common such as socioeconomic data, age, employment status at entry, level of education)

8. Who you’re serving…Data Sources and Documentation Sometimes have same meaning / not always Sometimes specified, or minimum requirements provided with details left up to grantees Some grantees assume “documentation requirements”that don’t exist What does your _________ say? 8

9. Who you’re serving…Citizenship Requirements “Participation in programs and activities … shall be available to citizens and nationals of the United States, lawfully admitted permanent resident aliens, refugees, asylees, and parolees, and other immigrants authorized … to work in the United States.” WIA, Sec. 188(a)(5) Documentation could include _________________ or ___________________ (formerly “________ card”) Do you currently capture this information? How?

10. Who you’re serving…Selective Service Requirements “The Secretary shall ensure that each individual participating in any program or activity… has not violated section 3 of the Military Selective Service Act (50 U.S.C. App. 453) by not presenting and submitting to registration as required…” WIA, Sec. 189(h) Registration required for all males born on or after 1/1/60 within 30 days of 18th birthday with late registrations accepted until age 26

11. Who you’re serving…Selective Service Requirements (2) From www.sss.gov – “Some men may have failed to register during the time they were eligible to do so and may now find they are ineligible for certain benefits.” “The Workforce Investment Act (WIA)…. is only open to those men who register with Selective Service.” Documentation could include __________________ ___________________ noting the person registered See Handout #1 11

12. Who you’re serving…Veterans’ Priority of Service Requirements “Grantees are required to provide priority of services for veterans and eligible spouses pursuant to 20 CFR part 1010, the regulations implementing priority of service for veterans and eligible spouses in [DOL] job training programs under the Jobs for Veterans Act…” Text from an SGA (ETA policies also referenced) “New” policy – Training & Employment Notice (TEN) 15-10 dated 11/10/10 contains implementation protocol See Handout #2

13. Who you’re serving…Veterans’ Priority of Service Requirements (2) Basically, __________has to be eligible and, of those, “veterans and eligible spouses” receive priority ______ grantees must collect veterans’ characteristics and information to determine if priority of service (POS) applies Documentation could include letter from veterans’ organization ______________ ok until documentation received 13

14. Who you’re serving…Non-Discrimination / Equal Opportunity Grantees must collect information on Ethnicity, Race, Sex, Age, Disability Status Requirements appear in many places including 29 CFR Part 37, Civil Rights Act of 1964, etc. Purpose of mandatory data collection is to ensure equal opportunity Data collected on participants and applicants You have to ask but they don’t have to tell Are your forms properly designed? Documentation __________________ unless otherwise stated since info is based on __________ ____________ See Handout #3

15. Who you’re serving…SSNs Collecting SSNs is usually required through the SGA and/or reporting instructions Example: “Applicants will be required to collect participants’ Social Security numbers as part of individual level data collection…. Applicants must ensure that Social Security numbers will be maintained in a secure and confidential manner.” (ARRA HGEI 6/24/09 SGA) Needed to calculate common measure outcomes

16. Who you’re serving…SSNs (2) Individuals cannot be ____________ ____________ for failure to provide a SSN Documentation could include ________ _________ _________ or military ID Do any of your policies or procedures [even unintentionally]require a SSN? Do your intake forms/documents unintentionally discourage the provision of SSNs? See Handout #4 16

17. Who you’re serving…Characteristics Could include: Age Dropout status LEP Ex-offender Economically disadvantaged Documentation can be specified in SGA and reporting instructions if populations are targeted Or sometimes a statutory reference is implied (e.g., WIA defines adult as 18+, a target for CBJT grantees) See Handout #5 17

18. Who you’re serving…‘Housing’ Considerations for the Data “An applicant must collect and reportparticipant-level data from the following categories: Demographic and socioeconomic characteristics; Services provided; and Outcomes achieved.” This text is in every SGA involving the enrollment of participants So what should grantees do with all the data that’s collected? 18

19. Who you’re serving…‘Housing’ Considerations for the Data (2) Participant data should live in both case files and the MIS Establish policy, procedures, protocols for case files (e.g., __________?) Case files serve as a repository for information and should be an ______________ of your MIS Example: Case file contains copy of certificate whereas MIS just notes certificate was attained Note: Grantees submit reports containing aggregate numbers but they’re based on ___________ ___________ 19

20. Who you’re serving…Connecting the Dots Do you have a written eligibility policy, associated forms (e.g., consent forms, intake forms) and protocols in place (especially if there are subs) that incorporate all of the foregoing? Are there gaps that could lead to compliance issues? Where are there opportunities for improvement or refinement? Would you be comfortable with a monitoring visit from your FPO next week? Sample/Handout #4 20

21. What they’re getting…General Grantee Requirements Grantees are responsible for program design and delivery—including the design and delivery of services—within federal parameters that are general as well as those that are grant-specific.

22. What they’re getting…Federal Parameters Participants Only Services (and/or Deliverables) Scope, Complexity ranges across grantees Terms / Definitions Management of / Coordination with subs, providers, partners

23. What they’re getting…Data Sources and Documentation “The applicant must collect participant-level data on . . . employment training and other services provided through the grant.” Text from an SGA 23

24. What they’re getting…Participants Only Participant—Individual determined eligible to participate in the program and who receives a service funded by the program (TEGL 17-05) ‘Qualifying’ Services – Know the services that trigger participation Example: Eligibility determination by itself does not trigger participation Participation Date = Date of _________ __________

25. What they’re getting…Scope/Complexity of Services Generally speaking, the more complex the scope or range of services, the more “controls” you need Key Consideration – Systems Needed Would the decision-making process be easier, more fair, and/or more consistent if we had a policy in place?

26. What they’re getting…Scope/Complexity of Services (2) Policies – Administrative tools describing what has to be done/why to help staff attain organizational goals Procedures – Actions comprising the acceptable manner of performance so policies are consistently implemented Steps 1—Define the issue, task, problem; 2—Determine responsibility for development; 3—Develop draft using agreed-upon template; 4—Review/revise/make final; 5—Adopt/approve; 6—Implementation (distribution and education); 7—Regular review (eventual archival) 26

27. What they’re getting…Service Terms/Definitions Sometimes, services that are allowable are specified as unallowable in the SGA Remember that Webster’s dictionary is not always applicable Example: “Received Education or Job Training Activities” means something different for CBJT grantees, HGEI grantees, and YouthBuild grantees

28. What they’re getting…Management/Coordination Critical Generally speaking, the more complex the arrangement of partners, subs, providers, the more ____________ you need Key Consideration – __________ Needed Navigating relationships is hard under any circumstances (add distance, varied experience, different understandings, existing “controls”) If you don’t say what you mean, people will interpret it for you 28

29. What they’re getting…Connecting the Dots Do you have a ‘participant flowchart’ or similar service design/delivery framework? Are the services provided consistent with the SOW (e.g., sequence, definitions, parameters)? Are protocols in place (especially if there are subs) that incorporate all of the foregoing? Are there gaps that could lead to compliance issues? Where are there opportunities for improvement or refinement? Would you be comfortable with a monitoring visit from your FPO next week? 29

30. With what results…General Grantee Requirements All grantees need a management information system that, at a minimum, captures required data, performs necessary calculations, and reports information needed to manage the grant and meet federal reporting requirements.

31. With what results…Federal Parameters Exiters Exit vs. Completion Outcomes Terminology; Common Measures; Other Measures Unexpected Circumstances Gap in Service; Performance Exclusions Quarterly Program Reporting Management of / Coordination with subs, providers, partners

32. With what results…Data Sources and Documentation “An applicant must collect and report participant-level data [including] outcomes achieved.” “Selected applicants will have to include projections and track such information as ‘total number of participants who complete employment and training activities that receive a degree, certificate, or other type of credential’.” Typical text (generally within the same SGA) 32

33. With what results…Exiters Exiter – A participant who hasn’t received a program or partner-funded service for 90 consecutive days and is not scheduled for future services (TEGL 17-05) Exit Date is applied retroactively* to the last date of service (after 90 days of no service) Exit Date = Date of ________ _________ Exit must be understood by all grantees Represents a trigger (e.g., inclusion in common measure outcomes, beginning of youth follow-up) *Exception for YouthBuild grantees

34. With what results…Exit vs. Completion Exit is defined for you; Completion is usually defined within general parameters (grantees add the meat) Completion often means successful completion Everyone exits but not everyone completes (see your program reporting instructions)

35. With what results…Outcomes—TERMINOLOGY If it’s not clear, get clarity! You’re accountable “Completion” The one-time-only nature of completion often presents challenges Completion of entire program vs. completion of one ‘step’ or ‘course’

36. With what results…Outcomes—TERMINOLOGY (2) “Entered Employment” To be [the common measure] or not to be Example: CBJTs and HGEIs are accountable for two different “entered employment” outcomes Projected “Enrollments” Your projections were based on your understanding of enrollment at the time you wrote the grant, which may not have been correct 36

37. With what results…Outcomes—COMMON MEASURES All discretionary grantees are responsible for some common measure outcomes Adult Entered Employment Rate Adult Employment Retention Rate Adult Average Earnings Youth Placement in Employment/Education Youth Attainment of Certificate/Credential Youth Literacy/Numeracy ALL calculations are based on the Exit Date with exception of Youth L/N

38. With what results…Outcomes—OTHER MEASURES Note: Required program report formats will never fully account for what you have done with the funds you received! You could ‘stay within the box’ and still tell your story

39. With what results…Unexpected Circumstances (1) [Slide #33 said Exit is based on 90 days of no service…] – With one exception A participant should not be considered as exited if there is a gap in service greater than 90 days based on one of three circumstances: 1. Delay before beginning of training 2. Health/medical condition of participant or family member 3. Temporary move that prevents participation Grantees need to understand federal parameters regarding gaps in service See Handout #6

40. With what results…Unexpected Circumstances (2) Certain conditions allow individuals to be excluded from common measure outcomes Institutionalization Health/Medical or Family Care Death Reservist Called to Active Duty Relocation to Residential or Non-Residential Program (youth only) Exclusions apply if circumstance takes place at exit or in the 3 quarters following

41. With what results…Quarterly Program Reporting “Grantees must agree to meet DOL reporting requirements.”(SGA Text) OMB-approved Program reporting requirements vary across discretionary grantees Across grantees. . . What Do FPOs Look For? General Suggestions About Data Quality

42. What Do FPOs Look For? 1. _____________ – Reports due NLT 45 days after the end of the quarter (program and fiscal reports) 2. _____________ – All sections included, no pertinent information missing, referenced attachments are included 3. _____________ – cumulative numbers go up and not down, number of males and females equal total enrollments With what results…Quarterly Program Reporting (2)

43. General Suggestions Work on it _____________ the quarter Maintain easy access to your _______ As they occur, check off/crosswalk activities, products, and deliverables to your SOW Keep list of events, meetings and activities you would report to the leadership of your organization (e.g., college president) Schedule time before the date to enter data, talk with staff, etc. …and make sure your subs/partners do the same With what results…Quarterly Program Reporting (3)

44. About Data Quality Ensure data elements are well-__________ and ___________ by project staff (including contractors) Note: Original source of information is more reliable than secondary information (e.g., info on ‘training completion’ from a community college should be used rather than second-hand info from participants through case managers) Ensure duplication of effort does not occur in data collection (costly, lack of consistency) Procedures needed (e.g., SOPs with timeframes and deadlines, responsibilities assigned) With what results…Quarterly Program Reporting (4)

45. With what results…Connecting the Dots Do you have policies and procedures defining outcomes (ours and yours) and associated data elements and how the data are gathered, processed, reported (especially if there are subs) that incorporate all of the foregoing? Are there gaps that could lead to compliance issues? Where are there opportunities for improvement or refinement? Would you be comfortable with a monitoring visit from your FPO next week? 45

46. Conclusion When you start connecting the dots you soon realize that, practically speaking, you cannot separate program design from program delivery, or either of those from program management, or any of those from program accountability. At least not without something important falling through the cracks. 46

47. Accountability & Grants Management: Connecting the Dots THANKS! U.S. Department of Labor, ETA, Region 4 Discretionary Grantee Training Conference March 1-2, 2011