PRESENTATION TO NSA M&E CAPACITY BUILDING MEETING

1. PRESENTATION TO NSA M&E CAPACITY BUILDING MEETING 03 August 2017

2. STRUCTURE • Background • Progress and Work plan • Findings from Interviews • Introduction of Functions • Options analysis • Initial Reflections on Options • Next steps

3. BACKGROUND • One of the biggest challenges facing South Africa today is the high rate of unemployment • South Africa has the highest Gini coefficient in the world largely attributable to the economy’s inability to create jobs because of: • Inadequate skills level and poor work readiness of young people leaving formal secondary and tertiary education and entering the labour market for the first time. • The challenges facing the unemployed who lack basic numeracy and literacy skills, and who do not have the experience and training to enable them to obtain work • Skills shortages in the artisanal, technical and professional fields

4. Background • A tendency to focus on NQF level 1 to 3 learnerships, with little progression towards intermediate and higher skills levels • Failure of business to equip the workforce to adapt to change as the economy becomes more knowledge-based • Lack of synergy between various post school subsystems.

5. Background • The establishment of SETAs and introduction of the levy has had little impact on the economy because of a paucity of effective leadership for the sector and an institutional vacuum in the areas of: • ensuring better planning by the SETAs and other role players. Plans are often not informed by an understanding of socio-economic development imperatives of the country and of specific geographic regions. • encouraging coordination amongst the different role players in the skills development sector and with other economic development initiatives in order to support development efforts • evidence based policy dialogue and development in order to ensure ongoing adjustments of policy and implementation efforts.

6. Background • The White Paper for Post School Education and Training (WP-PSET) has expanded the role of the NSA by stating that ‘a restructured NSA will concentrate specifically on monitoring and evaluating the SETAs’. This implies that it will become an expert body with high- level monitoring and evaluation skills . • The NSA approached GTAC to assist with technical support for the development of a business case and implementation plan for its repositioning including a service delivery model, organogram, MTEF costing. • The M&E framework is one of the first deliverables that has been produced

7. Progress and Workplan Project Plan, Inception report Interviews, Document reviews Functions and Options Development PSC Presentation Close Out and Final report Test and select options

8. CHALLENGES IN SETA PERFORMANCE & M&E • Inability to address the skills needs of the sectors • Labour market research based on levy paying workplaces and not credible research (Limited ability for complex labour market research) • The setting of goals that are not always aligned to sector priorities and societal goals • Inadequate reporting against goals in annual reports • Focus on the achievement of output level numerical targets with little focus on sector impact • Inadequate focus on reporting on governance and financial performance

9. DIAGNOSTIC FINDINGS • The policy mandate for the repositioning of the NSA as a centre of excellence is well established through the White Paper on Post School Education and Training (2013). A closer look at the need for some legislative amendments will be required in order to give the NSA the required M&E mandate and authority. This will be guided by the form and extent of M&E work that the NSA will undertake. • The current absence of an internal dedicated M&E function within the NSA, beyond the secretariat role is a major constraint to the provision of effective M&E that leads to good policy advice. The organisational structure is also inadequate both for the current and envisaged functions (Director, Deputy Director, 5 support staff). The current structure performs mainly secretariat functions for the Board and is not geared towards M&E functions.

10. DIAGNOSTIC FINDINGS • Currently the work of the NSA is largely carried out through its Board sub-committees. Working through sub-committees is not effective, as the sub-committees may not have the necessary capacity and time as sub-committee members have full time jobs. • The board functions well in terms of attendance at meetings, but there needs to be more clarity regarding the role of the board as some queries have been raised on whether there is a need for a Board in a repositioned NSA.

11. DIAGNOSTIC FINDINGS • The NSA board is constituency based and is constituted by the same stakeholders as at NEDLAC. Whilst the powers and functions are the same, often different views are given in response to skills policies by the two structures, which makes the process inefficient. The legislation provides that all skills policy and legislation must be consulted within the NSA, and has a similar provision for NEDLAC. Often the legislation and/or policy passes through NSA without any challenges but when it gets to NEDLAC there are different approaches.

12. DIAGNOSTIC FINDINGS • Whist the constituency based board provides an ideal platform for consultation within the sector, a purely constituency based structure could be a constraint to the quality of M&E and independence of advice that the NSA is able to provide to the Minister. There may thus be a need to relook at the structure of the board in order to provide for an injection of M&E expertise. • Part of the M&E function will require that the NSA reviews SETA board performances. This may cause a conflict of interest as some Board members are also Board members of the SETA boards and therefore may not be impartial in their exercise of M&E.

13. DIAGNOSTIC FINDINGS • Regarding the business model, the NSA in its M&E role needs to explore an approach that does not require the setting up of a big data gathering, monitoring and evaluation bureaucracy. Rather it should explore a partnerships model where it relies on existing databases for data sources and works in close collaboration with the DHET Skills Branch, which has a SETA monitoring function as one of its functions. Rather than repeating the work that the Skills Branch does, the NSA could focus on monitoring SETA board performance, financial performance and assessing impacts. In order to implement such a business model, the NSA will requires some internal capacity.

14. DIAGNOSTIC FINDINGS • Partnerships should also include working with other entities that are responsible for monitoring and evaluating different sub-systems of the education sector. These sub-systems include TVET, Universities and the basic education. Partnerships with the private sector and other private and public stakeholders on the skills demand side will enable the NSA to achieve a broader, whole system perspective of education provision to inform its M&E role. • The implementation of the envisaged changes will require serious resource commitment to the NSA.

15. DIAGNOSTIC FINDINGS • The NSA should strive to be more visible. Currently it is considered by others to be invisible. The organization should set a clear agenda for its operations including the production of papers and convening of seminars to discuss topical issues in skills development and to disseminate its M&E findings. This could include getting involved in research. It should take the lead in facilitating the identifying of best practice in M&E amongst SETAs and using that to facilitate sharing and learning by the other SETAs, playing the role of a learning hub for skills development.

16. DIAGNOSTIC FINDINGS Areas for M&E Focus • A view that the NSA should not only limit its M&E work on the SETAs but should look at the whole Post School Education and Training System. • The NSA could focus only on evaluation and rely on the Skills Branch whish is already collecting monitoring information from SETAs • There is a view that most effective route to take would be for the DHET to develop a central M&E system with the Department applying oversight over it and the NSA playing an evaluation role.

17. WHAT IS M&E Key conceptual issues on monitoring and evaluation • Monitoring and evaluation, whilst often linked together are integral but individually distinct parts of programme and project implementation. It is important to distinguish between monitoring and evaluation as this will have an influence on the future role of the NSA. Monitoring can be defined as the ongoing process by which stakeholders obtain regular feedback on the progress being made towards achieving their goals and objectives. • Evaluation is a rigorous and independent assessment of either completed or ongoing activities to determine the extent to which they are achieving stated objectives and impact, contributing to decision-making. An evaluation should provide evidence-based information that is credible, reliable and useful. It is also an important contributor to building knowledge and organizational learning

18. Proposed Functions for the NSA

19. PROPOSED FUNCTIONS

20. PROPOSED FUNCTIONS

21. PROSPOSED FUNCTIONS

22. PROPOSED FUNCTIONS

23. PRINCIPLES FOR INSTITUTIONAL FORM OPTIONS ANALYSIS

24. OPTION 1: NSA PHASE OUT • This option does not address the M&E challenges that have been raised by the sector • Overlooks the strategic and political role played by the NSA which revolves around stakeholder involvement and advice to the Minister on skills development. • Costs associated with the operations of the NSA would be eliminated Advantages Disadvantages Summary This option, whilst widely discussed by interviewees as it addressed the purpose and existence of the NSA, goes beyond the mandate of this project. The GTAC brief is to unpack the White Paper proposal that the NSA should reposition itself in order to conduct M&E of the SETA’s, and consider the implications thereof. However, the NSA should strive to be more visible. Currently it is considered to be invisible by relevant stakeholders

25. OPTION 2: GOVERNMENT COMPONENT • The M&E function might be duplicated with other units within the DHET; • NT not keen on establishing government components as this creates a “new” organisational form with additional costs. • Might result in the non- prioritisation of the institution in the DHET vote. • Difficult to fully integrate its M&E work with other units of DHET due to its relative autonomy. • The size and purpose of the M&E function does not warrant the establishment of a separate organisation. • The establishment of the GC might result in separation between the planning function in the department and M&E • Once established it will have the M&E mandate to carry out the M&E function • It will have authority as the Accounting Officer will report directly to the Executive Authority • It may be a more cost-effective when compared to the public entity option Advantages Disadvantages Summary The size and purpose of the monitoring and evaluation function of the NSA does not warrant the establishment of a separate organisation with its own Accounting Officer reporting directly to the Executive Authority.

26. OPTION 3: SCHEDULE 3A PUBLIC ENTITY • Would have the mandate for M&E over SETAs. Establishment however may require the enactment/amendment of enabling legislation to incorporate as a PE • It will have authority as the Accounting Authority will report directly to the Accounting Officer of DHET and to the Executive Authority • Would have full autonomy and authority to conduct M&E as a separate juristic person • Independent corporate and legal identity operating outside of the department but reporting to Parliament through the DHET/EA. • Decision making is assigned to the independent juristic person which can enhance public confidence in the decisions of the entity. • The size and purpose of the M&E function does not warrant the establishment of a separate public entity (in any form) outside of the public service. • Cost implications: As a PE, it would be costly to establish as it requires the creation of independent operational and governance functionalities. • As an independent body it would not be able to integrate its M&E work adequately with other units of DHET. • Requires establishment through legislation, reporting to a board, and the appointment of staff outside the Public Service Act. The NSA is essentially an advisory body, its M&E functions do not requires an independent entity outside the public service. • Degree of autonomy with decision making which could result in conflict between the Department and Board. Advantages Disadvantages

27. OPTION 3: SCHEDULE 3A PUBLIC ENTITY Summary The purpose, costs, and legislative requirements to establish a public entity outside of the public service are onerous and the M&E function does not warrant an institutional structure of this magnitude. In addition, an independent juristic entity responsible for M&E would make it difficult if not impossible for the NSA to integrate its work with the M&E work done by other units or branches of DHET

28. OPTION 4: SPECIALISED SERVICE DELIVERY UNIT • Will not be providing a specific customer focused service, but rather its service is for the DHET • Would create an arm’s length type of relationship with other key units in the DHET that play a role in M&E on SETAs. • The department may relegate all M&E functions to the unit • The department may ignore all reports from the unit. • The nature of the work of the M&E will not constitute direct services to be delivered to the end user. • Will have the mandate and authority through the delegation of the M&E function by the DHET Accounting Officer • It will have authority as the Accounting Officer will report directly to the Executive Authority • Will not be as costly as a Public Entity • A feasibility study is mandatory and the powers, responsibilities and duties of the unit are contained in a protocol between the executive authority and the head of the unit Advantages Disadvantages Summary A Specialised Service Delivery Unit may work but does not seem an appropriate organisational form as the nature of M&E work does not constitute direct services to be delivered to the end- user/consumer and therefore may not qualify as a direct service The M&E functions anticipated are more strategic and policy-orientated.

29. OPTION 5: RELOCATE WITHIN DHET • Since the White Paper does not clearly define the M&E functions, it will provide an opportunity to define these and ensure that no functions are being duplicated • Capacity for M&E would be clearly identified and enhanced within the DHET • The most cost-effective option • Provides an opportunity for M&E work to become more fully integrated in the DHET • Would not address most of the challenges that have been identified in the diagnosis, especially the evaluation challenges. • NSA has limited resources (both human and budgetary) and will not be in a position to re-deploy the resources.. • The consequence could be that nothing changes and the challenges in M&E will continue. • There may be greater focus on monitoring and little on evaluation. • There will be less stakeholder buy-in as the NSA will not be directly involved but merely reliant on data from the department. • May further reduce the role and impact of the NSA Advantages Disadvantages Summary This option, whilst raised by many interviewees and deliberated upon by the team may not bring any innovation in dealing with M&E over SETAs. It would continue the status quo and most of the challenges highlighted in the diagnosis will not be addressed.

30. OPTION 6: EVALUATION FUNCTIONS AS A DIRECTORATE WITHIN THE NSA • Retains the strategic role of the NSA as a place for stakeholder consultation • Makes a distinction between monitoring and evaluation and the emphasis is on the evaluation role of the NSA. • Focuses the mandate of the NSA on impact evaluation, which is key for evidence based policy advice to the Minister • Will address most of the challenges identified in the diagnosis that the current focus is primarily on monitoring with very little work done on evaluation. • Is more cost effective than most of the other options. • Would not require too much organisational restructuring as positioning as a Chief Directorate is already the envisaged long-term structure of the NSA. • NSA evaluation work will be integrated into the work of the DHET • Would take advantage of and build on the existing monitoring function within the DHET Skills branch and will assist in avoiding the duplication of the monitoring function. • Will reinforce and encourage collaboration within the Department. • Will assist in deepening the dialogue and engagement between sectors on the impact of services provided; • The NSA will act as a repository for Knowledge Management on Monitoring and Evaluation. Advantages Advantages

31. OPTION 6: EVALUATION FUNCTIONS AS A DIRECTORATE WITHIN THE NSA • Would require Evaluation and policy analysis capacities that may be difficult to attract into a government department • Limited authority and autonomy. • May have a problem if the decision of the Board of NSA differs from the Department. • Stakeholders may not perceive the work and recommendations as independent and objective as it is located within the DHET • The necessary expertise to provide effective oversight may not be present in the Board • The option depends to a large extent on the quality of monitoring data derived from the DHET. Summary Disadvantages This option allows for focus and integration of the NSA’s M&E work with the DHET, and avoidance of duplication. It enhances the evaluation function over SETAs which is currently a major weakness. It will provide the NSA board with key evaluation insights that would improve its policy advice without creating too much distance between the NSA board and the DHET. The NSA/DHET should consider direct accountability to the Director General through an appropriate system of delegations to give more autonomy.

32. NEW OPTION (REQUIRES MORE EXPLORATION) : M&E FUNCTION AS PART OF SAIVCET To provide support and coordination to TVET and Community Colleges in terms of responsiveness and quality. Role of SAIVCET Advantages Disadvantages • Would avoid duplication of the M&E function as an autonomous M&E unit is already envisaged for SAIVCET • Would facilitate enhanced focus on TVET & CET Colleges • Would be seen as independent and objective. • Not unless the focus of SAIVCET is broadened, to go beyond TVET & CET, it might be too limiting to the SETAs M&E focus of the NSA. • SAIVCET is still at a stage of conception, which might delay the implementation of the envisaged M&E work over the SETAs. • May take focus away from SETA’s role in Higher Education

33. INITIAL REFLECTIONS ON OPTIONS • SHORT TO MEDIUM TERM • An NSA chief-directorate is created at DHET, with a directorate that performs the secretariat function for the Board (current situation), and a new directorate which deals with the evaluation function (Option 6). • The new directorate could have the core capability of staff members who have evaluation expertise and are familiar with the history and functioning of the SETA’s; but should also be provided with a budget to commission expertise from academic institutions. • The chief-directorate could have the dual responsibility to provide secretariat support to the Board and report to the board on the evaluation function and implementation. • The chief directorate could also have the responsibility to report to the Department which will ensure integration of the work of the unit. • Other functions (directorates) that need to be created are those of policy analysis, knowledge management and planning coordination. • In order to improve internal efficiencies, consideration should be made to giving the NSA Chief Directorate more delegations such that the Executive Officer (Chief Director) reports directly to the DG

34. INITIAL REFLECTIONS ON OPTIONS • LONG-TERM • The option of integration of the M&E functions into SAIVCET could be kept open for the long-term, to be revisited once there is more clarity about the possibilities for expanding SAIVCET’s focus and other relevant considerations, including the finalisation of the SETA landscape. • DHET may need to review all its public entities and consider harmonisation and rationalisation of structures working in the same area. • REGARDING THE BOARD • Board composition and operations could be reviewed taking due consideration of the need to eliminate potential possible conflicts of interest arising from board members servicing both the NSA board and SETA boards. • Consider mechanisms for injecting M&E expertise on the board (e.g. including through amendments to the NSA constitution)

35. NEXT STEPS Make an option choice Develop an organisational structure with due consideration given to the recommendations from the capacity building feasibility study. Cost the structure Develop job descriptions for the positions on the structure Finalise the business case