1 / 53

Assessment Update 2009

Assessment Update 2009. Workshop objectives. To bring you up to date with developments relevant to the assessment process The new 2010 Training Model and it’s implications for assessment Changes to the Training Regulations

aubrey-cash
Download Presentation

Assessment Update 2009

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessment Update2009

  2. Workshop objectives • To bring you up to date with developments relevant to the assessment process • The new 2010 Training Model and it’s implications for assessment • Changes to the Training Regulations • Benchmarking the SA assessment process against international best practice • Results and implications of the survey conducted on the application of the ratings • Enhancements to the assessment instruments

  3. Responsibility of role players? • Trainees • Initiates the documents (PR / IRNA / DNA) • Takes responsibility for presenting adequate evidence of their competence • Reviewers • Directly involved with the trainee during the performance of a task • Completes the PR / IRNA • Does not have to be an Assessor • Evaluators (TIPP only) • Knows and regularly works with the trainee • Completes the DNA • Assessors • Reviews the DNA / IRNA • Completes and signs off the final TR

  4. Let’s get started then… • The new 2010 training model • All role players (“old” trainees, reviewers, evaluators) will become involved in some way with the assessment of new 2010 trainees and so you all need to understand the new 2010 training programme • The new assessment instruments are designed around this new training model so it makes sense to get more acquainted with this new model first

  5. The content of the CA 2010 Training Programme is based on the competency framework for the CA(SA) qualification CONTENT

  6. The structure and content combine to form a robust and flexible training model BASIC EXPERIENCE in a SIMPLE CONTEXT • ADVANCED EXPERIENCE in a COMPLEX CONTEXT

  7. The training programme is structured in such a way that • ALL trainees will gain exposure and achieve competence in ALL the skills areas, but to varying degrees of complexity; and • The depth and breadth of a trainee’s exposure will be determined by the nature, industry and business model of the training office. Compulsory skills Elective skills Residual skills

  8. The Programme provides each training office with the FLEXIBILITY to structure its training programme to meet its business, training and succession planning needs. In fact, an office may choose to offer more than one training programme. Residual skills Compulsory skills Elective skill Example 1 – For qualification as a CA(SA) and an RA

  9. Compulsory skills Elective skill Residual skills Example 2 – For qualification as a CA(SA)

  10. Compulsory skills Elective skills Residual skills Example 3 – For qualification as a CA(SA)

  11. The CA 2010 Training Programme results in one change in the requirements for the successful completion of the training contract

  12. The implementation of the CA 2010 Training Programme has been kept as simple and seamless as possible, requiring limited transitional arrangements

  13. What has SAICA achieved? SAICA have developed a training programme that • Gives effect to the competency framework • will give all trainees exposure to a wide range of competencies while still requiring areas of focus, thereby ensuring greater balance and exposure to a broad range of relevant competencies • allows training offices to design a training programme to meet their specific requirements, and in fact allows them to have multiple training programmes • Re-inforces the single brand of CA(SA) by removing the distinction between TIPP & TOPP

  14. Students’ questions answered

  15. Summary of key changes to the training regulations CHAPTER 5, RULE 21 – SECONDMENT There is now no longer any restrictions on the amount of time for which training officers may second trainees to another training office in South Africa or to a South African work environment which is not a training office. The time limitation on overseas secondments has also been extended. CHAPTER 5, RULE 23 – CANCELLATION OF A TRAINING CONTRACT The provision for the transfer of a training contract has been removed. Trainees and training officers now only have the option to cancel a training contract with a penalty, or, in certain limited circumstances, without a penalty.

  16. Summary of key changes to the training regulations CHAPTER 5, RULE 25 – DISCHARGE OF A TRAINING CONTRACT There are new discharge requirements applicable to trainees who entered into a training contract on or after 1 January 2010. These requirements are: “The training officer must initiate the discharge of a training contract in the prescribed manner if he is satisfied that the trainee accountant has – • completed the prescribed minimum hours of work attendance; • completed the prescribed minimum hours of core experience; • completed the term of the contract, including approved remissions and extensions; • successfully completed the ethics programme prescribed or recognised by SAICA; and • achieved the following competencies prescribed by SAICA: • The compulsory skills areas; • At least one elective skills area; and • The residual skills areas. A trainee accountant who wishes to register as an auditor with the IRBA must achieve the competencies prescribed for the Auditing and Assurance elective of the training programme.”  CHAPTER 6, RULE 28 – CORE EXPERIENCE, ASSESSMENT AND OTHER RELATED MATTERS  A provision has been introduced that, in the event of failure by the trainee to initiate two successive assessment forms within the prescribed time frames, the training officer will have to extend the training contract by six months.

  17. The impact of the new Training Model on assessment… • Major change is in the content and the way the training experience is achieved • There will still be “outcomes” and “assessment criteria” that will still need to be assessed – the need for assessment doesn’t change

  18. Impact on assessment • There will be one generic set of documents for all trainees • Documents are going to have to be compiled based on the trainee’s training “route” (Electives vsResiduals) • Different trainees within 1 firm may have several different “routes”

  19. Why have assessment at all? Assessment is… • A SAQA requirement in terms of the NQF • An IRBA requirement for SAICA to remain accredited to train prospective RA’s • An IFAC member requirement

  20. The SAICA assessment ratings • SAICA uses a rating scale of 1 to 4 to record demonstrated competence. • Let’s take a closer look…

  21. The SAICA assessment ratings • Consider the scenario on the next slide and rate it based on your current understanding of the SAICA rating scale…

  22. A trainee scenario… • A trainee completes a section of work on their own (using the prior year file / previous workings as a starting point) and hands it to you for your review. You are not happy with the work and raise a number of queries on the work (a few of them are fundamental / significant) which you then hand back (with the work) to the trainee who then finishes the work to your satisfaction.

  23. The SAICA assessment ratings • Let’s have a show of hands… • Who rated this scenario as a: • Level 1 competence? • Level 2 competence? • Level 3 competence? • Level 4 competence?

  24. The SAICA assessment ratings • Let’s consider a few more scenarios… • This exercise was based on scenarios included in the ratings survey exercise conducted in November 2008… • Take 10 minutes to complete the ratings exercise in your notes on page 4

  25. The SAICA assessment ratings • Let’s again have a show of hands for each scenario… • Who rated the scenario as a: • Level 1 competence? • Level 2 competence? • Level 3 competence? • Level 4 competence?

  26. SAICA Benchmarking survey on the assessment ratings • Conducted in November 2008 • All role players in TIPP were invited to participate • 606 responses… • small firms: 306 • medium sized firms: 81 • large firms: 181 • unspecified firm sizes: 38 • All levels within the assessment process • trainees in period A: 129 participants • trainees in period B: 145 participants • trainees in period C: 109 participants • Managers / seniors: 113 participants • Partners / Directors: 110 participants

  27. Ratings survey - findings • Understanding & Ability get combined… • Differences in interpretation of a level 2, 3 and 4… • Professional skill capabilities influence the assessment of technical skills… • Period of contract influences ratings… • Complexity of task influences ratings… • Prior knowledge of the trainee influences ratings… • TIPP DNA ratings are often just mathematical averages of PR scores…

  28. 2 Clarification guidelines… • More detailed explanations of the different rating levels • Decision trees to assist in the ratingdecision The objective of these? • To make assessment more consistent • To make assessment fairer to the trainee

  29. The Rating levels? • Expanded guidance on what differentiates the rating levels… • Let’s have a closer look…

  30. The Decision tree (Technical skills) • Do trainees require direction to enable them to complete the task? • How many outstanding matters are there after the first review? • How significant are the matters? • Can the matters then be resolved by the trainee?

  31. What about the rating of professional skills? • We also need to rate a trainee’s “Ability” to demonstrate these skills. • The rating scale is now the same as for the technical skills – 1 common scale for all assessments! • Also an expanded definition of the 4 levels (refer the assessment toolkit) • Also a decision tree to assist with consistency…

  32. The Decision tree (Professional skills) • Do trainees require direction to enable them to demonstrate the skill? • To what degree was the desired objective / outcome of demonstrating the skill achieved? • After receiving further guidance, can any outstanding matters then be resolved by the trainee?

  33. Is the rating scale any more clear now? • Using the decision tree, and your new understanding of the ratings scale, re-visit the scenarios and see whether you would now change any of your ratings? Record your new rating in the 2nd column.

  34. What about the context? • Does it matter how complex the assignment is? • Yes, but only when rating the trainee in the DNA / IRNA (which is a measure against the final required entry level into the profession) • The required context for this level is partially defined through the Training Guidelines (accreditation requirements). • This is however left largely to the professional judgement of the assessor (what would you expect from a newly qualified CA(SA)?). versus

  35. Context in the 2010 model “Advanced” • Complex / Involved • Preparing a budget for a company / large audit • Calculating income tax for a manufacturing company “Basic” • Simple / Straight forward • Preparing a budget for a division / small audit • Calculating income tax for a property company Will always be a matter of professional judgement to be exercised by the assessor in the DNA / IRNA…

  36. Enhancements to the SA assessment process… • Background to the changes • Review of relevance and efficiency of SA assessment process – June 2008 • SAICA benchmarking survey on the application of the rating scale – November 2008 • The new training model for 2010 • Resulted in • Several enhancements to the assessment instruments to bring them in line with best practice internationally, to make them more efficient, and to make them relevant to the new training model… • A big drive to clarify the ratings scale and try and make assessment as consistent as possible • Effective from 1 January 2010

  37. Key features of the new process… • One set of documents for all trainees (no more differentiation between TIPP and TOPP trainees). • Evidence of technical competence & professional capability will be gathered separately. • Evidence of technical competence will need to documented at least once every 2 months. • Evidence to be obtained of technical competence will only focus on the “ability to do” and will no longer need to be obtained for the “understanding” component. • Evidence of professional capability will be documented on an on-going basis. • Evidence obtained (technical & professional) will be formally evaluated and assessed on a 6 monthly basis.

  38. 2010 SAICA assessment process First 6 months Second 6 months Review of assessment status performed every 6 months to evaluate progress & assess competence to date Minimum of 1 Technical skills review every 2 months “Open” Professional skills review to be filled in on an on-going basis & progress to be assessed every 6 months

  39. Transitions for TIPP… • PR’s will be split into a technical skills review (TSR) and a professional skills review (PSR). • Technical skills reviews will now only address the “ability to do” component. • There is a new format for the professional skills review. • DNA (now called the “Assessment Needs Analysis”) will stay much as it currently is although there will now be a requirement to summatively assess trainees every 6 months

  40. Transition for TOPP… • IRNA sections A and B will be split • Section A will be split further intotechnical and professional skills review documents. • Technical skills reviews (TSR) will now only address the “ability to do” component. • There will be a new format for the professional skills review (PSR). • Technical reviews (old “Section A”) will need to be completed at least once every 2 months. • The “IRNA Section B” will now only need to be completed every 6 months .

  41. The technical skills review… • Refer summary in notes • Refer to the actual document and note the following: • Completed at least once every 2 months. • Cover page now includes dates of completion by trainee and reviewer. • Outcome = “Competency” • Assessment criterion = “Task” • Trainee to reflect on the professional skills utilised in demonstrating their technical competence.

  42. The professional skills review… • Refer to the summary in notes • Refer to the document and note the following: • Open and “updated” on an on-going basis • Review within 1 month of documenting the evidence • Examples will now be up-front as a reference tool for trainees to utilise (still to be populated…) • New rating scale (now consistent with the technical skills review). • 1 outcome per page – to provide space. • Evidence in support of ability to demonstrate a skill needs to be presented / documented.

  43. The 6 monthly assessment… • Refer to the summary on page 29 • Refer to the document and note the following: • Objective is to evaluate the evidence presented by the trainee and to assess their demonstrated competence at this point in timeagainst final entry levels into the profession • The ANA should involve a formal meeting between the trainee and their evaluator / assessor • No longer any “suggested levels for period A, B & C” – only the guide as to the required final level in section 1. • Evaluator to evaluate evidence presented and, based on this evidence, rate the trainee against the required final levels • Overall rating from previous ANA brought forward cumulatively • There is no minimum or maximum number of TSR / PSR ratings required to be able to reach a level 4 competence

  44. The 6 monthly assessment… • Refer to the document and note the following: • Assessor to consider the evidence presented and assess whether the trainee has reached the required final levels (initial against each outcome). • Additional space for comments by evaluator and assessor (if there is a need to justify their decisions). • Multiple professional skills reviews for an outcome to be summarised (4 blocks available)… • Note that all professional skills will now require a final competence level of 4… • Section 2 reduced and section 3 removed…

  45. Practical considerations… • Even more important to understand the difference when applying the rating levels during a review as opposed to during the 6 monthly assessment… • Reviewers rate competence at a point in time based on evidence presented of a specific task / skill. • 6-monthly assessments assess demonstrated competencies (evidence presented) in the context of the final sign off levels (entry level into the profession)

  46. Practical considerations… • When is “development” required? • When an overall rating is at a level lower than it should be, given: • the exposure that the trainee has already had to that competence / those tasks • The nature of “development required” is that it is remedial • Development cannot be required in instances where the trainee has not yet had an opportunity to demonstrate their competence – this is a planning issue!

  47. The final certificate… • Refer summary in notes • Let’s have a look at the document… • Simply becomes the final sign off of the training contract to indicate that all the requirements have been met. • Summative assessment will already have been performed on a 6 monthly basis. • Will still incorporate the “Certificate of Completion”.

  48. An exercise • Read through the scenario presented to you for the trainee, Hugh Slacker… • Review the document he has presented to you on your own first and… • Complete your reviewer rating of the evidence he has presented (if you can!) • Note any errors in the completion of the document • Raise any areas of concern (that you would like to address with Hugh) • On completion of your review, discuss your findings with each other IFRSRools Hugh SlackerProspective CA…

  49. Discussion points” • Grammar is a problem and would need to improve to become more professional (should be discussed with Hugh i.t.o. his poor “presentation of views”) • There is no specific review of his tax work (lack of interest & poor quality of work) in the PR – note it is the responsibility of the trainee to present the evidence (and not of the reviewer to ensure everything is presented!)

  50. Discussion points • The PR should have been reviewed after each addition by Hugh – and not at the end of the 6 months. • Hugh’s disappearance during the Gungho audit and the presentation issues with his work have not been reflected in the PR? That’s fine – Hugh is choosing to not present the evidence… • The last assignment showed a lot more initiative and commitment on Hugh’s part but these have not been specifically mentioned in the PR? Again, this is Hugh’s responsibility and not yours.

More Related