EVALUATING THE PERFORMANCE OF BRAZILIAN FINANCIAL MANAGERS WITH THE AHP

1. INFORMS ANNUAL MEETING 2012 Phoenix, AZ, October 14-17 EVALUATING THE PERFORMANCE OF BRAZILIAN FINANCIAL MANAGERS WITH THE AHP Renata Andrade, HR Division, Brookfield Brasil, Rio de Janeiro, Brazil, randrade@brookfieldbr.com Luiz F. Autran M. Gomes, Professor, Ibmec/Rio de Janeiro, Brazil, autran@ibmecrj.br

2. Characteristics of thestudy: • Context: The on-goingprocess of functional performance evaluation of asset managers in a major financial organisation in Brazil • Justifications: (1) Performance evaluation is understood as a strategic tool for favouring the creation and maintaining a competitive advantage in this particular organisation. (2) Performance evaluation in this organisation serves two key, tactical objectives: rewarding employees for targets met and identifying the proposals not achieved so that an action plan can be defined. (3) After some previous experiences with other performance evaluation approaches (e.g., BSC) theorganisationaccepted quite well the idea of using for the first time a well-structured, multicriteria approach due to the need to increase the effectiveness and the transparency of thatprocess • Scope:From translating the organisational mission into objectives asset managers were evaluated based on their individual contribution to the realisation of the organisational mission. The AHP was then chosen as the analytical method to be used • Nature of the problem: This was a demonstration study with twelve asset managers of the organisation that led to solving a ranking (i.e., P) problem through computing vectors of priorities

3. Theorganisationalenvironment: • The Brazilian branch of a transnational company, acting in the assets management market. With a 100 year history of investing in the country, the economic group in Brazil has currently a few thousand employees and one of the largest investment platforms in the country, including its own and institutional clients’ assets, investing in the property, renewable energy, agriculture, forestry and infrastructure sectors. • A couple of years ago, that Brazilian branch radically altered its strategy, ceasing to act solely as controller of the companies in the country and starting to work in assets management through the use of investment funds. From this alteration and the much closer intervention of a shareholder company, it became necessary to understand how the performance evaluation processes concerning the mapping of the competences are aligned to the mission, vision and values, making the shareholder company one of the largest asset management companies in the world.

4. Why the AHP? • The problem of performance evaluation is suitable to beformulatedbybuilding a hierarchyof objectives / criteria • Softwares are availablethatcanfacilitate a constructivist approach to the problem (i.e., objectives / criteria, alternatives as well as preferencescan evolve during the process of performance evaluation) • The basicstructure of the methodhasbeenwellknown for manyyears and there are techniquesthatcanbeused in order to facilitate and improve its practical use • A vastliteratureon the use of the AHP/ANP for performance evaluation is available • In spite of theexistingcriticisms of themethod, usingthe AHP canalsoaddthefollowingbenefits: (i) it helps in building a commonrepresentation of theproblembymakingparticipantsgetinvolved in theevaluationprocess; and (ii) it facilitatestheacceptance of MCDM as a key approach to major organisationaldecisions

5. A few examples of the many available references on the use of the AHP/ANP for performance evaluation: • AHMADI, S.A.A. (2012) Performance Evaluation of Tehran Province PayameNoor University Staffs (Open University) by AHP Technique. Interdisciplinary Journal of Contemporary Research in Business, 4, 1, 225-234. • BEŞKESE, A.; BOZBURA, F.T.; ÇAVUŞOĞLU, N.G. (2011) A Fuzzy AHP Method to Prioritize Individual Attributes in a Performance Appraisal System. 15th International Research/ExpertConference “Trends in the Development of Machinery and Associated Technology” (TMT 2011), Prague, CzechRepublic, 12-18 September. • CHENG, E. W. L.; LI, H. (2006) Job Performance Evaluation for Construction Companies: An Analytic Network Process Approach. Journal of Construction Engineering and Management, 132, 8, 827-835. • ISLAM, R.; RASAD, S. M. (2006) Employee Performance Evaluation by the AHP: A Case Study. Asia Pacific Management Review, 11, 3, 163-176. • LIU, C.-H.; HSU, H.-S.; WANG, S.-T.; LEE, H.-K. (2005) A Performance evaluation Model Based on AHP and DEA. Journal of the Chinese Institute of Industrial Engineers, 2151-7606, 22, 3, 243-251.

6. Structuring of thestudy: • The performance evaluation process is intrinsically linked to the strategy, in other words, to the mission and the vision for the future, which, in turn, are also influenced by the environment in which the company is inserted, and by the demands of the major shareholders. • From the construction of the current ideal scenario the managerial competences are identified, together with the objectives, making it possible to build the hierarchy of the model and the weighting of the degrees of dominance of the attributes and the alternatives • The study sample was composed of 12 asset managers from the same branch, including the controlling & accounts area • Objectives were mapped from the organisational mission and competences were defined from the described mission; indicators were created for the evaluation of behavioural skills • Since this was a demonstration study it was decided that the SuperDecisions software could be used

7. Structuring of the study

8. Thehierarchy of objectives / criteria: • The mission: Provide shareholders with information and controls, create and maintain fiscal and societary structures, identify the best investment practices in the Brazilian market • This mission is unfolded in two major criteria: Competences and Adherence to Strategical Objectives • The Competences criterion is unfolded in Behavioral and Technical & Scientifical Skills – Indicators were then developed in order to evaluate each employee under these two sub-criteria • The Strategical Objectives that were considered were the following: 1. Provide shareholders and directors with quarterly, annual and periodical information; 2. Provide due accounting and fiscal reports; 3. Participate in preparing the company’s annual report and provide auditors with the necessary information; 4. Constantly enhance fiscal, accounting and operational processes, focusing on their optimization and improved efficacy; 5. Serve internal clients in a timely and efficient manner. • Judgements were to be provided by the managers of the controlling & accounts area as well as of the HR division.

10. Hierarchical structure of the problem (SuperDecisions)

11. Comparingbehavioral versus technical & scientificskills: • Behavioural skills were considered moderately more important than the technical skills: the competences responsible for differentiated performance are those which can be observed, tested and modified, in other words, the technical skills, while the behavioural competences are those of difficult access. • The small dominance of the behavioural aspects over the technical ones is due to the fact that, as they are difficult to monitor and modify, the organisation has difficulty in managing them, in contrast to the technical skills, which can be improved through training and development; the organisation expected to lead the employees to reflect on those elements of which only they have profound knowledge about.

12. Judgement in relation to the competences

13. Inconsistency in the “competences” judgement

14. Producingrelative and global weights, pairwisecomparison of alternatives: • Once all judgements have been made, the relative weights and the inconsistency indices of each evaluation matrix were made explicit, as well as the global weights which represent the weighting of each element in a general analysis: the global weight is equal to the relative weight multiplied by the weight of the corresponding indicator. • Having identified the relative and global weights, all the employees (i.e., the alternatives) are compared in pairs, according to the objectives (1, 2, 3, 4 and 5), technical and scientific skills (technical knowledge of the legislation in vigour, mastery of the work tools, experience in the role and in similar businesses, specific scientific background in the area of actuation and level of proficiency in the English language), and behavioural indicators related to efficient communication (C1, C2, C3 and C4), focus on the business (F1, F2, F3 and F4), initiative and pro-activeness (I1, I2 and I3) and team work (T1, T2, T3 and T4)

15. Relative and global weights, second and third levels

16. Relative and global weights, third and fourth levels

17. Results of the evaluations of the alternatives

18. Final results, after a sensitivityanalysis: • The decreasing order of the employees who present the best performance, in other words, who most contribute to the realisation of the mission was: Employee 6, Employee 5, Employee 1, Employee 4, Employee 3, Employee 10, Employee 2, Employee 11, Employee 9, Employee 7, Employee 12 and Employee 8, as presented in the coming slide. • Thus, practices of variable remuneration should compensate the employees with better performance, while the training and development unit (HR) must focus on the employees and competences to be improved. • In this way, the performance evaluation, according to the AHP model, provided for a more transparent and just process, as well as greater effectiveness in investment in training and development. • In addition, after mapping the competences, as well as the levels characterised as high performance, the recruitment and selection unit (HR) can certainly be more assertive in its processes.

19. Synthesis graph of the result

20. Comparingthe use of the AHP againstsimplescoring: • When comparing the use of the AHP against the simple, direct scoring utilized by the same overall process manager in the last period it could be observed that the rankings were different by the two approaches  under direct scoring we had the following ranking: Employee 10, Employee 6, Employee 1, Employee 9, Employee 11, Employee 3 and Employee 5; it is important to highlight that Employees 2, 4, 7, 8 and 12 do not appear in that direct scoring because they did not form part of the staff in the last appraisal carried out. • However, two key aspects must be considered here: (1) the context of the evaluation changed from the last period to the present one, due not only to the imposition of the new scenario by the shareholding company, but also due to changes in the assets management team; and (2) using the AHP provided for a broader, better-structured evaluation framework, where quantitative as well as qualitative objectives / criteria were taken into consideration

21. Dispersion graph of the performances with the use of the AHP

22. Dispersion graph of the performances without the use of the AHP

23. Conclusions: • The AHP was suited to the problem presented, providing comprehensible and wide-ranging results for this practical decision case as well as permitting transparency and systematization of the performance evaluation process. • This study provided, through the application of the AHP, the use of a performance evaluation process aligned with the organisational strategy, as well as the clarification of individual targets and the definition of the competences necessary to accomplish the vision of the organisation’s future. • The use of the AHP model in the employee performance evaluation process favoured effectiveness, to the extent in which it scored objectives and competences based on the realisation of the organisational strategy; consistency checks of the judgements made were provided by making use of SuperDecisions. • Another very real benefit from this study was the reduction of a disequilibrium caused by internal conflicts resulting from the clash of thoughts, actions and values between employees and the company • The organisation has decided that other MCDM analytical methods may be utilized to tackle quite important evaluation and decision problems.

24. For the complete paper please visit: http://dx.doi.org/10.1590/S0101-74382012005000003 or: http://www.scielo.br/pdf/pope/v32n1/aop0312.pdf Acknowledgement:Work on this paper was partially supported by CNPq (National Council for Scientific and Technological Development of Brazil) through Research Project No. 310603/2009-9 Thankyou!