1 / 35

Consultant Performance Evaluation Procedure

American Council of Engineering Companies of Metropolitan Washington DDOT/ACEC QUALITY FORUM . May 26, 2011. Consultant Performance Evaluation Procedure. PURPOSE. Establish and implement a uniform program for systematic evaluation and assessment of consultant performance.

kennan-summers
Download Presentation

Consultant Performance Evaluation Procedure

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. American Council of Engineering Companiesof Metropolitan WashingtonDDOT/ACEC QUALITY FORUM. May 26, 2011 Consultant Performance Evaluation Procedure

  2. PURPOSE • Establish and implement a uniform program for systematic evaluation and assessment of consultant performance. • Enhance the Department’s ability in the consultant selection process. • Provide an objective and consistent method for measuring Consultant performance and benefits both the Department and the Consultant. • Provides the Department with a means for rewarding those Consultants who perform good work, provides essential data to be used in the selection process for future projects • Provides Consultants with the opportunity to improve job performance from one rating period to the next.

  3. PROCEDURE • The Project Leader/Project Manager (PLPM) will rate consultant performance in the applicable general categories (i.e. Schedule, Quality & Project Management) • Evaluations may also include work specific checklists. • The rating criteria for each discipline below are specifically defined under each category. • Design • Engineering Services during Construction • Bridge Management and Inspection • Construction Management and Inspection

  4. PROCEDURE • The evaluations for Design contracts will be project specific. All other evaluations may be project specific or agreement specific. • Design project evaluations will include the work for Preliminary design, Intermediate Design, Final Design and Construction Phase. • Design evaluations may also include project work for Concept Development and Feasibility Assessment. • Design Phase related work such as; Environmental Investigations and/or Permitting (Wetlands, Cultural Resources, Hazardous Waste, Investigations/Permits, etc.), performed by a Consultant through a standalone Task Order Agreement will be rated according to the Design Consultant Evaluation (Design Phase) criteria. • Subconsultant work will not be rated independently but will be included in the prime Consultant’s rating.

  5. RATING SCALE The rating scale for consultant performance evaluations shall be as follows: 5 = Excellent performance 4 = above SATISFACTORY performance 3 = SATISFACTORY performance 2 = marginal performance 1 = unacceptable performance

  6. RATING CATEGORIES AND WEIGHT FACTORS The following rating categories and weight factors will be used for the appropriate work discipline. Design Rating Category Weight Percentage I. Schedule 30% II. Quality Errors & Omissions 60% III. Project Management 10% Post construction evaluation of design will be based on 100% quality. Final Design Rating evaluation will be based on 70% weight during design phase and 30% post construction.

  7. RATING CATEGORIES AND WEIGHT FACTORS Engineering Services during Construction: Rating Category Weight Percentage I. Schedule 30% II. Overall Quality 60% III. Project Management 10%

  8. RATING CATEGORIES AND WEIGHT FACTORS Bridge Management and Inspection Rating Category Weight Percentage I. Schedule 30% II. Overall Quality Errors & Omissions 50% III. Project Management 20%

  9. RATING CATEGORIES AND WEIGHT FACTORS Construction Management & Inspection Rating CategoryWeight Percentage I. Overall Quality 90% II. Project Management 10%

  10. Design Phase Example Consultant: A & E Consultants, Inc., Project: Reconstruction of Street/Bridge Agreement No./Contract No. 000000 ScheduleQualityProject Management Category Rating 3 4 4 Weight Factor 30% 60% 10% Weighted Category Rating 0.9 2.4 0.4 Total Weighted Category Rating = 3.7

  11. Design- Final Rating : Consultant: A & E Consultants, Inc., Project: Reconstruction of Street/Bridge Agreement No./ Contract No. 000000 ScheduleQualityProject Management 30% 60% 10% WEIGHTED AVERAGE Design Phase Ratings Design Preliminary: 3x 0.3=0.9 4x 0.6 =2.4 5x 0.1 =0.5 No. 1 = 3.8 Design Intermediate: 4x 0.3=1.2 3x 0.6 =1.8 5x 0.1 =0.5 No. 2 = 3.5 Design Final: 5x 0.3=1.5 5x 0.6 =3.0 4x 0.1 =0.4 No. 3 = 4.9 Quality WEIGHTED AVERAGE 100% Construction Phase Ratings Construction Phase 4 Rating No.: Average Rating for Design Phase: (3.8 + 3.5 + 4.9) /3 = 4.1 Rating of Design post Construction: 4.0 FINAL DESIGNPROJECT Rating (4.1 x 70% = 2.9) + (4.0 x 30% = 1.2) = 4.1 Example of a final rating for a design project

  12. AVERAGE DISCIPLINE RATING An average discipline rating for each consultant firm will be computed for all his projects within a discipline. Utilizing the example of the Deign Project, the Average Design Discipline rating ofconsultant XYZ above will be as follows: AVERAGEDESIGN DISCIPLINE RATING: Design Project No. 1 - Average Rating = 4.6 Design Project No. 2 - Average Rating = 3.0 Design Project No. 3 – Average Rating = 2.7* Average Design Discipline Rating = 3.4 (Consultant XYZ) * CAUTION: Any single Project Average Rating below SATISFACTORY may affect consideration of the consultant for future consulting services with the Department.

  13. USE OF THE RATING SYSTEM • Consulting firm’s performance with the Department over a three-year period will be assessed. • The ratings for past performance will be taken into account during Department review of the Expressions of Interest (EOI) for a new project. Ratings will be generated for each Consultant based on the firm’s ratings in the system: Final Project Average - a rating for each project. Discipline Average - an average rating of all projects within a discipline

  14. USE OF THE RATING SYSTEM- (Continued) If a Consulting firm has no established ratings, the Department will apply the consultant average rating in other disciplines past performance during Department review of the Expressions of Interest (EOI). The Consultant’s current rating will be used until the firm establishes other Discipline ratings. If a Consulting firm has not acquired ratings with the Department, he/she may receive the average of the ratings from the raters of his/her past work performance in other states or agencies for the applicable discipline until ratings by the firm are established.

  15. CONSULTANT DEBRIEFING • PLPM and the reviewer/supervisor will discuss the rating reports with the Consultant’s Project Manager (CPM) before they are submitted to the Chief Engineer. The consultant or his/her representative is not required to sign the Consultant Performance Evaluation Form (CPEF). • The consultant may request a rating debriefing within 30 days from receipt of the rating. A written request by the Consultant should be made to the Chief Engineer, IPMA. • A debriefing meeting will be scheduled by the Chief Engineer or the Program Manager to discuss the rating. The Chief Engineer, the PLPM and/or his/her Supervisor, the CPM and/or Consultant Principal in Charge will be required to attend the meeting. • Any single Project Average Rating below SATISFACTORY may affect consideration of the consultant for future consulting services with the Department. • For any rating lower than Satisfactory, the Consultant may be required to provide a letter indicating the manner in which the firm intends to resolve problems and issues concerning the rating and how it intends to prevent their recurrence on future projects.

  16. Design Rating (Design Phase) Schedule (30%) (5)Work under the phase is completed in advance of the agreed scheduled date. Projected schedule for deliverables or project completion can be accelerated. (4) Work under the phase is completed in advance of the agreed scheduled date. Projected schedule for deliverables or project completion cannot be accelerated. (3) Work under the phase is completed at the agreed scheduled date. Projected schedule for deliverables or project completion maintained but cannot be accelerated. • Work under the phase is completed after the agreed scheduled date. Projected scheduled for deliverables or project completion maintained. • Work is completed after the agreed scheduled date for critical date for critical path items. Projected schedule for deliverables or project completion delayed.

  17. Design RatingDesign Phase (Cont.) Quality (weighted 60% during design phase) (5) Changes were required for clarity of document presentation only. No technical errors and omissions that influenced the quality of the work. (4) Documented errors and omissions were corrected upon notification. Not considered major error. A resubmission was not required. (3)Documented errors and omissions. Few critical errors identified that required resubmission. (2) Documented errors and omissions. Significant errors identified. Resubmissions were required to correct the work. (1) Documented errors and omissions. Fatal critical errors. Required resubmission of the work and/or reassignment of work by the DDOT.

  18. Design RatingDesign Phase (Cont.) PROJECT MANAGEMENT (10%) • Is organized and proficient with administrative, procedural and technical skills. • Performs the work of the project as required in the Scope of Services and as directed by the PLPM. • Supervises the progress of the work of his staff and that his Sub consultants. • Is proficient with verbal and written communication skills. • Is cooperative with the Department and /or joint operating agencies involved with the project. • Keeps the PLPM advised of general matters and also identifies and works to resolve problems that arise. • Is available for Department phone calls, e-mails and meetings. • Receives Department approval prior to making any changes to Consultant Contract Management or team structure established through the agency. (5) Has met all of the above requirements. No improvements needed. (4) Above average performance, does not meet one of the above requirements. (3) Average performance, does not meet two of the above requirements. (2) Below average performance, does not meet three of the above requirements. • Does not meet three of the above requirements and/or a change of the Consultant’s Contract Management is required by the Department.

  19. DESIGNRATING (POST CONSTRUCTION PHASE) (5) There were no design errors or omissions in the plans and/or specifications that impacted the project schedule and/or cost. (4) There were minor design errors or omissions in the plans and/or specifications, which did not significantly impact the project schedule and/or cost (cost impact less than 0.5% of the project cost; schedule impact less than 2%). (3) There was one design error or omission in the plans and/or specifications, which impacted theproject schedule and/or cost. Cost impact less than 1% of the project cost; schedule impact less than 4%. (2) There were one or more design error or omission in the plans and/or specifications, which impacted the project schedule and/or cost. Cost impact less than 2.0% of the project cost; schedule impact less than 6%. (1) There were two or more errors or omissions in the plans and/or specifications, which impacted the project schedule and/or cost. Cost impact greater than 2.0% of the project cost; schedule impact greater than 6%. Category Weight 100%

  20. PERFORMANCE EVALUATION(ENGINEERING SERVICES DURING CONSTRUCTION) Consultants Responsiveness to the Needs of Construction: The Consultant perform the engineering services for design related support functions such as, shop drawing review, response to construction questions on design issues, required paperwork, preparation of change of plans for unforeseen conditions, and other pertinent requests of the PLPM in a timely and professional manner. Each support function requested of the Consultant shall be considered a “Task Assignment”.

  21. PERFORMANCE RATING(ENGINEERING SERVICES DURING CONSTRUCTION Cont.) Schedule The Department PLPM and the Consultant will mutually establish reasonable schedules for submissions/responses to each Task Assignment. Each Task Assignment will be tracked (assignment date to completion date) by the PLPM. The Consultant’s deliverables should be completed on or before the agreed upon scheduled date(s). The rating will be based on ON-TIME submission of each assignment based on total assignments (i.e., if there are a total of 10 Task Assignments, of which 8 (80%) were completed by the Consultant on or before the agreed upon scheduled date, then the rating would be 4).

  22. PERFORMANCE RATING(ENGINEERING SERVICES DURING CONSTRUCTION Cont.) Schedule 5 = 90 to 100% of the time 4 = 80 to 89% of the time 3 = 70 to 79% of the time 2 = 60 to 69% of the time 1 = 0 to 59% of the time Category Weight 60%

  23. PERFORMANCE RATING(ENGINEERING SERVICES DURING CONSTRUCTION Cont.) QUALITY  (5) There were NO inaccuracies in technical review or presentation NO errors or omissions that influenced the quality of the work. (4) There were MINOR documented inaccuracies in technical reviews, presentation, or errors and omissions that required clarification only. A resubmission of the documents was NOT required. (3) There were documented inaccuracies in technical reviews, presentation, or errors and omissions. One resubmission was required to correct the work. (2) There were documented inaccuracies in technical reviews, presentation, or errors and omissions. Two resubmissions were required to correct the work. (1) There were documented inaccuracies in technical reviews, presentation, or errors and omissions after three resubmissions of the work and/or reassignment of work by the Department was required. Category Weight 30%

  24. PERFORMANCE RATING(ENGINEERING SERVICES DURING CONSTRUCTION Cont.) Project Management • is organized and proficient with administrative, procedural and technical skills. • performs the work of the project as required in the Scope of Services and as directed by the PLPM. • supervises the progress of the work of his staff and that of his Subconsultants. • is proficient with verbal and written communications skills. • is cooperative with the Department and/or joint operating agencies involved with the project. • keeps the PLPM advised of general matters and also identifies and works to resolve problems that arise. • is available for Department phone calls and meetings. • receives Department approval prior to making any changes to the Consultant’s Contract Management or team structure established through the agreement. • Completes work at the time frame noted in the Agreement with no impact on the construction cost or schedule.

  25. PERFORMANCE RATING(ENGINEERING SERVICES DURING CONSTRUCTION Cont.) Project Management (Cont.) 5 Has met all of the above requirements. No improvement needed. 4 Above average performance, does not meet one of the above requirements. 3 Average performance does not meet two of the above requirements. 2 Below average performance, does not meet three of the above requirements. 1 Does not meet three of the above requirements and/or a change of the Consultant’s Contract Management is required by the Department. Category Weight 10%

  26. PERFORMANCE EVALUATIONBRIDGE MANAGEMENT AND INSPECTION CONSULTANT SCHEDULE:   The work performed shall be in accordance with the various schedules as stated in the scope of work and the agreement. The rating will not reflect factors outside the Consultant’s control, such as delays by various authorities, which require right-of way permits etc. RATING (5) Field inspections completed in conformance with the approved schedule, 100% of the reports are submitted in conformance with the submittal schedule. (4) Field inspections completed in conformance with the approved schedule, greater than 90% of the reports are submitted in conformance with the submittal schedule. (3) Field inspections completed in conformance with the approved schedule, greater than 80% of the reports are submitted in conformance with schedule. (2) Field inspections are completed later than approved schedule and/or greater than 70% of the reports are submitted in conformance with schedule. (1) Field inspections are completed later than approved schedule and inspection dates are not in compliance with NBIS. Final Reports are not accepted since bridge inspections were in violation of NBIS compliance. Category Weight 30%

  27. PERFORMANCE EVALUATIONBRIDGE MANAGEMENT AND INSPECTION CONSULTANT OVERALL QUALITY: Consultant Errors And Omissions/Corrective Work. Based on the performance checklists on the PROJECT SUMMARY FORM, reportssubmitted should not require changes due to inaccuracies in Technical areas of the report or Consultant errors or omissions on the Structural Inventory and Appraisal Sheets and Pont is data. Corrective work should not require repeated submissions to the Department. RATING   (5) Documented errors and omissions do not exceed 10% of the reports that were reviewed by the Department. No resubmission of final reports to the Department was required. (4) Documented errors and omissions do not exceed 20% of the reports that were reviewed by the Department. No resubmission of final reports to the Department was required. (3) Documented errors and omissions do not exceed 30% of the reports that were reviewed by the Department and these errors did not affect the assessment of the structural condition of the structure. No resubmission of final reports to the Department was required. (2) Documented errors and omissions did exceed 30% of the reports that were reviewed by the Department and no more than 10% of the report contained errors that have affected the assessment of the structural condition of the structure. No resubmission of final report to the Department was required. (1) Documented errors and omissions affected the assessment of the structural condition more than 10% of the report. Resubmission of final reports to the Department was required. Weight of Category 50%

  28. PERFORMANCE EVALUATIONBRIDGE MANAGEMENT AND INSPECTION CONSULTANT PROJECT MANAGEMENT This category rates the management and not the individual(s) serving in the position. Manages and reviews the work of his staff and sub-consultants to assure compliance with D.DOT requirements. The Consultant Project Management: The Consultant:- • is organized and proficient with administrative, procedural and technical skills. • has properly staffed personnel in a cost effective manner to save money for the Department. • has properly managed the project by matching skills of personnel with the work assignments. • is proficient with verbal and written communications skills. • is cooperative with the Department and/or joint operating agencies involved with the project. • keeps the DCPM advised of general matters and also identifies and works to resolve problems that arise. • is available for Department phone calls and meetings. • receives Department approval prior to making any changes to the Consultant’s Contract Management or team structure established through the agreement. • responds to emergency situations in a timely manner.

  29. PERFORMANCE EVALUATIONBRIDGE MANAGEMENT AND INSPECTION CONSULTANT PROJECT MANAGEMENT (Cont.) (5) Has met all of the above requirements. No improvement needed. (4) Above average performance, does not meet one of the above requirements. (3) Average performance, does not meet two of the above requirements. (2) Below average performance, does not meet three of the above requirements. (1) Does not meet three of the above requirements and/or a change of the Consultant’s Contract Management is required by the Department. Category Weight 20%

  30. PERFORMANCE EVALUATION CONSTRUCTION MANAGEMENT /INSPECTION CONSULTANT OVERALL QUALITY: Based on the CONSTRUCTION MANAGEMENT /INSPECTION - Overall Quality Checklist responses and using the rating system noted below an overall rating is determined. RATING (5) Consultant received 100% “yes” or N/A ratings. (4) Consultant received between 85% and 99.9% “yes” ratings. (3) Consultantreceived between 84.9% and 75% “yes” ratings. (2) Consultant received between 74.9% and 65% “yes” ratings. (1) Consultant received 64.9%and below “yes” ratings. Category Weight 90%

  31. CONSTRUCTION MANAGEMENT / INSPECTION Overall Quality Checklist

  32. PERFORMANCE EVALUATION CONSTRUCTION MANAGEMENT /INSPECTION CONSULTANT Project Management: This category rates the management and not the individual(s) serving in the position. Manages and reviews the work of his staff and sub-consultants to assure compliance with DDOT requirements. The Consultant:- • Is organized and proficient with administrative, procedural and technical skills. • Is proficient with verbal and written communications skills. • Has properly staffed personnel in a cost effective manner to save money for the Department. • Has properly managed the project by matching skills of personnel with the work assignments. • Is cooperative with the Department and/or joint operating agencies involved with the project. • Is available for Department phone calls and meetings. • Keeps the Department Field Manager or Resident Engineer advised of general matters and also identifies and works to resolve problems that arise. • Receives Department approval prior to making any changes to the Consultant’s Contract Management or team structure established through the agreement. • Responds to emergency situations in a timely manner.

  33. PERFORMANCE EVALUATION CONSTRUCTION MANAGEMENT /INSPECTION PROJECT MANAGEMENT RATING (Cont.) 5 Has met all of the above requirements. 4 Above average performance, does not meet one of the above requirements. 3 Average performances, does not meet two of the above requirements. 2 Below average performance, does not meet three of the above requirements. Inspection personnel required improvement and/or training to meet the requirements of the agreement. 1 Does not meet three of the above requirements and/or a change of the Consultant’s Contract Management is required by the Department. CPM had to be called more than two times to resolve issue(s) concerning performance of inspection personnel. Category Weight 10%

  34. Contact information Muhammed Khalid, P.E. Program Manager District Department of Transportation 55 M Street, S.E. 4th Floor Washington, DC 20003 202-671-4577

  35. Questions !

More Related