1 / 23

Evaluation Methodology and Scoring Criteria for Firm Selection

Review and recommend an improved methodology and scoring criteria for selecting firms based on qualifications, with a focus on firm size, experience, staff experience, location, and previous work. Aim to encourage teamwork and maximize opportunities for small firms.

borelli
Download Presentation

Evaluation Methodology and Scoring Criteria for Firm Selection

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Task Team 13.5

  2. Background Task Identified for Team 13.5 • Review the current methodology for the workload, compatibility and location and criteria and make recommendations for either a subjective or analytical (revised) method and review the General Scoring Criteria

  3. Team 13 Approved Business Objective Developed by Team 13.1 Select a firm based on qualifications that best meets the needs of the project scope.

  4. Measurement The objective is being met when at least 85% of the firms doing business with DOTD have past performance ratings of “meets requirements” or above. The target date for meeting this goal is 3 years after implementing the “To be” process. From that time forward, this measurement will be taken annually to ensure the selection process is achieving the stated objective.

  5. Desirable Outcomes • Encourage Teams and Teamwork among consultants. • Maximize the opportunity for firms to perform DOTD work for the purpose of maintaining a healthy resourceful consultant community. • This can be achieved by easing the entry of small firms through the Pre-qualification Selection Program

  6. Law - Weighting Factors • The current weights for the rating factors are: • Firm Size (3) • Firm experience (3) • Staff experience (4) • Location (4) • Previous work (6) • Workload (5).

  7. Firm Size (3) Definition/Objective • Existing Definition: • The intent of this category is to optimize the firm’s size (total numbers of full-time employees) to the size of the project (total estimated cost of the project). A small firm responding on a large project will not score well in this category. Likewise, a large firm responding on a small project will not score well. The “penalty” for being too small is greater proportionately than for being too large. Prime consultant’s compatibility rating will be used for the sub-consultant’s rating when the percentage of the work to be performed by the sub-consultant is small. The same compatibility rating may also be used for all respondents for small retainer contracts. The optimum project size (O) for the firm (in $) is calculated as: O = (Z) ($3000/person/mo.) (contract time) where Z is the firm size entered on the consultant record. The cost of the work to be performed by the firm (FEE) is computed as follows: FEE = sum of (c(i) * p(i)) where c(i) is the project cost of category i, the percentage of category i to be done by the firm The ratio of the optimum project size to the fee:     RATIO = O / FEE

  8. Firm Size Recommendation • Size of firm will be defined as the number of transportation-related employees • Firms will not rated below 2 for being too big for moderate to complex projects, but can be rated a 0 if too small. • $3000/person/month will be defined as the salary of an engineer and 4 staff for statewide average.

  9. Pros Will match firm size with project size or complexity Does not penalize larger firms for moderate to complex projects Cons Determine Firm Size Pros/Cons

  10. Firm Experience (3) Definition/Objective Existing • Each committee member independently evaluates each respondent for firm experience relevant to the project advertised. This category is also concerned with the experience of the top-level managers, as a unit, for the responding firm. This rating is generally obtained from Section 12 of DOTD Standard Form 24-102. For reference, the ratings are based on the following: 0 – very weak; 1 – weak; 2 – acceptable; 3 – strong; and 4 – very strong. The weighting factor for this category is three.

  11. Staff Experience (4)Definition/Objective Existing • Each committee member independently evaluates each respondent for key personnel experience relevant to the project advertised at all levels within the firm performing the work on the project. Experience gained with any employer is “counted”. This rating is generally obtained from Sections 10 and 11 of DOTD Standard Form 24-102. For reference, the ratings are based on the following: 0 – very weak; 1 – weak; 2 – acceptable; 3 – strong; and 4 – very strong. The weighting factor for this category is four.

  12. Experience of the Firm and Key Staff – Brainstormed Ideas Proposed • Combine experience of the firm and key staff as one criterion with a weighting of 7 Instruction for CEC • Review the relevant projects the firm has completed and ensure the resumes of the employees that have worked on these projects are still employed by the firm. • Firm experience - On the form the firm should only submit similar projects performed by the employees that are still employed by the firm. • Key staff - Should submit only information on relevant projects with specifics of their involvement, dates and if they worked for the submitting firm (yes or no) performing that work.

  13. Recommendation • Combine experience of the firm and key staff as one criterion with a weighting of 7 Instruction for CEC • Review the relevant projects the firm has completed and ensure the resumes of the employees that have worked on these projects are still employed by the firm. • Firm experience - On the form the firm should only submit similar projects performed by the employees that are still employed by the firm. • Key staff - Should submit only information on relevant projects with specifics of their involvement, dates and if they worked for the submitting firm (yes or no) performing that work.

  14. Pros Hard to separate The staff is the firm Evaluating past projects with different staff is not relevant for a firm Cons May have to change law Lessens higher ratings for firms that have been in business vs. new firms with powerful staff Determine Combining Pros/Cons

  15. Location (4) Definition/Objective Existing • The location factor is based on the distance between the firm’s office(s) to the project site. The firm’s office will be the location where the key staff and the majority of the personnel performing the work for the project are located. Basically, the closer the firm’s office is to the project site, the higher its score. A coordinate system or a GIS-based procedure is used to compute the distance to the project site. For district- and state-wide projects, coordinates for the District Office or Marksville (for state-wide) will be used for the project location unless otherwise noted in the advertisement. The weighting factor for this category is usually four. For Urban System projects, the factor is increased to six. The table below describes the rating. • Distance Rating 0 To 50 Miles = 4 50 To 450 Miles = Interpolated between 4 and 0More than 450 Miles = 0

  16. Location Recommendation • For statewide and specialty projects this factor will be neutralized and everyone will receive the same score if recommended by the PM or use Marksville, LA or the district office as the project location. • For other specialty projects the weighting factor can be increased in the advertisement to ensure a local firm has more consideration if recommended by PM.

  17. Pros More input from PM Cons Statewide contracts will have out-of-state firms with same advantage as in-state firms Determine Location - Pros/Cons

  18. Past Performance (6) Definition/Objective Existing • This rating is based on the results of input from the Project Managers at various project milestones. They evaluate the firms that they have managed within the last five years. The weighting factor for this category is six. The firms are evaluated in five categories: knowledge and technical expertise, quality of plans and other deliverables, completion of work within the terms of the contract, cooperation and problem resolution, and communication and documentation. Project Managers rate on the same 0-4 point scale that is used by the CEC. For reference, the ratings are based on the following: 0 – unsatisfactory; 1 – needs improvement; 2 – meets requirements; 3 – exceeds requirements; and 4 – outstanding. The ratings are given for several categories of work. Project specific ratings may also be obtained from the pertinent department personnel. Each firm receives the average rating from all Project Managers who have evaluated them. Project Managers’ ratings are subject to a review by their direct supervisors. The Committee screens the Project Managers’ ratings for: those who may grade either too strictly or too leniently; consistency; grading which conflicts markedly with other coordinators, or with the Committee’s perceptions; etc. The Committee reserves the right to investigate any discrepancy they may identify, and change or eliminate an old rating, if justified. To date, changes have been rare.Only those firms who have had projects monitored by DOTD within the last five years are rated. Firms who have not received a rating for a work category are given the average of all respondents who have a rating, subject to a maximum limit.

  19. Past Performance Recommendation • Leave as 5-years pending the recommendation from 13.2 - 3-year existing to 5-year proposed phased in over 2-years • Use the average of the other firms submitting on the project for firms having no Past Performance rating in system

  20. Pros Allows new firms to have an opportunity Rewards firms that have performed well for 5-years The phase-in allows the consultant to be on notice to closely monitor their ratings without being penalized for the past Cons A bad rating within the past 3-years could penalize the firms that have worked to rectify the problem Determine Past Performance Pros/Cons

  21. Workload (5) Definition/Objective Existing • This category considers the firm size (as explained in COMPATIBILITY), and the workload with the Department in dollars. Projects that are “counted” are: all contracts with others on behalf of the Department using Department administered funds; all contracts with others with supervision provided by the Department; and all contracts with the Department. Note that a project is counted as soon as the selection is made. For retainer type contracts, only the remaining balances on open Task Orders are counted. The maximum workload for a firm with DOTD equals the firm size multiplied by $75,000. The greater the firm’s workload is, the lower the rating in this category. A firm with no workload with DOTD would receive a rating of four. The weighting factor for this category is five.

  22. Workload Recommendation • Segregate by complexity of project • Simple and moderate projects leave formula as is. • Complex projects neutralize by assigning same value to all • Review $75000 and recommend to increase – Ed will review history and make recommendation (It was $75,000 in 1990) • If a project is projected to be on “hold” or suspended for 6 months or greater, the remaining fee is not “counted” against the consultant in section 13 of the 24-102 (with approval of the DOTD PM)

  23. Pros Increase $75,000 - Firms could carry a higher $ value on books Segregate Work – Complex projects would encourage a QBS selection Cons Penalizes firms for having work Allows firms that have work to get more Determine Workload Pro/Con

More Related