Past Performance Information

1. Past Performance Information John Savicky Arizona State University May 2013

2. PHASE 1 PHASE 2 PHASE 3 Identification of Potential Best-Value Pre Planning and Quality Control Award | Measurement & Documentation PIPS Best-Value Process

3. PIPS Has Multiple Filters • No single step will complete minimize all risk. Process has several steps that act as filters. • Filters do not limit competition, but make it difficult for non-performers to be competitive. • Looking for dominant information

4. What is the Objective of PPI? • PPI should not be used as the sole factor to ensure that the vendor can perform on a current project. • Communicate to the vendors that the owner will purchase based on value (performance and price). • Identify if a vendor has capability to perform. • Encourage vendor to use and partner with the best people / best subs 4

5. How Should We Collect PPI? • Survey all of the vendors past projects? • Survey all projects done for just your organization? • Select a random number of past projects (good and bad)? • Survey the last ten projects? • Survey all the projects completed in the last 2 years? • Survey only projects that are similar to the type of project being procured?

6. In PIPS, Vendors Are Allowed To Only Select Their Best Past Projects

7. Is This Any Different Than The Traditional Approach?

8. What Questions Should We Ask? • Ability to manage cost • Ability to manage schedule • Ability to communicate • Ability to manage subcontractors • Ability to follow facility rules • Ability to minimize change orders • Ability to minimize claims • Ability to minimize punch list items • Knowledge of codes • Management abilities • Quality of submittals • Quality of workmanship • Professionalism • Response time to emergencies • Response time to warranty work • Knowledge of regulations • Willingness to do value engineering • Contractor’s level of honesty • Overall construction knowledge • Overall construction quality • Overall performance of contractor • Customer public relations • Your overall customer satisfaction • Was job completed on time • Skill of craftsmen 8

9. What Questions Should We Ask? 9

10. Statistical Analysis • Objective: • Determine if survey questionnaire could be minimized • Methodology: • Analyzed 175 contractor PPI databases (GC, Elec, Fire Alarm) • Analyzed 37 criteria • Perform statistical analysis • Results: • Results indicated 8 criteria would generate same results as asking 37 • Re-tested 8 criteria on 50 randomly selected contractors. • Further tested on 29 randomly selected HVAC, Plumbing, and Painting vendors. 10

11. Survey Form 11

12. Collecting Surveys • Option 1 – Client collects surveys on all vendors • Option 2 – Client hires a third party to collect surveys • Option 3 – Vendors collect their own surveys

13. PPI Analysis • If Client collects surveys, be prepared for additional work • Must call to confirm • Must have several machines to accept surveys / paper • Must have people to contact regarding surveys • Must enter data and double check for errors • City of Peoria = over 8,000 surveys • University of Minnesota = over 4,000 surveys

14. In PIPS, Vendors Are Required to Survey, Collect, and Submit Their Own Surveys

15. DETAILED STEPS

16. Step 1 ENTITY Prepare Reference List Vendor Step 2 Prepare and Send Survey Questionnaires to Past Clients Step 3 Collect/Receive Completed Surveys Step 4 Enter data into Reference List Step 5 Package all material(Reference List and Surveys) and Submit Past Performance Information • PPI will be collected on the critical Entities: • Firm • Project Manager (Individual) • Key Subs or Suppliers • Each Entity must prepare and submit a list of references, customer surveys, and Past Performance Information Scores

17. Reference List • Maximum number of references that can be submitted is 10 for each Entity. • The past projects must be installed and operational. • The reference list must contain different projects. • The reference list must contain different clients. • The client must complete the survey (no consultants or third parties)

18. Survey Questionnaire • Each Entity must prepare, send out, and collect survey questionnaires. • All returned survey MUST be evaluated AND signed by the client. If a survey is not signed, it will NOT be counted or considered. • Returned surveys must be submitted with Proposer's proposal (Proposer should keep a copy of all returned surveys).

19. PPI Score • Proposer shall input the survey scores, and then average all of the responses to obtain the “Overall Average Score”. • Proposer shall also count the “Total Number of Returned Surveys”. • The Client may contact the reference to clarify or to obtain additional information. If the reference cannot be contacted, the survey may be deleted.

20. Summary • PPI is only one step in the process. • PPI is not used as the sole factor to determine future performance. • The PBSRG has collected PPI in many different ways: • Sent surveys to past owners, called owners, collected data • Allowed vendors to send surveys, but collected the data • Allowed vendors to send and collect surveys • The goal is to be efficient. Research has shown that the most optimal way to collect PPI is to place the vendor at risk and allow them to be completely responsible for their PPI.

21. Comments / Questions W WW . P B S R G . C O M