Jesus M. de la Garza, Ph.D. Vecellio Professor Department of Civil and Environmental Engineering

1. Development of a Framework for the Efficiency Measurement of Road Maintenance Strategies using Data Envelopment Analysis Jesus M. de la Garza, Ph.D. Vecellio Professor Department of Civil and Environmental Engineering Virginia Tech Georgia Transportation Institute February 27, 2009

2. PERFORMANCE MEASUREMENT Surveys by the National Council on Public Works Improvement and the American Society of Civil Engineers indicate a substantial deterioration in the road system from 1988 to 2008 The Federal Highway Administration endorsed “asset management” to be the future approach of road maintenance for all state departments of transportation • Capital investment to build. • Maintenance to upkeep what is built For the last two decades, the road maintenance concept has been gaining a lot of attention

3. Performance of an organization or process should be measured with respect to different dimensions • Sink and Morris define performance as an “integrated relationship among seven dimensions: effectiveness, efficiency, quality, productivity, quality of work life, innovation, and profitability” • Effectiveness: the degree to which an output (product/service) conforms to the requirements • Efficiency: the degree to which the process produces the output (product/service) at a minimum resource level • The Transportation Research Board, in 2006, pointed out that: • The fundamental relationships between the maintenance level-of-service (i.e., effectiveness) and the budget requirements (i.e., efficiency) of road maintenance need more investigation • Best practices in specifying maintenance and operations performance need to be identified “Doing the right things” “Doing the things right”

4. It is challenging to measure the overall efficiency of a process which has multiple inputs/outputs Efficiency = Partial Efficiency Measure: Has a potential to result in serious misunderstandings about the overall efficiency of a process when only a single partial efficiency ratiois used Total Factor Efficiency Measure: May result in subjectivity as the decision-maker prescribes weightsto be assigned to each input and output variable Regression Analysis: Compares the efficiency of units against a hypothetical average performance System Dynamics: Requires the definition of the mathematical relationships between key variables

5. Mathematical method based on linear programming DEA can simultaneously deal with multiple outputs and multiple inputs Self-determined weights for inputs and outputs Units in the data set are called Decision-Making Units (DMUs) Data Envelopment Analysis (DEA) enables one to assess how efficiently a unit uses the resources available (inputs) to generate a set of outputs relative to other units in the data set

6. The challenge is to find the position of the efficient frontier and then compute the distance from it to each inefficient DMU to identify the efficiency score of such DMU DMUs located on the efficient frontier act as the benchmarks (peers) for the inefficient DMUs in the data set DEA models can also consider the effects of external and uncontrollable factors on the efficiency of DMUs The main idea of DEA is to construct a frontier of efficient DMUs representing the best practices

7. Identification of the efficient frontier formed by the efficient units Efficient Frontier Output 4 3 2 1 5 Input

8. Location of the efficient frontier changes based on the data set; hence the concept of relative efficiency Efficient Frontier Output 4 3 2 1 Input

9. A simple and hypothetical example: Computing the relative highway maintenance efficiency of 12 state DOTs • The DMUs are different state DOTs undertaking the maintenance operation in different but comparable (in terms of climate, traffic, etc.) sections of the highway

10. A graphical representation can be plotted for this DEA model 6’ Timeliness of Response / $ Spent Efficient Frontier (100% Efficient DMUs) Level of Service / $ Spent

11. Strengths of DEA • DEA can simultaneously deal with multiple outputs and multiple inputs • DEA does not require the specification of a priori weights for the input/output variables • In addition to identifying the efficient units, DEA identifies the peers for those units • DEA has the capability to consider external and uncontrollable factors while measuring the overall efficiency of DMUs • DEA focuses on the best-practice frontiers rather than the central tendency frontiers (as obtained through regression analyses)

12. Limitations of DEA • Errors in data for input-output variables may result in significant problems • Even though DEA can identify inefficiencies, it does not directly pinpoint the underlying causes of inefficiencies of DMUs. Nonetheless, it triggers the decision-makers to identify the differences between DMUs that might have resulted in the differences in efficiency scores • When there are more than two inputs and outputs, it is difficult to explain the process of DEA to the non-technical audience and/or decision-makers

13. Research Objective • Using the DEA approach to develop and implement a generic and comprehensive framework for measuring efficiency of Road Maintenance Strategies. • Data • 215 miles of Virginia’s Interstate over fiscal years 2003 to 2007, maintenance is performed by VDOTon a traditional basis • 250 miles of Virginia’s Interstate over fiscal years 2003 to 2005, maintained is performed by VMS using a performance base method

14. A DEA-based framework was developed and implemented for highway maintenance case to compare the relative highway maintenance efficiency of 8 counties of Virginia (215 miles of interstate) over a three-year period

15. What is a DMU in this Project? • Each Countyat each fiscal yearforms a DMU n County t Periods n*t DMUs

16. Virginia County Map

17. For the DEA model of paved lanes’ maintenance, a total of 22input/output variables and uncontrollable factors were identified

18. Model Specifications

19. AHP Method • AHP: A multi-criteria technique to find the relative importance of the factors affecting deterioration of the pavement.

20. AHP Method Environmental Harshness Factor = 0.38*(1/Load) + 0.10*(1/Traffic) + 0.29*(1/Precipitation) + 0.10*(Min Temperature) + 0.13*(1/Max Temperature)

21. Regression Method • Regressing output variables on the set of input variables: Regression Coefficients indicate the importance of the uncontrollable factors on the pavement deterioration

22. Regression Method Environmental Harshness Factor = -0.00028 *Load + 2.73 * Snowfall Quartile + 2.71 * Temp Diff Quartile

23. Environmental Classification Method • Dadson, et al. (2002) used the climate/terrain condition data, traffic data and the statistical techniques to developed an environmental classification to measure the relative rate of deterioration of bridge parts in different regions of the state of Virginia.

24. Comparison of Obtained Efficiency Scores NOTE: 31 DMUs out of 44 DMUs obtained the same efficiency scores in all three methods

25. Once the DEA model for the paved lanes’ maintenance was implemented, peer-relationship diagrams were developed to effectively communicate the findings to the decision-makers

26. Efficiency Evaluation Using Data Envelopment Analysis

27. Efficiency Frontier • AHP Method • IRI Output • Traditional & performance-based DMUs

28. By utilizing DEA, an analysis including a large number (if not all) of the state transportation agencies in the United States can be performed to identify: The efficiency performance of the state transportation agencies (with respect to different transportation components such as maintenance, operations, etc.) relative to each other Benchmarks (peers) and best practices DEA is used to perform comparative efficiency performance measurement for: Police forces (e.g., Wales, England, U.S.A.) Health care services (e.g,. Italy, U.S.A.) Nonprofit organizations (e.g., chapters of the American Red Cross) Banks Public schools (e.g., New York) Academic departments within a university This presentation strived to introduce the DEA concept to the audience as another tool that can be added to the “comparative performance measurement toolbox”

29. A National Cooperative Highway Research Program study (NCHRP 2003*) recommends, as a part of the action plan to address the challenges and opportunities facing the state DOTs, to initiate a national effort to identify the best practices and to benchmark the performance (with respect to different measures, e.g. effectiveness, efficiency, etc.) of peer states ARE WE THERE YET? *: NCHRP. (2003). "CEO Leadership Forum for State Departments of Transportation: A Summary Report." University of Minnesota Center for Transportation Studies.

30. For further information: Ozbek, M. E. (2007). "Development of a Comprehensive Framework for the Efficiency Measurement of Road Maintenance Strategies using Data Envelopment Analysis." PhD Dissertation, Virginia Polytechnic Institute and State University, Blacksburg. Ozbek, M. E., de la Garza, J. M., and Triantis, K. (2007). "Data and Modeling Issues Faced during the Efficiency Measurement of Road Maintenance using Data Envelopment Analysis." ASCE Journal of Infrastructure Systems. (In Review) Ozbek, M. E., de la Garza, J. M., and Triantis, K. (2007). "Efficiency Measurement of Bridge Maintenance using Data Envelopment Analysis." ASCE Journal of Infrastructure Systems. (In Review) Ozbek, M. E., de la Garza, J. M., and Triantis, K. (2008). "Data Envelopment Analysis as a Decision Making Tool for the Transportation Professionals." ASCE Journal of Transportation Engineering. (In Review) Ozbek, M. E., de la Garza, J. M., and Triantis, K. (2009). "Development of a Comprehensive Framework for the Efficiency Measurement of Road Maintenance Strategies using Data Envelopment Analysis." 12th AASHTO/TRB Maintenance Management Conference. July, Annapolis, MD. REFERENCES FOR FURTHER INFORMATION