By: Dr. Mamata Singh, Dr. Atul K. Mittal, and Dr. V. Upadhyay Presented by: Dr. V. Upadhyay

1. Benchmarking of Indian Urban Water Sector: Performance Indicator System versus Data Envelopment Analysis By: Dr. Mamata Singh, Dr. Atul K. Mittal, and Dr. V. Upadhyay Presented by: Dr. V. Upadhyay Professor Indian Institute of Technology, Delhi, Hauz Khas, New Delhi – 110016, India

2. Problems of Water Sector • Exponential population growth • Industrialization and urbanization • Infrastructure inadequacies • Inadequate funds • Inefficient water use

3. Objective of the Paper • To demonstrate the benchmarking approach using DEA and PIS methods • For the selected 12 Indian urban water utilities (municipal bodies) of Maharashtra state/province • Compare and analyze the results

4. Benchmarking Concept • A “benchmark” is a reference or measurement standard used for comparison • “Benchmarking” is the continuous activity of identifying, understanding and adapting best practice and processes that will lead to superior performance

5. Benchmarking Methods • Performance Indicator System - PIS (partial metric method) • Performance Relative to a Model Company (engineering approach –viz. DEA) • Performance Scores based on Production or Cost Estimates (“total” methods) • Process Benchmarking (involving detailed analysis of operating characteristics) • Customer Survey Benchmarking (identifying customer perceptions); and • SWOT (strengths, weaknesses, opportunities and threats) analysis

6. Performance Indicator System (PIS) • PIS comprises a set of performance indicators (PIs) and related data elements which represents the real instances of utilities’ context • Performance indicator (PI) is simply the ratio of an output to an input, or an input to an output • Each PI reflects only one input and one output level, so difficult to view overall performance of a DMU • Need to adopt a summary (or overall) measure of performance values • Methods of aggregating PIs are generally subjective • PIs may be tailored for different objectives and priorities

7. Data Envelopment Analysis (DEA) • Charnes, Cooper and Rhodes (1978) first introduced the term data envelopment analysis (DEA) • CCR considered constant returns to scale (CRS) model with input orientation • DEA is a linear programming technique that incorporates multiple inputs and outputs for assessing relative efficiencies • The most efficient utilities are rated to have an efficiency score of one, while the less efficient utilities score between zero and one • The utilities lying on efficient frontier are identified as best practice utilities by DEA

8. Inputs and Outputs in DEA • Resources utilized by the units or conditions affecting their operation are typical inputs • Measurable benefits generated or service levels of the utility constitute the outputs • Number of DMUs to be considered for DEA should at least be three times the sum total number of inputs and outputs • There should be positive correlation between inputs and outputs • The basic CRS-DEA model with an input orientation has been considered and detailed below

9. Discussion: PIS versus DEA Technique • DEA measures of performance are based on simultaneous consideration of multiple inputs and outputs, while each PI measures performance in relation to one input and one output only • A unit offering unremarkable values on individual PIs can still be deemed a good performer in the context of DEA when its all-round performance is taken into account • DEA is suitable for setting targets which would render a DMU relatively efficient but offers no indication of how relatively efficient DMUs might improve their performance • PIs do give indications as to the specific aspects of performance that a DMU might strengthen • DEA generally reflects overall, and PIs, factor-specific performance

10. Performance Indicators considered for the study

11. Performance Indicator System • Relative Performance Score (RPS) of a DMU = Overall Performance Score of that DMU/ Highest Overall Performance Score • The DMUs are then ranked on the basis of their RPS

12. DEA • Input Oriented CRS DEA Model • DEA is performed using free version of DEAF software (by Joe Zhu) to obtain their relative efficiency scores • DMUs are ranked on the basis of their relative efficiency scores

13. Inputs and Outputs for DEA

14. Relative Performance Score and Ranking using PIS

15. PIS Results • Wardha has a highest RPS of 1.0 (100%) and is ranked first • Yavatmal is ranked the last with least RPS of 0.34 • There are 3 DMUs with RPS < 50% and 4 DMUs with RPS > 75% ≤ 100% • For 5 DMUs RPS varies between 50-75% • Minor difference in overall performance scores of better performing utilities • Wardha and Bhusawal, • Bhusawal and Aurangabad, • Aurangabad and Chandrapur.

16. TE Score and Ranking using DEA

17. DEA Results • 4 DMUs Aurangabad, Bhusawal, Parbhani and Wardha have highest TE of 1.0 • Chandrapur has higher TE score of 0.850 • Yavatmal has the least TE score of 0.487 and is the only DMU with TE score < 50% • 2 DMUs with TE score < 75% and > 100% • 5 DMUs with TE score ranging between 50 - 75%

18. Potential for Input Reduction • Maximum for Yavatmal • Wardha, Bhusawal, Chandrapur and Prabhani deliver higher output levels at relatively lower input usage • For most of the DMUs, potential for % reduction in total expenditure and staff size are same (Amravati, Chandrapur, Kolhapur, Nanded Waghala, Solapur and Yavatmal) or closer (Nashik) except for a DMU Dhule • For Dhule, potential for % reduction in staff size is lesser than that of total expenditure as also implied by its PI values

19. Note: Higher is the ranking – lower is the performance level.

20. PIS vs DEA Ranking • Same for 6 UWUs (Amravati, Dhule, Nanded - Waghala, Solapur, Wardha and Yavatmal) • Differs maximum by six positions for the UWU Parbhani • Differs by one position for 2 UWUs (Bhusawal and Chandrapur) • Differ by two to four positions for rest of the 3 UWUs • Rank correlation coefficient is 0.833. The high correlation further indicates the fact that PIS and DEA methods agree strongly on the UWUs performance rankings • UWUs with lower ranking positions (Yavatmal, Solapur, Nanded Waghala, Amravati and Nashik) under both PIS and DEA methods

21. Conclusion • Better performing UWUs under both PIS and DEA are - Wardha, Aurangabad, Bhusawal and Chandrapur {except Prabhani) • Performance of Yavatmal is found to be lowest in both the methods • UWUs having lower performance levels under both PIS and DEA are - Yavatmal, Solapur, Nanded Waghala, Amravati and Nashik • PIS and DEA results are complimentary • DEA efficient DMU Prabhani may focus on reducing its staff size but not expenditure level as can be observed from PI values • DEA efficient DMU Bhusawal indicates its superior performance as regards staff size but reasonable scope for reduction in total expenditure

22. Recommendation • DEA inefficient DMUs need to focus on reducing their input (total expenditure and staff size) usage altogether • DEA efficient DMUs need to explore the opportunity for reduction in specific input usage (total expenditure or staff size or both) by analyzing PIS outcomes as well • Similar analysis may be performed by the state/independent regulator to devise suitable performance linked incentive mechanism

23. Thank You