alternative methods for measuring efficiency and an application of dea in education n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
ALTERNATIVE METHODS FOR MEASURING EFFICIENCY AND AN APPLICATION OF DEA IN EDUCATION PowerPoint Presentation
Download Presentation
ALTERNATIVE METHODS FOR MEASURING EFFICIENCY AND AN APPLICATION OF DEA IN EDUCATION

Loading in 2 Seconds...

play fullscreen
1 / 37

ALTERNATIVE METHODS FOR MEASURING EFFICIENCY AND AN APPLICATION OF DEA IN EDUCATION - PowerPoint PPT Presentation


  • 532 Views
  • Uploaded on

Emmanuel Thanassoulis Professor in Management Sciences Aston Business School University of Aston Birmingham B4 7ET Tel: +44(0) 121 3593611 Ext 5033 Fax: +44 (0) 121 359 5271 Email: e.thanassoulis@aston.ac.uk.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'ALTERNATIVE METHODS FOR MEASURING EFFICIENCY AND AN APPLICATION OF DEA IN EDUCATION' - lotus


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
alternative methods for measuring efficiency and an application of dea in education
Emmanuel Thanassoulis

Professor in Management Sciences

Aston Business School

University of Aston

Birmingham B4 7ET

Tel: +44(0) 121 3593611 Ext 5033

Fax: +44 (0) 121 359 5271

Email: e.thanassoulis@aston.ac.uk

ALTERNATIVE METHODS FOR MEASURING EFFICIENCY AND AN APPLICATION OF DEA IN EDUCATION
presentation outline
PRESENTATION OUTLINE

PART 1: OVERVIEW OF COMPARATIVE EFFICIENCY ASSESSMENT METHODS

  • A conceptualisation of comparative efficiency measurement
  • How Corrected and Modified OLS regression work.
  • How Stochastic Frontier Analysis (SFA) works.
  • How Data Envelopment Analysis (DEA) works in outline.

PART 2: A REAL LIFE APPLICATION OF DEA ON BEHALF OF THE UK DEPARTMENT FOR EDUCATION AND SKILLS

  • Decomposing pupil attainment between school and pupil effects using DEA.
  • Managerial information derived through decomposing pupil and school effects using DEA.
slide3

PART 1: OVERVIEW OF COMPARATIVE EFFICIENCY ASSESSMENT METHODS

Useful references on parametric efficiency assessment methods:

  • An Introduction to Efficiency and Productivity Analysis by Tim Coelli, D S Prasada Rao and George Battese (1998) Kluwer Academic Publishers (chapters 2, 3, 8, 9) ISBN: 0792380622
  • S.C. Kumbhakar and C.A. Knox Lovell (2000) Stochastic Frontier Analysis (Cambridge University Press) ISBN: 0521481848
  • Greene W H (1998) Frontier Production Functions in H. Peasaran and P. Schmidt (eds) Handbook of Applied Econometrics, volume 2, microeconomics (Blackwells)
the problem of comparative efficiency
THE PROBLEM OF COMPARATIVE EFFICIENCY

As an integral part of managing organisations in the public and private sector we need to have information such as:

- How efficiently are the operating units using their resources?

- Is the industry such that there are economies of scale?

- Is the industry such that there are economies of scope?

- What are the efficient marginal costs of outputs;

- Has there over time been productivity change within the industry?

- How has a given operating unit fared on productivity change?

the fundamental view of the problem

Inputs

Outputs

Transformation

THE FUNDAMENTAL VIEW OF THE PROBLEM

The units to be assessed transform inputs into outputs

The basic requirement is to compare the Decision Making Units (DMUs) on the levels of outputs they secure relative to their input levels.

measures of comparative efficiency

Inputs

Outputs

Transformation

MEASURES OF COMPARATIVE EFFICIENCY

In a given operating context the measure of efficiency is normally one of:

- The distance between observed and maximum possible output for given inputs (output efficiency);

- The distance between observed and minimum possible input for given outputs (input efficiency);

slide7

There are two broad types of method for arriving at measures of comparative efficiency: parametric and non-parametric methods.

The parametric methods typically hypothesise a functional form and use the data to estimate the parameters of that function. The estimated function is then used to arrive at estimates of the efficiencies of units.

The non-parametric methods, best known as Data Envelopment Analysis (DEA), create virtual units to act as benchmarks for measuring comparative efficiency.

slide8

PARAMETRIC METHODS FOR COMPARATIVE EFFICIENCY MEASUREMENT

Hypothesise a production function. E.g. the expression

[1]

Where y is output, xik are inputs, and ei is the residualfor firm i.

It is the residual ei the captures any inefficiency.

Unfortunately, the residual also captures other random effects (e.g. omitted variables, measurement error, etc.) which makes it difficult to disentangle the component of inefficiency.

Two approaches exist: ignoring and not ignoring the random effects.

slide9

IGNORING RANDOM EFFECTS IN THE RESIDUAL

Let us treat the residual ei in [1] as capturing ONLY inefficiency ignoring other random effects. Then the model becomes

[2]

where

ui >= 0.

Now suppose we use OLS regression to estimate the model in [2].

OLS regression assumes implicitly the ui have zero mean.

Therefore we need to adjust the OLS model, to get the true residuals ui.

Normally one of two types of adjustment is used: Corrected or Modified OLS.

slide10

They work as follows.

The OLS model we estimated when we assumed residual mean of zero was in effect model [2] implicitly modified to

[3]

Let a* = [a-E(ui)] be the intercept of [3] that OLS regression yields. In order to retrieve the true underlying model in [2] we need to add E(ui) to a*.

The problem is that E(ui) is an estimate of the mean inefficiency of firms which we do NOT know.

To get an estimate of the mean inefficiency of firms we make an assumption about the theoretical probability density function of ui and then use the OLS regression residuals

to estimate .

=

slide11

Three estimates of normally are used as follows:

COLS.

Adjust the intercept a* of the OLS model in [3] to

where ei is the residual for the ith firm in model [3] (OLS model).

MOLS - half normal

If ui is assumed to follow a half-normal distribution

adjust the intercept a* of the OLS model in [3] to

where

MOLS - exponential

If ui is assumed to follow an exponential distribution

adjust the intercept a* of the OLS model in [3] to

where

slide12

The exponential and half-normal assumption reflect the belief that larger values of inefficiency are less likely.

slide13

frontiers for Penn 90 data

12

11

10

OLS

log gdp per worker

LP/QP

COLS

9

MOLS exp

8

7

6

5

6

7

8

9

10

11

12

log capital per worker

Illustrative application of COLS/MOLS to a set of data.

log gdp/worker

Inefficiency of country A: The colour of the arrow identifies the referent boundary

MOLS half-normal

A

Adapted from Weyman-Jones lecture notes, Aston Business School

slide14

NOT IGNORING RANDOM EFFECTS IN THE RESIDUAL: Stochastic Frontier Analysis (SFA)

Key departure from COLS and MOLS is that we now have a composed error term

v is an identically distributed conventional two sided error term with zero mean. It stands for random noise, omitted variables etc.

u is an identically distributed one sided error termwith anon-zeromean. It stands for inefficiency.

u is typically assumed to be exponential, half-normal or truncated normal.

slide15

Illustration of the composed error term ei = vi - ui when ui is half normal.

Note the negative skew. We use a test for negative skew to decide whether there is evidence of inefficiency among the producers.

If

we reject the SFA model.

Source Tom Weyman Jones, lecture notes Aston Business School.

slide16

The SFA model is usually fitted using Maximum Likelihood estimation.

We need to estimate the inefficiency of the ith producer (ui) by using its composed residual ei = vi - ui .

Depending on the assumption we make about the distribution of the inefficiency ui we arrive at a different formula for the conditional value

We plug into this formula the values of ei and other values we derive from the ei to arrive at an estimate of the conditional inefficiency ui of the ith producer.

The formulae differ depending on the distribution assumed for ui but are coded in available software such as Limdep.

slide17

A GRAPHICAL OUTLINE OF THE BASIC DEA MODEL FOR ASSESSING COMPARATIVE EFFICIENCY

An introduction to Data Envelopment Analysis can be found

in:

E. Thanassoulis (2001) Introduction to the Theory and Application of

Data Envelopment Analysis: A foundation text with integrated software.

Kluwer Academic Publishers, Boston, Hardbound, ISBN 0-7923-7429-0

by virtue of interpolation between c and d all input output correspondences on cd are feasible
By virtue of interpolation between C and D all input - output correspondences on CD are feasible
slide20

Using interpolations between observed units the set of all feasible input -output correspondences is constructed and its boundary identified

slide21

Input Efficiency of E: HI/HE

  • Scope for resource conservation at E:IE

A Closer Look at DEA

Using the set of all feasible input output correspondences the comparative efficiency and other information in respect of a unit (e.g. unit E) is derived as illustrated here:

  • Output Efficiency of E:FE/FG
  • Output benchmarks for E:Units C and D
  • Scope for output augmentation at E:EG
  • Returns to scale (increasing, decreasing, constant):Revealed by the intercepts of the segments of the efficient boundary.
  • Scale elasticity revealed by the slope of the segments on the efficient boundary.
contrasting the alternative efficiency assessment methods
Contrasting the Alternative Efficiency Assessment Methods

COLS

Output

DEA

12

E

  • All depend on identifying a reference boundary relative to which efficiency is assessed.
  • DEA is non-parametric, can handle multiple inputs and outputs but assumes all distance from the boundary is inefficiency.
  • SFA allows for random noise in the distance from the boundary but needs assumptions on inefficiency distribution;
  • DEA reveals unit-specificpeers, type of returns to scale, productivity change.
  • SFA and regression methods reveal industry level information.
  • COLS/MOLS can be very susceptible to outliers and do not allow for random noise.

D

G

C

10

SFA

B

8

6

A

F

4

OLS Regression

2

H

0

2

4

6

8

10

Input

slide23

PART 2: A REAL LIFE APPLICATION OF DEA ON BEHALF OF THE UK DEPARTMENT FOR EDUCATION AND SKILLS

slide24

DECOMPOSING PUPIL ATTAINMENT USING DEA

One non - DEA approach which uses pupil - level data is

Multi-level Modelling. It recognises the nested nature of school data (pupil within school, school within LEA etc.).

Most studies measuring school performance use school level data and are parametric (regression-based).

It decomposes variance in pupil attainment into pupil, school etc. effects.

In a similar manner our approach uses pupil level data in a DEA framework to decompose pupil attainment into any number of components (pupil, school, type of school, gender etc.)

how the decomposition works
How the Decomposition Works

Take pupils in a number of schools.

We have a two-level model [pupil is level 1, school is level 2].

Any difference in attainment by pupils is as a result of a combination of:

- Random Noise;

- School effectiveness;

- Differences in effort made by pupils.

Our approach allows for random noise and attempts to separate school from pupil effects.

how the decomposition works1
How the Decomposition Works

The graph shows data from school 1(dots) and school 2 (crosses)

BCD - Pupil-within-all-schools efficient boundary

BCH and GDPupil- within-school efficient boundaries

OZ/OZ’ = Pupil-within-school efficiency measure of pupil Z.

OZ/OZ’’ = Pupil-within-all-schools efficiency measure of pupil Z.

OZ/OZ’’ = OZ/OZ’ * OZ’/OZ’’

OZ’/OZ’’ is a measure of school-within-all-schools efficiency at pupil Z

slide27
Pupil-Within-All-Schools DEA efficiency =

Component attributable to pupil Component attributable to school

application of the methodology
Application of the Methodology
  • Pupils of 122 schools who sat GCSE’s in 1992 and A or AS levels in 1994 were assessed.

DEA Input - Output Variables:

estimation of dea efficiencies
Estimation of DEA efficiencies

Given a set of n pupils, the DEA efficiency of pupil j0 relative to that set is 100/j0* % where j0* is the optimal value of in:

Max 

GCSEptsjj <= GCSEptsj0

GCSEpts_attjj < = GCSEpts_attj0

Aptsjj > =  Aptsj0

Apts_attjj > =  Apts_attj0

j = 1

S.t.

j ,  0,  free

decomposing pupil within all schools efficiencies
Decomposing Pupil-within-All Schools Efficiencies:

EFFwj0 - pupil-within-school efficiency

EFFij0- pupil-within-all-schools efficiency EFFwj0

EFFsjo- school-within-all-schools efficiency

EFFij0=EFFsjo * EFFwj0

summary statistics of efficiency scores
Summary Statistics of Efficiency Scores

Results obtained considering all the pupils sampled irrespective of school:

-The median pupil attains only 65% of the A-level scores of the benchmark pupil(s) in his/her own school;-The median pupil attains only 46.6% of the A-level scores of the benchmark pupil(s) across schools-The median pupils benchmark within own school attain only 82.76% of the benchmark pupils across schools.

slide32

Attribution of efficiencies

Mean school-within-all-schools efficiency:

School 121 93.8%

School 60 60.56%.

identifying differential school effectiveness
Identifying Differential School Effectiveness

- School 121 is more effective with stronger pupils

- School 2 is more effective with weaker pupils

A school has differential effectiveness if it has a different effect on different groups of pupils.

GCSE points per attempt can be taken as indicative of the innate ability of a pupil

pupil targets and peers
Pupil targets and peers

Pupils targets can incorporate several components

of interest both to the pupil and to the school:

- Within-school targets - achievement of these targets depends only on the pupil. (Peers are within-school.)

- Within-all-schools targets - A pupil can reach these by a combination of reaching within school targets while his/her school becomes more effective. (Within-all-schools peers can indicate role model schools.)

other decompositions possible
Other Decompositions Possible

We could in effect control for any number of categorical variables and estimate the impact of each one on pupil attainment: E.g:

  • Gender ( Estimate efficient boundaries within each gender and then pool the genders to compare the distance of the two boundaries.)
  • Type of school ( Estimate efficient boundaries within each type of school and then pool the types of school to compare the distance of the boundaries.)
  • Socio-economic factors ( Estimate efficient boundaries within each group of pupils (e.g. eligible v non-eligible for free school meals) and then pool the samples to compare the distance of the boundaries.)
  • Combinations of categorical variables (E.g. Gender and eligibility for free school meals) Estimate efficient boundaries within each combination of factors and then pool the samples to compare the distance of the boundaries.)
conclusion
Conclusion
  • The approach outlined recognises that pupil attainment is a combination of various effects including effort by the pupil, the school and other categorical factors.
  • It can provide more complete information to schools, teachers and other parties in managing school and pupil performance. Eg: - targets for individual pupils using other pupils within the school as benchmarks; - targets for schools to raise their boundary closer to the inter-school boundary; - identification of any differential school effectiveness with a view to its elimination or management.