Evaluation of us dispute resolution programs for environmental and complex public policy disputes l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 22

EVALUATION OF US DISPUTE RESOLUTION PROGRAMS FOR ENVIRONMENTAL AND COMPLEX PUBLIC POLICY DISPUTES PowerPoint PPT Presentation


  • 114 Views
  • Uploaded on
  • Presentation posted in: General

Andy Rowe GHK International g [email protected] www.ghkint.com. EVALUATION OF US DISPUTE RESOLUTION PROGRAMS FOR ENVIRONMENTAL AND COMPLEX PUBLIC POLICY DISPUTES. DISCLAIMER.

Download Presentation

EVALUATION OF US DISPUTE RESOLUTION PROGRAMS FOR ENVIRONMENTAL AND COMPLEX PUBLIC POLICY DISPUTES

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Evaluation of us dispute resolution programs for environmental and complex public policy disputes l.jpg

Andy Rowe

GHK International

[email protected]

www.ghkint.com

EVALUATION OF US DISPUTE RESOLUTION PROGRAMS FOR ENVIRONMENTAL AND COMPLEX PUBLIC POLICY DISPUTES


Disclaimer l.jpg

DISCLAIMER

This presentation has been prepared by the author and represents only my views. The agencies participating in or contracting the work on which this presentation is based do not necessarily subscribe to any or all of what is contained in the presentation.

Andy Rowe


Cheaper faster better l.jpg

CHEAPER FASTER BETTER?

  • There appears to be more belief than evidence underlying this claim

  • This is really a statement about program effectiveness:

    “Collaborative processes are more effective than a reasonable or likely alternative process”

  • The evaluation system being used by important U.S. federal and state agencies is starting to provide evidence for this claim of effectiveness


Background l.jpg

BACKGROUND

  • The evaluation system specifically addresses environmental and complex public policy disputes

  • Features of the system:

    • Takes a results-based accountability / performance management approach

    • Takes an improvement orientation

    • Addresses expected internal and external performance reporting requirements (state and federal)

    • Self administered

    • Has a high level of face validity for both ADR and evaluation professions

    • Is proving to have a high level of validity and reliability

  • Focus on utilization. Results will:

    • Be easy to interpret

    • Be useful for improving performance

    • Be easily comprehended by ADR professionals


Applications l.jpg

APPLICATIONS

  • U.S. Federal agencies

    • USEPA Conflict Prevention and Resolution Center

    • US Institute for Environmental Conflict Resolution

    • Other federal environmental agencies through contact to USIECR

  • U.S. State agencies

    • Oregon Dispute Resolution Commission

    • Florida Conflict Resolution Consortium

  • Other applications

    • Base for Maryland Alternative Dispute Resolution

    • Applied in pilot project of 9th Circuit


A performance management approach l.jpg

A PERFORMANCE MANAGEMENT APPROACH

  • Primary goal is to use evaluation as a mechanism to focus organizations on results and achieve success

    • While satisfying evaluation goals and standards

    • Address possible Government Performance Results Act (GPRA) requirements

  • John A. Koskinen’s three GPRA questions are what the evaluation attempts to address (John McLaughlin version):

    • What is your program / agency supposed to achieve?

    • How will you know when you have achieved this?

    • How are you doing now?


Implementation l.jpg

IMPLEMENTATION

  • Questionnaires to parties, practitioners and project manager at conclusion of the process and a follow-up to parties at a later date

    • Mail, web and email administration options

  • Response rates are high (will exceed 70% in Oregon pilot) and 100% of practitioners (included their contracts)

  • Information was systematically tested and has proved valid and reliable

  • Meets performance standards for the design

    • Managers are “can answer” to “fluent” for first two questions

    • Resource requirements = 10 hours per case for admin, analysis & reporting

    • Culture of the organizations shifts towards results focus

    • Internal stakeholders very satisfied, external not yet known


Outcome charts are the key to the design l.jpg

OUTCOME CHARTS ARE THE KEY TO THE DESIGN

  • Similar to logic models / logframes but much more focused on results

  • Outcome charts are best developed collaboratively with the program managers / staff

  • Illustrations that follow are from the Public Policy Program of the Oregon Dispute Resolution Commission

    • This is an ADR collaborative process – similar to applications in other agencies


Slide9 l.jpg

OREGON PUBLIC POLICY DISPUTE RESOLUTION PROGRAM OUTCOMES

Decreased cost of resolving disputes

Increased efficiency of agencies through use of ADR

Increase agency and public satisfaction with process and results of ADR

ADR in State Government

Case Management

Dispute Systems Designs

Oregon state agencies have practices and systems that ensure ADR is available

Practice of ADR supports agency mission, and receives positive feedback

ADR is successful (time and resources spent in disagreement and conflict are now redirected to more constructive purposes)

Dispute resolution systems are improved

Mediator Roster

Good agreements

Good Process

Roster provides expedited access to qualified and appropriate providers


Slide10 l.jpg

EXTRACTED OUTCOMES FOR OREGON CASE MANAGEMENT

ADR successful (resources spent in disagreement are now used more constructively)

Good outcomes from process

Good agreements

Party interactions constructive

All parties involved in an ADR process are satisfied that the process was fair and open.

ADR more effective (better benefits for resources expended) than other options for this dispute

Durable and implementable agreement reached using ADR

Agreements reached with ADR are complete - there are no hard issues left or deferred.

Right parties continue to be engaged, new parties added as required.

Use of ADR helps adversarial parties collaborate

Use of ADR narrows disagreements

Stakeholder capacity to use ADR is improved through experience with this case

ADR is used where it is the best approach for this case

Design of the ADR process is appropriate for the dispute and needs of parties

ADR used appropriately

Government decision making is improved through use of the process

Appropriate mediator leads ADR

Right parties engaged in process

Non-ADR processes used where the best approach


Simplifying complex practices l.jpg

Simplifying Complex Practices

  • Outcome charts seek to display the responses to the first of the GPRA questions:

    • What is my agency/unit trying to achieve?

  • The complexity of any one element of ADR practice is difficult to articulate in a single outcome but this complexity is required to respond to the second GPRA question:

    • How will we recognize this?

    • The mechanism I am currently using to simplify this is nested outcomes – mini outcome charts under individual outcomes

  • Even the nested outcomes can be at too high a level of generalization to be able to respond to the third GPRA questions

    • How are we doing now?

    • This is either addressed through further nesting or directly by the questions we ask


Slide12 l.jpg

At not time did one of the parties dominate to the detriment of the process or others

The neutral helped us manage our time well.

The neutral made sure that the concerns of all parties were heard.

The parties followed the direction of the neutral.

The neutral made sure that the concerns of all parties were addressed.

The neutral ensured that all parties were fully engaged in the process.

When things got tense the neutral was always able to find ways to move ahead constructively

ILLUSTRATION OF CHARTED AND NESTED OUTCOMES

Nested Outcomes

Expressed as questions

Nested Outcomes

Charted Outcome

Appropriate neutral – parties satisfied with neutral

Appropriate neutral – neutral understands issues

Appropriate Neutral Guides the Process

Appropriate neutral – manages process well

Appropriate neutral – available for process

Appropriate neutral – parties understand process


Other elements in the evaluation system l.jpg

OTHER ELEMENTS IN THE EVALUATION SYSTEM

  • Collaborative Processes

    • Program Intake

    • ECR/ADR/Collaborative Processes (longer and shorter forms

    • Conflict Assessment / Collaborative Assessment / Situation Assessment

    • Meetings: information gathering / public input

    • Dispute System Design

  • Other ADR Services

    • Training

    • Roster program

    • Contract for neutral services


The evaluation design is successful l.jpg

THE EVALUATION DESIGN IS SUCCESSFUL

  • Have established a program theory for ADR in environmental and complex public policy disputes

  • Approach is being implemented by a growing number of agencies

  • Positive peer external reviews (practice and evaluation)

  • Information is helping program improve

  • Information is addressing internal and external accountability requirements


Main remaining challenges l.jpg

MAIN REMAINING CHALLENGES

  • Incremental Benefits

    • Inherently difficult because of nature of many ADR cases

    • In some settings this can be addressed without difficulty

  • Effectiveness

    • Inherently difficult because of nature of most or perhaps all ADR cases

    • Lack of useful applied research in this area

  • Both incremental benefits and effectiveness are highly charged concepts in the field

    • Many strong believers have vigorously argued the merits of ADR based on intuitive appeal, inappropriate straw dogs, limited conceptual rigor and poor evidence

    • These are also linked to an even more charged debate between proponents of process versus proponents of agreements

  • There is a compelling logical and some empirical evidence that ADR will prove to deliver incremental benefit and be an effective approach when used appropriately


Challenge 1 incremental benefits l.jpg

CHALLENGE 1: INCREMENTAL BENEFITS

  • Disputes tend to be heterogeneous

    • Arises from some or all of: characteristics of the dispute, parties, practitioner, general setting in time and place

    • Very challenging to find a counterfactual

    • Counterfactuals will always need to be unique to the case

    • Disputes often engage a variety of resolution options over their life

  • Litigation is not the only alternative

    • When litigation is the likely alternative we can get some reasonable information

    • For many disputes the likely alternatives are: do nothing, legislation or administrative rule making

  • Addressing the challenge

    • The strength of the program theory, fact that claims made by the program theory are not exceptional  “likely to” should suffice as an evaluation judgment of incremental benefits


Challenge 2 effectiveness l.jpg

CHALLENGE 2: EFFECTIVENESS

  • Costs and benefits occur in three phases:

    • Prior to the dispute resolution process. Many costs have already occurred such as fact finding, establishing the technical, legal and economic claims, identifying rights

    • During the dispute resolution process. Main costs are time of parties, attorneys, practitioner, travel. Ability to pay can vary dramatically (e.g. dispute involving a major utility, environmental groups and individual small farmers)

    • After the dispute resolution process. The hypothesis is that a good agreement will significantly reduce costs

  • Shelf life of this information is relatively short

  • The classes and characteristics of costs and benefits have not been enumerated, nor weights or likely ranges


Addressing the challenges l.jpg

ADDRESSING THE CHALLENGES

  • Incremental Benefits

    • Use litigation as an alternative where feasible

    • Undertake GOOD case studies to create illustrative comparators that can be referred to

    • Use “likely to” approach in the interim

  • Effectiveness

    • Case studies to establish the categories, ranges and coefficients

    • Include limited number of cost questions in the meantime

    • Manage the dance of “doing no harm” by not collecting bad cost data in the interim


My claim l.jpg

MY CLAIM

  • Getting the organization to focus on results generates immediate and significant benefits

    • Outcome chart  program intent is clear = improved evaluability

    • 3 GPRA questions  requirement for monitoring outcomes = performance information

  • Worst case  Improved evaluation

    • Utilization enhanced

    • Can report on outcomes

    • Credible information for program improvement

  • Better case  Organizations adopt an evaluative and performance culture


Contact information l.jpg

CONTACT INFORMATION

Andy Rowe

GHK International

2707 Stratford Rd.

Columbia SC 29204

(803) 782-2182 voice (803) 782-3618 fax

[email protected]

www.ghkint.com


  • Login