Independent evaluation insights from public accounting
This presentation is the property of its rightful owner.
Sponsored Links
1 / 36

Independent Evaluation: Insights from Public Accounting PowerPoint PPT Presentation


  • 61 Views
  • Uploaded on
  • Presentation posted in: General

Independent Evaluation: Insights from Public Accounting. Abigail Brown (Harvard) Jacob Alex Klerman (Abt Associates) Presentation for Improving the Quality of Public Services, Moscow, June 2011. This Paper/Presentation ….

Download Presentation

Independent Evaluation: Insights from Public Accounting

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Independent evaluation insights from public accounting

Independent Evaluation: Insights from Public Accounting

Abigail Brown (Harvard)

Jacob Alex Klerman (Abt Associates)

Presentation for Improving the Quality of Public Services, Moscow, June 2011


This paper presentation

This Paper/Presentation …

  • Reviews the history, theory, and experience in “independent audits” for insights about “independent evaluations”

  • Our thesis: In both fields

    • The disjunction between who oversees the audit/evaluation and the ultimate consumers of the results

    • Leads to problematic incentives for the auditor/evaluator and bad outcomes

    • But that recognizing the “disjunction” and the experience in auditing suggest several feasible changes that would improve the situation

Brown & Klerman

Brown & Klerman

May 9, 2011

2


Outline

Outline

The Auditing Problem(s)

Some Auditing “Solutions”

The Analogy to Evaluation

Discussion

Brown & Klerman

Brown & Klerman

May 9, 2011

3


Owners and managers

Provide the capital

Owners and Managers

Owners

  • Managers run the firm day to day

Managers

Brown & Klerman


But who watches the mangers

Owners are too busy to “watch well”

But need to “watch”

Malfeasance

Insufficient effort

Hire/Fire

Incentive compensation (bonuses, stock options)

But Who Watches the Mangers?

Owners

Managers

Brown & Klerman


Enter independent auditor

Paid by owners(corporate funds)

Enter “Independent Auditor”

Owners

  • Origins in 1840s England

  • Required by 1934 (U.S.) Securities Exchange Act

Auditor

To watch (and report on) managers

Managers

Brown & Klerman


But why should this work

But Why Should this Work?

  • Managers

    • Choose auditors—this time and next time—and how much to pay them

    • Oversee auditor’s contract, including scope

    • Can give auditors other (non-auditing/consulting) work

  • So why should auditors report truthfully to owners?

    • Why not just report what managers want reported?

Paid by owners(corporate funds)

Owners

Auditor

To watch (and report on) managers

Managers

Brown & Klerman


Outline1

Outline

The Auditing Problem(s)

Some Auditing “Solutions”

The Analogy to Evaluation

Discussion

Brown & Klerman

Brown & Klerman

May 9, 2011

8


Four partial solutions

Professionalism

Reputation

Liability

Owner Oversight

Four Partial Solutions

Brown & Klerman


Aicpa code of conduct

Professionalism

Reputation

Liability

Owner Oversight

A distinguishing mark of a profession is acceptance of its responsibility to the public. …

In discharging their professional responsibilities, members may encounter conflicting pressures from among each of those groups. In resolving those conflicts, members should act with integrity, guided by the precept that when members fulfill their responsibility to the public, clients' and employers' interests are best served. ..

Those who rely on certified public accountants expect them to discharge their responsibilities with integrity, objectivity, due professional care, and a genuine interest in serving the public.

AICPA Code of Conduct

Brown & Klerman


A simple equilibrium argument

Professionalism

Reputation

Liability

Owner Oversight

Suppose there are two types of auditors— “independent” and “not independent”

The auditor selected by management is publicly observable

What would owners conclude if management chose a “not independent” auditor?

So management has to choose an “independent auditor”

A Simple Equilibrium Argument

Brown & Klerman


Sue them

Professionalism

Reputation

Liability

Owner Oversight

Auditors can be sued for errors

Have been large payouts

Should be a strong financial incentive for “independence”

But, liability requires

Clear standards for malfeasance—which the profession has tried hard to avoid

Strong liability—which the profession has successfully weakened

Sue Them …

Brown & Klerman


Sox sarbanes oxley act of 2002

Professionalism

Reputation

Liability

Owner Oversight

If the problem is too little owner (too much management) involvement in the audit, fix that directly!

SOX requires that an “Independent Audit Committee of BoD/Board of Directors” (i.e., excluding management)

Hire the auditor

Oversee the audit

Approve other business with the auditor (to prevent side payments)

SOX/Sarbanes-Oxley Act of 2002

Brown & Klerman


Outline2

Outline

The Auditing Problem(s)

Some Auditing “Solutions”

The Analogy to Evaluation

Discussion

Brown & Klerman

Brown & Klerman

May 9, 2011

14


Evaluation is like auditing

Evaluation is Like Auditing

What are the implications of experience in “independent auditing” for “independent evaluation”?

Brown & Klerman

Brown & Klerman

May 9, 2011

15


Applying these strategies to evaluation

Professionalism

Reputation

Liability

Owner Oversight

Applying these Strategies to Evaluation

Brown & Klerman


Same problems as in auditing

Professionalism

Reputation

Liability

Owner Oversight

Also problematic in evaluation

Evaluation is also a business

Stakes are high: own employment and bonus

Perhaps professionalism more effective in evaluation

Public release of results and methods

Results published in academic journals

Results reviewed and used by academics

Same Problems as in Auditing

Brown & Klerman


Does reputation matter

Professionalism

Reputation

Liability

Owner Oversight

To some extent

But clients also seem to value (perhaps value more) a reputation for

“Client service”

Not revealing client’s secrets

Not publishing without permission

Does Reputation Matter?

Brown & Klerman


Liability does not seem operative

Professionalism

Reputation

Liability

Owner Oversight

There does not seem to be any statutory basis on which to sue evaluators for failure to be “independent”

Who would sue?

For what?

What are the damages?

Ideas for making this work?

Liability Does Not Seem Operative

Brown & Klerman


0wner oversight seems promising

Professionalism

Reputation

Liability

Owner Oversight

Currently, agency

Selects the evaluator

Oversees the evaluation

Reviews the report and any presentations

Up to and including (indefinitely) preventing publication and presentation

For evaluation seems easier to change

By analogy with SOX Audit Committee reforms

“0wner”Oversight Seems Promising

Brown & Klerman


By analogy with sox audit committee

Professionalism

Reputation

Liability

Owner Oversight

Expand membership of TWG/Technical Working Group to include

Congress (e.g., CBO, GAO)

Higher executive (e.g., OMB, agency’s Inspector General, agency’s PA&E)

General policy community (academics, think tanks from both sides)

Change role of TWG

Select the evaluator

Oversee the evaluation

Review the report and any presentations

Decide what and when to release reports

By Analogy with SOX, Audit Committee …

Brown & Klerman


Encourage independence by

Professionalism

Reputation

Liability

Owner Oversight

Agency comments on, but need not approve, materials going to TWG

Remove restrictions on evaluator contact with the TWG

Encourage reporting of attempts by agency to subvert independence of the evaluation

Encourage Independence by …

Brown & Klerman


Outline3

Outline

The Auditing Problem(s)

Some Auditing “Solutions”

The Analogy to Evaluation

Discussion

Brown & Klerman

Brown & Klerman

May 9, 2011

23


Owner oversight in practice

“Owner Oversight” in Practice

  • Explicitly specify which tasks are “independent evaluations”

    • Sometimes in statute—specific evaluations, certain “evaluation funds”

    • Sometimes by programs wanting strong evidence of effectiveness

  • Define “independent evaluation” in the FAR/Federal Acquisition Regulations; i.e.,

    • A TWG that includes key consumers, and

    • Rules for how the TWG would operate

    • Limit other business contact between “independent evaluators” and agencies (to prevent “side payments”)

Brown & Klerman

Brown & Klerman

May 9, 2011

24


Plus facilitate market for reputation

Plus, Facilitate Market for Reputation

  • Publicize deviations attempts to subvert independence

    • By the agency, and

    • By the evaluator

  • Distinguish “independent evaluations” from others

    • In making funding decisions (e.g., official policy reviews by NAS/IOM)

    • At conference presentations and at journal publication

Brown & Klerman

Brown & Klerman

May 9, 2011

25


Outline4

Outline

Introduction

The Auditing Problem(s)

Some Auditing “Solutions”

The Analogy to Evaluation

Discussion

Brown & Klerman

Brown & Klerman

May 9, 2011

26


Independent evaluation insights from public accounting1

Independent Evaluation: Insights from Public Accounting

Abigail Brown (Harvard)

Jacob Alex Klerman (Abt Associates)

Presentation for Improving the Quality of Public Services, Moscow, June 2011


Evaluation is like auditing1

Evaluation is Like Auditing

Brown & Klerman

Brown & Klerman

May 9, 2011

28


Owners and managers1

Provide the capital

Owners and Managers

Owners

For most of human history, these were the same.

  • Managers run the firm day to day

Managers

Brown & Klerman


Owners and managers2

Separation of “owners” (i.e., stockholders) and “control” (i.e., management) is a cornerstone of the modern corporation

Allowing:

Poor entrepreneurs to start companies/implement their ideas

Owners to invest all of their money and to diversify

Owners and Managers

Owners

Managers

Brown & Klerman


And the issues are subtle

And the Issues Are Subtle

  • Pure fraud is clear, but

    • How much to spend looking?

    • Where to look?

  • Management often has a desired “spin”; owners may want something else

    • GE’s earnings smoothing

    • Lehman’s “Repo 105”

  • Good accounting requires judgment

    • When judgment is required, which way does the accountant lean?

Paid by owners(corporate funds)

Owners

Auditor

To watch (and report on) managers

Managers

Brown & Klerman


Independent evaluation insights from public accounting

Professionalism

Reputation

Liability

Owner Oversight

Their own private (financial) interests are large

Continued employment, repeat audit business, consulting business

Auditing’s professional ethos emphasizes confidentiality and client (i.e., management) service

Not independence (and serving owners)

But …

Brown & Klerman


Deviation should be costly

Professionalism

Reputation

Liability

Owner Oversight

Now suppose management suggests “creative accounting”

Perhaps with some “side payment” to the auditor (future auditing, consulting work, etc.)

Auditor weighs

Side payment vs.

Loss of reputation as an “independent auditor” if side payment is discovered

Deviation Should be Costly

Brown & Klerman


Does this reputation work

Professionalism

Reputation

Liability

Owner Oversight

Yes:

Arthur Anderson collapsed after criminal conviction in relation to Enron scandal

No:

Other accounting scandals did not cause similar collapse (Arthur Anderson was the exception)

Some evidence of collusion among firms (rather than competition on “independence”)

Too few to fail (currently only four large accounting firms)

Does this “Reputation” Work?

Brown & Klerman


Will sox work

Professionalism

Reputation

Liability

Owner Oversight

Duh! Wasn’t this obvious?

This was “best practice” before SOX; but inconsistently applied

Why not? And what does that suggest?

Management often controls the BoD

Management often “suggests” BoD members

Who are not independent—relatives, interlocking BoD, other business relationships

And board membership is lucrative, so members want to stay in management’s good graces

Will SOX Work?

Brown & Klerman


Evaluation is like auditing2

Evaluation is Like Auditing

Brown & Klerman

Brown & Klerman

May 9, 2011

36


  • Login