1 / 36

Independent Evaluation: Insights from Public Accounting

Independent Evaluation: Insights from Public Accounting. Abigail Brown (Harvard) Jacob Alex Klerman (Abt Associates) Presentation for Improving the Quality of Public Services, Moscow, June 2011. This Paper/Presentation ….

gaston
Download Presentation

Independent Evaluation: Insights from Public Accounting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Independent Evaluation: Insights from Public Accounting Abigail Brown (Harvard) Jacob Alex Klerman (Abt Associates) Presentation for Improving the Quality of Public Services, Moscow, June 2011

  2. This Paper/Presentation … • Reviews the history, theory, and experience in “independent audits” for insights about “independent evaluations” • Our thesis: In both fields • The disjunction between who oversees the audit/evaluation and the ultimate consumers of the results • Leads to problematic incentives for the auditor/evaluator and bad outcomes • But that recognizing the “disjunction” and the experience in auditing suggest several feasible changes that would improve the situation Brown & Klerman Brown & Klerman May 9, 2011 2

  3. Outline The Auditing Problem(s) Some Auditing “Solutions” The Analogy to Evaluation Discussion Brown & Klerman Brown & Klerman May 9, 2011 3

  4. Provide the capital Owners and Managers Owners • Managers run the firm day to day Managers Brown & Klerman

  5. Owners are too busy to “watch well” But need to “watch” Malfeasance Insufficient effort Hire/Fire Incentive compensation (bonuses, stock options) But Who Watches the Mangers? Owners Managers Brown & Klerman

  6. Paid by owners(corporate funds) Enter “Independent Auditor” Owners • Origins in 1840s England • Required by 1934 (U.S.) Securities Exchange Act Auditor To watch (and report on) managers Managers Brown & Klerman

  7. But Why Should this Work? • Managers • Choose auditors—this time and next time—and how much to pay them • Oversee auditor’s contract, including scope • Can give auditors other (non-auditing/consulting) work • So why should auditors report truthfully to owners? • Why not just report what managers want reported? Paid by owners(corporate funds) Owners Auditor To watch (and report on) managers Managers Brown & Klerman

  8. Outline The Auditing Problem(s) Some Auditing “Solutions” The Analogy to Evaluation Discussion Brown & Klerman Brown & Klerman May 9, 2011 8

  9. Professionalism Reputation Liability Owner Oversight Four Partial Solutions Brown & Klerman

  10. Professionalism Reputation Liability Owner Oversight A distinguishing mark of a profession is acceptance of its responsibility to the public. … In discharging their professional responsibilities, members may encounter conflicting pressures from among each of those groups. In resolving those conflicts, members should act with integrity, guided by the precept that when members fulfill their responsibility to the public, clients' and employers' interests are best served. .. Those who rely on certified public accountants expect them to discharge their responsibilities with integrity, objectivity, due professional care, and a genuine interest in serving the public. AICPA Code of Conduct Brown & Klerman

  11. Professionalism Reputation Liability Owner Oversight Suppose there are two types of auditors— “independent” and “not independent” The auditor selected by management is publicly observable What would owners conclude if management chose a “not independent” auditor? So management has to choose an “independent auditor” A Simple Equilibrium Argument Brown & Klerman

  12. Professionalism Reputation Liability Owner Oversight Auditors can be sued for errors Have been large payouts Should be a strong financial incentive for “independence” But, liability requires Clear standards for malfeasance—which the profession has tried hard to avoid Strong liability—which the profession has successfully weakened Sue Them … Brown & Klerman

  13. Professionalism Reputation Liability Owner Oversight If the problem is too little owner (too much management) involvement in the audit, fix that directly! SOX requires that an “Independent Audit Committee of BoD/Board of Directors” (i.e., excluding management) Hire the auditor Oversee the audit Approve other business with the auditor (to prevent side payments) SOX/Sarbanes-Oxley Act of 2002 Brown & Klerman

  14. Outline The Auditing Problem(s) Some Auditing “Solutions” The Analogy to Evaluation Discussion Brown & Klerman Brown & Klerman May 9, 2011 14

  15. Evaluation is Like Auditing What are the implications of experience in “independent auditing” for “independent evaluation”? Brown & Klerman Brown & Klerman May 9, 2011 15

  16. Professionalism Reputation Liability Owner Oversight Applying these Strategies to Evaluation Brown & Klerman

  17. Professionalism Reputation Liability Owner Oversight Also problematic in evaluation Evaluation is also a business Stakes are high: own employment and bonus Perhaps professionalism more effective in evaluation Public release of results and methods Results published in academic journals Results reviewed and used by academics Same Problems as in Auditing Brown & Klerman

  18. Professionalism Reputation Liability Owner Oversight To some extent But clients also seem to value (perhaps value more) a reputation for “Client service” Not revealing client’s secrets Not publishing without permission Does Reputation Matter? Brown & Klerman

  19. Professionalism Reputation Liability Owner Oversight There does not seem to be any statutory basis on which to sue evaluators for failure to be “independent” Who would sue? For what? What are the damages? Ideas for making this work? Liability Does Not Seem Operative Brown & Klerman

  20. Professionalism Reputation Liability Owner Oversight Currently, agency Selects the evaluator Oversees the evaluation Reviews the report and any presentations Up to and including (indefinitely) preventing publication and presentation For evaluation seems easier to change By analogy with SOX Audit Committee reforms “0wner”Oversight Seems Promising Brown & Klerman

  21. Professionalism Reputation Liability Owner Oversight Expand membership of TWG/Technical Working Group to include Congress (e.g., CBO, GAO) Higher executive (e.g., OMB, agency’s Inspector General, agency’s PA&E) General policy community (academics, think tanks from both sides) Change role of TWG Select the evaluator Oversee the evaluation Review the report and any presentations Decide what and when to release reports By Analogy with SOX, Audit Committee … Brown & Klerman

  22. Professionalism Reputation Liability Owner Oversight Agency comments on, but need not approve, materials going to TWG Remove restrictions on evaluator contact with the TWG Encourage reporting of attempts by agency to subvert independence of the evaluation Encourage Independence by … Brown & Klerman

  23. Outline The Auditing Problem(s) Some Auditing “Solutions” The Analogy to Evaluation Discussion Brown & Klerman Brown & Klerman May 9, 2011 23

  24. “Owner Oversight” in Practice • Explicitly specify which tasks are “independent evaluations” • Sometimes in statute—specific evaluations, certain “evaluation funds” • Sometimes by programs wanting strong evidence of effectiveness • Define “independent evaluation” in the FAR/Federal Acquisition Regulations; i.e., • A TWG that includes key consumers, and • Rules for how the TWG would operate • Limit other business contact between “independent evaluators” and agencies (to prevent “side payments”) Brown & Klerman Brown & Klerman May 9, 2011 24

  25. Plus, Facilitate Market for Reputation • Publicize deviations attempts to subvert independence • By the agency, and • By the evaluator • Distinguish “independent evaluations” from others • In making funding decisions (e.g., official policy reviews by NAS/IOM) • At conference presentations and at journal publication Brown & Klerman Brown & Klerman May 9, 2011 25

  26. Outline Introduction The Auditing Problem(s) Some Auditing “Solutions” The Analogy to Evaluation Discussion Brown & Klerman Brown & Klerman May 9, 2011 26

  27. Independent Evaluation: Insights from Public Accounting Abigail Brown (Harvard) Jacob Alex Klerman (Abt Associates) Presentation for Improving the Quality of Public Services, Moscow, June 2011

  28. Evaluation is Like Auditing Brown & Klerman Brown & Klerman May 9, 2011 28

  29. Provide the capital Owners and Managers Owners For most of human history, these were the same. • Managers run the firm day to day Managers Brown & Klerman

  30. Separation of “owners” (i.e., stockholders) and “control” (i.e., management) is a cornerstone of the modern corporation Allowing: Poor entrepreneurs to start companies/implement their ideas Owners to invest all of their money and to diversify Owners and Managers Owners Managers Brown & Klerman

  31. And the Issues Are Subtle • Pure fraud is clear, but • How much to spend looking? • Where to look? • Management often has a desired “spin”; owners may want something else • GE’s earnings smoothing • Lehman’s “Repo 105” • Good accounting requires judgment • When judgment is required, which way does the accountant lean? Paid by owners(corporate funds) Owners Auditor To watch (and report on) managers Managers Brown & Klerman

  32. Professionalism Reputation Liability Owner Oversight Their own private (financial) interests are large Continued employment, repeat audit business, consulting business Auditing’s professional ethos emphasizes confidentiality and client (i.e., management) service Not independence (and serving owners) But … Brown & Klerman

  33. Professionalism Reputation Liability Owner Oversight Now suppose management suggests “creative accounting” Perhaps with some “side payment” to the auditor (future auditing, consulting work, etc.) Auditor weighs Side payment vs. Loss of reputation as an “independent auditor” if side payment is discovered Deviation Should be Costly Brown & Klerman

  34. Professionalism Reputation Liability Owner Oversight Yes: Arthur Anderson collapsed after criminal conviction in relation to Enron scandal No: Other accounting scandals did not cause similar collapse (Arthur Anderson was the exception) Some evidence of collusion among firms (rather than competition on “independence”) Too few to fail (currently only four large accounting firms) Does this “Reputation” Work? Brown & Klerman

  35. Professionalism Reputation Liability Owner Oversight Duh! Wasn’t this obvious? This was “best practice” before SOX; but inconsistently applied Why not? And what does that suggest? Management often controls the BoD Management often “suggests” BoD members Who are not independent—relatives, interlocking BoD, other business relationships And board membership is lucrative, so members want to stay in management’s good graces Will SOX Work? Brown & Klerman

  36. Evaluation is Like Auditing Brown & Klerman Brown & Klerman May 9, 2011 36

More Related