Mindy Crain-Dorough, PhD Southeastern Louisiana University mindy.dorough@selu - PowerPoint PPT Presentation

The Evolution of
1 / 30

  • Uploaded on
  • Presentation posted in: General

The Evolution of Accountability Reporting: Lessons Learned Over Nine Years By a State Education Agency. Mindy Crain-Dorough, PhD Southeastern Louisiana University mindy.dorough@selu.edu Robyn Madison-Harris, EdD Louisiana Department of Education robyn.madison@la.gov.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Mindy Crain-Dorough, PhD Southeastern Louisiana University mindy.dorough@selu

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Mindy crain dorough phd southeastern louisiana university mindy dorough selu

The Evolution of Accountability Reporting:Lessons Learned Over Nine Years By a State Education Agency

Mindy Crain-Dorough, PhD

Southeastern Louisiana University


Robyn Madison-Harris, EdD

Louisiana Department of Education


Overview of presentation

Overview of presentation

  • Background on report development and what Louisiana does

  • History of Accountability Changes

  • Problems/Solutions

  • Recommendations for future directions

Purpose of this study

Purpose of this study

  • To compile a list of lessons learned as a reference for other states

  • To document the history of accountability reporting in Louisiana

  • To explain the complexities of the report development process

  • To provide recommendations for improvements accountability reporting in Louisiana and other states

Report development not an easy thing to do

Report development – Not an easy thing to do

  • Present complicated information and data in a format that is aesthetically pleasing and understandable

  • Negotiate many conflicting opinions about content

  • Turn complicated and detailed policy into computer programs that calculate scores

  • Coordinating many large data files

  • Extensive checking of all aspects of calculations and report generation has to be conducted.

  • All of these must be done in a very rigid timeframe.

Why do we go to all this trouble

Why do we go to all this trouble?

  • Not just to meet federal and state mandates

  • Parents make decisions about which schools to which they will send their children and even where to live.

  • Parents are also driven to take action at their children’s schools.

  • Teachers, principals, and district staff use the results to make decisions about school improvement efforts and funding priorities

Louisiana s choices

Louisiana’s choices

  • Reports produced at the state level for districts and schools

  • School Report Card for Parents in paper format and sent home with every child

  • Electronic PDFs of reports are available on LDE website

  • Assessment and accountability reporting are separate

  • LDE works with a contractor to produce reports

Back to the beginning

Back to the beginning

  • Louisiana had report cards as far back as 1996-1997 as part of a legislative mandate

  • Transformed report cards into accountability reports in 1998-1999

School report card for parents

School Report Card for Parents



Mindy crain dorough phd southeastern louisiana university mindy dorough selu


Changes to reports over the years

Changes to reports over the years

Along comes nclb

Along comes NCLB

  • First reports containing NCLB mandates in 2002-2003

  • Louisiana had to incorporate NCLB into policy and thus the existing reports

  • A dual system created – SPS Component and Subgroup Component

  • Conducted an evaluation of existing reports for what was and was not working

  • Prioritized contents

  • Put highlights in report cards and details in a new report

Incorporation of nclb

Incorporation of NCLB

School report card for parents1

School Report Card for Parents



Mindy crain dorough phd southeastern louisiana university mindy dorough selu




  • Qualitative – authoethnographic, case study, and phenomenological approaches

  • Data Collection

    • Authoethnographic recount by authors

    • Interviews of current and former accountability staff

    • Document analysis

  • Data Analysis – cross-case examination and content analysis strategies

Problems solutions


  • Accountability system changing every year

    • Be organized

    • Planning meetings with key parties

    • Preparation for planning meetings “Regrouping” at the end of each year

Problems solutions1


  • Errors in the reports

    • Improve the checking process every year using a log of problems found in previous year(s)

    • Include cross-report consistency checking in the process

    • Create thorough processing rules, “data maps,” and record layouts that work together

    • Communication between report developers, data calculation staff, and the contractor on these issues is key

    • Allow districts to review data in aggregated form before inclusion in reports, especially for new indicators

    • Have emergency plans in place should errors occur

    • Use everyone’s time effectively – hire temps if necessary to do more tedious checking

Problems solutions2


  • Too many conflicting opinions about content

    • Use focus groups to obtain evidence of what works best for consumers

    • Develop a protocol for making decisions regarding conflicting viewpoints

Problems solutions3


  • Miscommunication within a state agency

    • Is the agency structure conducive to functional report development?

    • Develop an issues database to track issues and their resolutions as well as a process for maintaining the database

    • Develop a thorough timeline with a process for ensuring everyone is on task and a protocol for revising the timeline should unexpected events arise

    • Find a method of meeting regularly to discuss issues and timelines that is most effective for all involved – we are currently having weekly conference calls with the contractor and impromptu meetings in-between

Problems solutions4


  • Vendor relations

    • Good vendor qualities include flexibility, availability to devote extensive time to the project, advanced technical skills, and good quality assurance practices

    • Balance of working for us and working with us

    • Establish protocol for staff turnovers and management changes

    • Set standards and roles for quality assurance procedures and processes

    • Regular communication and established communication procedures

Problems solutions5


  • New mandates (internal and external to state)

    • Evaluate the current reports for what works and what doesn’t

    • Prioritize contents of the report (current and anticipated)

    • Take advantage of the opportunity to make enhancements to the reports that are undergoing major revision anyway

    • Look at what other states are doing to get ideas

Problems solutions6


  • Disasters occur

    • Meet with those most impacted by the events to obtain all information before making decisions

    • Consider all aspects of the accountability system and brainstorm possible impacts

Problems solutions7


  • Giving one individual too much responsibility

    • Have a team of report developers

    • Rotate lead responsibility for individual reports every two or three years

Problems solutions8


  • Promoting and supporting report release

    • Use Press Conferences, written press releases, and radio announcements

    • Create a “Help Desk” to answer questions about accountability and other agency programs

    • Provide accountability trainings on policy changes, the importance of data quality, and description of report contents to school and district staff

Problems solutions9


  • Making reports useful

    • Brainstorm areas in the agency that can be highlighted in the report cards

    • Provide a message to parents in the report card from the superintendent to inspire parents to become more involved in their child’s education

Validity and reliability

Validity and reliability

  • Validity

    • Confirm valid use of reports through focus groups and training

  • Reliability

    • Thorough and detailed checking process

The current parent report card

The current “Parent Report Card”



  • Transition to interactive accountability reporting and tie to school improvement efforts

  • Combine assessment and accountability data

  • Reduce the number of releases and release the Parent Report Cards earlier

  • Re-evaluate the content and purposes of the current reports

  • Have an “Accountability Czar”

  • Login