The Evolution of
This presentation is the property of its rightful owner.
Sponsored Links
1 / 30

Mindy Crain-Dorough, PhD Southeastern Louisiana University [email protected] PowerPoint PPT Presentation


  • 116 Views
  • Uploaded on
  • Presentation posted in: General

The Evolution of Accountability Reporting: Lessons Learned Over Nine Years By a State Education Agency. Mindy Crain-Dorough, PhD Southeastern Louisiana University [email protected] Robyn Madison-Harris, EdD Louisiana Department of Education [email protected]

Download Presentation

Mindy Crain-Dorough, PhD Southeastern Louisiana University [email protected]

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Mindy crain dorough phd southeastern louisiana university mindy dorough selu

The Evolution of Accountability Reporting:Lessons Learned Over Nine Years By a State Education Agency

Mindy Crain-Dorough, PhD

Southeastern Louisiana University

[email protected]

Robyn Madison-Harris, EdD

Louisiana Department of Education

[email protected]


Overview of presentation

Overview of presentation

  • Background on report development and what Louisiana does

  • History of Accountability Changes

  • Problems/Solutions

  • Recommendations for future directions


Purpose of this study

Purpose of this study

  • To compile a list of lessons learned as a reference for other states

  • To document the history of accountability reporting in Louisiana

  • To explain the complexities of the report development process

  • To provide recommendations for improvements accountability reporting in Louisiana and other states


Report development not an easy thing to do

Report development – Not an easy thing to do

  • Present complicated information and data in a format that is aesthetically pleasing and understandable

  • Negotiate many conflicting opinions about content

  • Turn complicated and detailed policy into computer programs that calculate scores

  • Coordinating many large data files

  • Extensive checking of all aspects of calculations and report generation has to be conducted.

  • All of these must be done in a very rigid timeframe.


Why do we go to all this trouble

Why do we go to all this trouble?

  • Not just to meet federal and state mandates

  • Parents make decisions about which schools to which they will send their children and even where to live.

  • Parents are also driven to take action at their children’s schools.

  • Teachers, principals, and district staff use the results to make decisions about school improvement efforts and funding priorities


Louisiana s choices

Louisiana’s choices

  • Reports produced at the state level for districts and schools

  • School Report Card for Parents in paper format and sent home with every child

  • Electronic PDFs of reports are available on LDE website

  • Assessment and accountability reporting are separate

  • LDE works with a contractor to produce reports


Back to the beginning

Back to the beginning

  • Louisiana had report cards as far back as 1996-1997 as part of a legislative mandate

  • Transformed report cards into accountability reports in 1998-1999


School report card for parents

School Report Card for Parents


Inside

Inside


Mindy crain dorough phd southeastern louisiana university mindy dorough selu

Back


Changes to reports over the years

Changes to reports over the years


Along comes nclb

Along comes NCLB

  • First reports containing NCLB mandates in 2002-2003

  • Louisiana had to incorporate NCLB into policy and thus the existing reports

  • A dual system created – SPS Component and Subgroup Component

  • Conducted an evaluation of existing reports for what was and was not working

  • Prioritized contents

  • Put highlights in report cards and details in a new report


Incorporation of nclb

Incorporation of NCLB


School report card for parents1

School Report Card for Parents


Inside1

Inside


Mindy crain dorough phd southeastern louisiana university mindy dorough selu

Back


Methodology

Methodology

  • Qualitative – authoethnographic, case study, and phenomenological approaches

  • Data Collection

    • Authoethnographic recount by authors

    • Interviews of current and former accountability staff

    • Document analysis

  • Data Analysis – cross-case examination and content analysis strategies


Problems solutions

Problems/Solutions

  • Accountability system changing every year

    • Be organized

    • Planning meetings with key parties

    • Preparation for planning meetings “Regrouping” at the end of each year


Problems solutions1

Problems/Solutions

  • Errors in the reports

    • Improve the checking process every year using a log of problems found in previous year(s)

    • Include cross-report consistency checking in the process

    • Create thorough processing rules, “data maps,” and record layouts that work together

    • Communication between report developers, data calculation staff, and the contractor on these issues is key

    • Allow districts to review data in aggregated form before inclusion in reports, especially for new indicators

    • Have emergency plans in place should errors occur

    • Use everyone’s time effectively – hire temps if necessary to do more tedious checking


Problems solutions2

Problems/Solutions

  • Too many conflicting opinions about content

    • Use focus groups to obtain evidence of what works best for consumers

    • Develop a protocol for making decisions regarding conflicting viewpoints


Problems solutions3

Problems/Solutions

  • Miscommunication within a state agency

    • Is the agency structure conducive to functional report development?

    • Develop an issues database to track issues and their resolutions as well as a process for maintaining the database

    • Develop a thorough timeline with a process for ensuring everyone is on task and a protocol for revising the timeline should unexpected events arise

    • Find a method of meeting regularly to discuss issues and timelines that is most effective for all involved – we are currently having weekly conference calls with the contractor and impromptu meetings in-between


Problems solutions4

Problems/Solutions

  • Vendor relations

    • Good vendor qualities include flexibility, availability to devote extensive time to the project, advanced technical skills, and good quality assurance practices

    • Balance of working for us and working with us

    • Establish protocol for staff turnovers and management changes

    • Set standards and roles for quality assurance procedures and processes

    • Regular communication and established communication procedures


Problems solutions5

Problems/Solutions

  • New mandates (internal and external to state)

    • Evaluate the current reports for what works and what doesn’t

    • Prioritize contents of the report (current and anticipated)

    • Take advantage of the opportunity to make enhancements to the reports that are undergoing major revision anyway

    • Look at what other states are doing to get ideas


Problems solutions6

Problems/Solutions

  • Disasters occur

    • Meet with those most impacted by the events to obtain all information before making decisions

    • Consider all aspects of the accountability system and brainstorm possible impacts


Problems solutions7

Problems/Solutions

  • Giving one individual too much responsibility

    • Have a team of report developers

    • Rotate lead responsibility for individual reports every two or three years


Problems solutions8

Problems/Solutions

  • Promoting and supporting report release

    • Use Press Conferences, written press releases, and radio announcements

    • Create a “Help Desk” to answer questions about accountability and other agency programs

    • Provide accountability trainings on policy changes, the importance of data quality, and description of report contents to school and district staff


Problems solutions9

Problems/Solutions

  • Making reports useful

    • Brainstorm areas in the agency that can be highlighted in the report cards

    • Provide a message to parents in the report card from the superintendent to inspire parents to become more involved in their child’s education


Validity and reliability

Validity and reliability

  • Validity

    • Confirm valid use of reports through focus groups and training

  • Reliability

    • Thorough and detailed checking process


The current parent report card

The current “Parent Report Card”


Recommendations

Recommendations

  • Transition to interactive accountability reporting and tie to school improvement efforts

  • Combine assessment and accountability data

  • Reduce the number of releases and release the Parent Report Cards earlier

  • Re-evaluate the content and purposes of the current reports

  • Have an “Accountability Czar”


  • Login