The Evolution of
Download
1 / 30

Mindy Crain-Dorough, PhD Southeastern Louisiana University mindy.dorough@selu - PowerPoint PPT Presentation


  • 136 Views
  • Uploaded on
  • Presentation posted in: General

The Evolution of Accountability Reporting: Lessons Learned Over Nine Years By a State Education Agency. Mindy Crain-Dorough, PhD Southeastern Louisiana University mindy.dorough@selu.edu Robyn Madison-Harris, EdD Louisiana Department of Education robyn.madison@la.gov.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

Mindy Crain-Dorough, PhD Southeastern Louisiana University mindy.dorough@selu

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The Evolution of Accountability Reporting:Lessons Learned Over Nine Years By a State Education Agency

Mindy Crain-Dorough, PhD

Southeastern Louisiana University

mindy.dorough@selu.edu

Robyn Madison-Harris, EdD

Louisiana Department of Education

robyn.madison@la.gov


Overview of presentation

  • Background on report development and what Louisiana does

  • History of Accountability Changes

  • Problems/Solutions

  • Recommendations for future directions


Purpose of this study

  • To compile a list of lessons learned as a reference for other states

  • To document the history of accountability reporting in Louisiana

  • To explain the complexities of the report development process

  • To provide recommendations for improvements accountability reporting in Louisiana and other states


Report development – Not an easy thing to do

  • Present complicated information and data in a format that is aesthetically pleasing and understandable

  • Negotiate many conflicting opinions about content

  • Turn complicated and detailed policy into computer programs that calculate scores

  • Coordinating many large data files

  • Extensive checking of all aspects of calculations and report generation has to be conducted.

  • All of these must be done in a very rigid timeframe.


Why do we go to all this trouble?

  • Not just to meet federal and state mandates

  • Parents make decisions about which schools to which they will send their children and even where to live.

  • Parents are also driven to take action at their children’s schools.

  • Teachers, principals, and district staff use the results to make decisions about school improvement efforts and funding priorities


Louisiana’s choices

  • Reports produced at the state level for districts and schools

  • School Report Card for Parents in paper format and sent home with every child

  • Electronic PDFs of reports are available on LDE website

  • Assessment and accountability reporting are separate

  • LDE works with a contractor to produce reports


Back to the beginning

  • Louisiana had report cards as far back as 1996-1997 as part of a legislative mandate

  • Transformed report cards into accountability reports in 1998-1999


School Report Card for Parents


Inside


Back


Changes to reports over the years


Along comes NCLB

  • First reports containing NCLB mandates in 2002-2003

  • Louisiana had to incorporate NCLB into policy and thus the existing reports

  • A dual system created – SPS Component and Subgroup Component

  • Conducted an evaluation of existing reports for what was and was not working

  • Prioritized contents

  • Put highlights in report cards and details in a new report


Incorporation of NCLB


School Report Card for Parents


Inside


Back


Methodology

  • Qualitative – authoethnographic, case study, and phenomenological approaches

  • Data Collection

    • Authoethnographic recount by authors

    • Interviews of current and former accountability staff

    • Document analysis

  • Data Analysis – cross-case examination and content analysis strategies


Problems/Solutions

  • Accountability system changing every year

    • Be organized

    • Planning meetings with key parties

    • Preparation for planning meetings “Regrouping” at the end of each year


Problems/Solutions

  • Errors in the reports

    • Improve the checking process every year using a log of problems found in previous year(s)

    • Include cross-report consistency checking in the process

    • Create thorough processing rules, “data maps,” and record layouts that work together

    • Communication between report developers, data calculation staff, and the contractor on these issues is key

    • Allow districts to review data in aggregated form before inclusion in reports, especially for new indicators

    • Have emergency plans in place should errors occur

    • Use everyone’s time effectively – hire temps if necessary to do more tedious checking


Problems/Solutions

  • Too many conflicting opinions about content

    • Use focus groups to obtain evidence of what works best for consumers

    • Develop a protocol for making decisions regarding conflicting viewpoints


Problems/Solutions

  • Miscommunication within a state agency

    • Is the agency structure conducive to functional report development?

    • Develop an issues database to track issues and their resolutions as well as a process for maintaining the database

    • Develop a thorough timeline with a process for ensuring everyone is on task and a protocol for revising the timeline should unexpected events arise

    • Find a method of meeting regularly to discuss issues and timelines that is most effective for all involved – we are currently having weekly conference calls with the contractor and impromptu meetings in-between


Problems/Solutions

  • Vendor relations

    • Good vendor qualities include flexibility, availability to devote extensive time to the project, advanced technical skills, and good quality assurance practices

    • Balance of working for us and working with us

    • Establish protocol for staff turnovers and management changes

    • Set standards and roles for quality assurance procedures and processes

    • Regular communication and established communication procedures


Problems/Solutions

  • New mandates (internal and external to state)

    • Evaluate the current reports for what works and what doesn’t

    • Prioritize contents of the report (current and anticipated)

    • Take advantage of the opportunity to make enhancements to the reports that are undergoing major revision anyway

    • Look at what other states are doing to get ideas


Problems/Solutions

  • Disasters occur

    • Meet with those most impacted by the events to obtain all information before making decisions

    • Consider all aspects of the accountability system and brainstorm possible impacts


Problems/Solutions

  • Giving one individual too much responsibility

    • Have a team of report developers

    • Rotate lead responsibility for individual reports every two or three years


Problems/Solutions

  • Promoting and supporting report release

    • Use Press Conferences, written press releases, and radio announcements

    • Create a “Help Desk” to answer questions about accountability and other agency programs

    • Provide accountability trainings on policy changes, the importance of data quality, and description of report contents to school and district staff


Problems/Solutions

  • Making reports useful

    • Brainstorm areas in the agency that can be highlighted in the report cards

    • Provide a message to parents in the report card from the superintendent to inspire parents to become more involved in their child’s education


Validity and reliability

  • Validity

    • Confirm valid use of reports through focus groups and training

  • Reliability

    • Thorough and detailed checking process


The current “Parent Report Card”


Recommendations

  • Transition to interactive accountability reporting and tie to school improvement efforts

  • Combine assessment and accountability data

  • Reduce the number of releases and release the Parent Report Cards earlier

  • Re-evaluate the content and purposes of the current reports

  • Have an “Accountability Czar”


ad
  • Login