Background: ADP Assessment Consortium Consortium Organization and Governance Potential Application Requirements: Demon

2. Background: ADP Assessment Consortium • Consortium Organization and Governance • Potential Application Requirements: Demonstrating Capacity for Success • Increasing Odds of Success

3. American Diploma Project Assessment Consortium • Initiated in 2005 by 9 states in Achieve-led ADP Network • consortium has grown to 15 states • State share broader college- and career-ready policy agenda • Started with Algebra II EOC exam, expanded to include Algebra I • Agreed-upon policy purposes • Improve curriculum and instruction and ensure consistency of content and rigor • Enable states to compare results across and within states • Indicate readiness for college credit bearing mathematics • Agreement to college-ready cut score set by Achieve

4. ADP Assessment Consortium In summer 2006, nine states issued RFP for the development of an Algebra II EOC Exam: • Arkansas, Indiana, Kentucky, Maryland, Massachusetts, New Jersey, Ohio, Pennsylvania, Rhode Island • Ohio acted as “lead state” in unprecedented multi-state procurement arrangement • Pearson awarded contract in 2007 with addition of Algebra I in 2008 • Since the consortium began, six additional states have joined: Arizona, Florida, Hawaii, Minnesota, North Carolina, and Washington. More states are expressing interest in the exam and joining the consortium…

5. Consortium Organization and Governance • ADP Consortium Member States • Ohio – lead state • Coordination and Direction Team (CDT) • Achieve • Center for Assessment – technical advisor • Pearson – test developer selected competitively

6. New State Members Membership is extended to any ADP Network state wanting to join New member states must: Agree to terms of Partnership Agreement and MOU Agree to terms stated in the contract (but may add language) Participate in all CDT meetings, item development and review meetings Make firm, long range commitments Contractual Personnel Financial 6

7. Role of the Lead State: Ohio Appoint a procurement officer Conduct procurement Provide guidance to the CDT Administer the contract Manage contract amendments Incorporate new states into the contract 7

8. The Coordination and Direction Team Includes assessment directors or other high-ranking policy-making officials from each member state in the consortium Oversees production and implementation of the Algebra I & II program Ensures that legal and policy needs of each state are addressed during team deliberations and decision making 8

9. The CDT and Decision Making The CDT states have: Multiple representatives One vote Process of consensus, collaboration, and compromise: Relating to the policies of each state Repeated as necessary Less discussion time needed as process matures For contractual matters, decisions must be unanimous 9

10. Achieve’s Role • Convener and consensus builder • Brought chiefs, state content and assessment directors, governors & postsecondary leaders to the table • Consortium Management • Communications with and among states • Consensus building • Project manager/oversee development contract • Assembled additional expertise (technical, legal) • Content expertise • ADP Benchmark standards • Model course descriptions

11. Achieve’s Role • Promote Postsecondary participation • Math faculty at 2- and 4-year institutions • System leadership • With key national higher education organizations • Define college readiness • Conceptually • Set achievement levels and cut scores

12. Key Lessons for States: Participating in a Consortium Takes Time and Communication • Active involvement of chief state school officer in start up phase is essential, so that state policy objectives and improvement strategy drive assessment design • Essential but difficult to sustain their engagement over time, as project moves to development and implementation phases • Governors and legislatures play critical roles in state testing policy; they must be engaged and informed along the way • The transaction costs for SEA staff are high compared with single-state assessment development

13. Key Lessons for States: It’s Not Just About the Test • Developing common summative assessments is difficult • Summative assessments not enough – coherent approach that includes formative and interim assessments, curriculum and instructional materials, professional development is necessary to support instructional improvement • That requires more ambitious effort, and broader and deeper consensus among participating states • Common standards provides critical foundation for such an approach

14. Key Lessons for States: Shared Vision is Essential • States must have common standards and must hold firm to a shared vision: • of policy objectives • of uses and purposes of assessments • of assessment system design • Scope of the vision will determine other practices must also be held in common. The more comprehensive and coherent the vision, the more issues states must address in common • Within a consortium, willingness to compromise on behalf of larger vision and shared goals is essential • To develop summative tests alone, states must still be willing to compromise on scores of details (e.g. testing windows, testing modes, test security, test guidelines, calculator policy for mathematics, accommodations, etc.) • Additional compromise will be needed in more ambitious approaches

15. Potential Application Requirements: Consortia Should be Asked to Demonstrate: High degree of consensus around: • Policy Objectives • Instructional Improvement • Vision of assessment system • Leadership Capacity in each state • Support of Governor • Active Leadership by and commitment of Chief State School Officer (not just state testing directors) and Postsecondary leaders (for college-readiness) • Engagement of State Board of Education and Legislature as appropriate in each state

16. Potential Application Requirements: Consortia Should be Asked to Demonstrate: • Technical capacity across the consortium, matched to assessment system design • Consortium Governance and Decision-Making Structure • Consortium Procurement Strategy • Roles and Responsibilities of Consortium members and partners, including supporting organizations

17. Increasing the Odds of Success Over Time Applications should include a sustainability plan that addresses: • Anticipated (and unanticipated) leadership turnover, especially for governors and chiefs • Strategies for building and sustaining support among key stakeholders • Ongoing communications with teachers, students and parents about purposes and uses of assessments • Annual assessment costs, especially if: • Projected annual costs expected to exceed current costs • Continuous development (e.g., new items) is part of design, beyond life of federal grant • Ongoing commitment of partner organizations through grant period