Mark D. Colley 555 Twelfth Street, NW. Washington, D.C. 20004 202-942-5720

1. Department of EducationRace to the Top Assessment ProgramJanuary 14, 2009 Public MeetingProcurement Issues Mark D. Colley 555 Twelfth Street, NW. Washington, D.C. 20004 202-942-5720 mark.colley@aporter.com

2. How will state procurement rules impact consortium management and design? • States will need to have authority to act via a consortium • Cooperative purchasing • Common, but not uniform or universal • Waivers and approvals may be necessary • Potentially greater impact on the conduct and design of the vendor selection than on the consortium management and design • Test design • Vendor selection criteria and procedure • Specific state issues • Procurement requirements • Testing requirements/preferences

3. How will state procurement rules impact consortium management and design? • Most issues relate to surrender of autonomy and control • Source selection • Consortium management--leadership, meetings, administrative • Technical direction to vendors • Contract changes and modifications • Important to give early consideration to consortium relationship and governance • State procurement and legal officials not accustomed to such arrangements • Most cooperative purchasing is for generic supplies or standardized services • Sharing of programmatic decision-making take this out of the ordinary • Confronting and overcoming these issues early can avoid later delays and problems

4. How will state procurement rules impact consortium management and design? • Consider making this an element for consideration in consortium selection for funds • Has the consortia assessed the applicable state procurement rules of its members, and addressed/accommodated the differences? • Has the consortia developed and agreed to a sound management construct for making decisions and administering the consortia?

5. How best to facilitate state access to vendor information and ideas? • Many options for the states • RFIs • Draft RFP/specs • Industry meetings • Guidance from OCI law developments at the Federal level • Key is openness and transparency • Firewalls and mitigation • Public notice

6. How best to facilitate state access to vendor information and ideas? • Distinguish between technical advice contractors and test development contractors • Expect vendors to have some reluctance to share proprietary details in advance of contract award, particularly without assurances of protection from disclosure

7. How to structure the funding application to enable competitive state procurements? • Competitive procurement should be assumed • Consistent with Federal principles and requirements in nearly all states • Exceptions would not apply to this sort of a procurement • Do not worry too much over the particular methodology or procedure adopted • States have established procurement systems and practices; legal requirements for fairness, transparency, etc. • If multiple participating states find an approach acceptable, that should be a sufficient check • ABA Model Procurement Code can serve as a guide

8. How to structure the funding application to enable competitive state procurements? • Consider requesting in the application a description of the procurement process the consortium has agreed to follow, and make it part of the selection criteria • This will force the member states to confront and resolve the differences in their procurement systems in advance • Early involvement by responsible procurement and legal authorities within each state will be essential • The differences can be overcome and accommodated, but considerable negotiation and review may be required • Confirm approval by procurement and legal authorities

9. How to design a process flexible enough to accommodate challenges the States might encounter? • Require that the consortium address concerns rather than meet strict requirements • Limit grant selection criteria to how states recognize and propose to address broad goals • For example, the judgment regarding “appropriate accommodations” may vary state-to-state • Likewise with “appropriate use of technology” • states vary with regard to what’s “appropriate” (i.e. calculators) • disparity as to technology available or funded • variations in desired use of test results can affect scoring timing

10. How to design a process flexible enough to accommodate challenges the States might encounter? • Many of the “required characteristics” for the “Design of Assessment Systems” appear to relate to what the vendors will do, rather than what the consortium will do— • How much of this will “flow down” into the criteria the consortia apply in vendor selection • Who decides what’s most important (i.e. weighting the criteria)? • How “customizable” will the tests be; options per state? • Need to address the tension over who decides how the funds will be spent, and against what criteria.

11. How to design a process flexible enough to accommodate challenges the States might encounter? • Fulfilling requirements may depend on Intellectual Property ownership • Government typically owns the IP that it pays to develop • Test developers may use proprietary software/systems that they have developed themselves • Government will not pay for and own that IP • But, the test items developed may not work on other systems or with different software • Alternatives • Developed tests may be “Vendor specific” • Recognize or work around the limitations  licensing rights • Is there some sort of “Open Architecture” in this industry to specify in the procurement specifications?