230 likes | 240 Views
Pilot College Expectations and Implementation Steps. Mu l ti p le Measu r es Assessme n t P r oje c t Pilot College Convening h t tp:// w w w .rp g r oup . o r g /p r ojects/m u l t i pl e - me as u r e s-as s es s ment - p r oject. General Expectations.
E N D
Pilot College Expectations and Implementation Steps MultipleMeasuresAssessmentProjectPilot College Convening http://www.rpgroup.org/projects/multiple-measures-assessment-project
General Expectations • Identification of local point person/persons/committee • Participation/communication with MMAP • Local retrospective analysis (optional) • Webinar: http://bit.ly/MMAPCARTWebinar • DED: http://bit.ly/MMAPRetrospectiveDED • For files, contact: Dan Lamoree: dlamoree@edresults.org
General Expectations - 2 • Implement cohort for Spring 2016 • Upload cohort of applicants (supplemented by CCC Apply) • Receive recommended placements • Help pilot measurement of non-cognitive variables(NCVs) • NCV Scales: http://bit.ly/MMAPNCVScales • NCV Information Sheet: http://bit.ly/MMAPNCVInfo • Contact: Craig Hayward: chayward@rpgroup.org • Help test reliability of self-reported HS data • Unweighted cumulative HS GPA, last math course and grade, last English course and grade • Contact: John Hetts: jhetts@edresults.org • Upload student level data to CalPassPlus • Placements • First semester course enrollments and outcomes
Keyaspectsofimplementation • Initiation • EarlyImplementation Programdevelopment Resourceplanning ProfessionalDevelopmentCommunicationplan Outreach/K-12 – – – – • • Finalizingimplementation • Localresearch&customizationofrulesets • Placementexecutionandcommunication • Evaluationplan • Finalizingtimeline &trackingexecution
Some caveats:Contextmatters • Whileall colleges havecapacity todothis,colleges are atvery differentplacesintheprocess • Notall stepsarenecessaryoreven alwaysdesiredat every college • Processisn’tnecessarilylinear • Implementationmattersbutimplementationmatters
Step1:Initiation • Developdecision-makinggroup(s)appropriatetolocal context Workgroup,subcommittee,campus-widecommittee… • Identify & recruitkey stakeholders/decision-makers • • Disciplinefacultyin EnglishandMath(aswellas ESL andReading) • Assessmentcommittees/subcommittees • Counseling/Matriculation/StudentSupportServices StaffandFaculty – – StudentSuccessStaffandFaculty(Tutoring,SuccessCourses) – AcademicSenaterepresentation – AcademicandExecutiveAdministration – Students –
Step1:Initiation • Developregularmeetingschedule • Establishcommonground • Introductionto&reviewof backgroundresearch • Use ofdiscussion activities/questions • Identificationoflocalbarriers/challengesand developmentofplans/strategiestoaddress • Setstretch goals for desiredchange atcollege • Attend/reviewwebinars • Review/discussstatewide MMAPresearch
Step 1: Initiation: Develop Comprehensive Plans Engagement Plan Faculty involvement Professional development Engaging special programs Assessment Plan Placement process Non Cognitive measures Students w/o transcripts Student Services Plan Counseling Support services (tutoring) Outreach – HS Assessment Enrollment Plan Class offerings Implementation process Communication Plan Informing stakeholders Informational materials Informing feeder HS Research Plan Quarterly analysis Longitudinal analysis Equity impacts
Step2:ProgramDevelopment • Eligibilityguidelines(who/whatconditions) • Multiplemeasuresrulesets/protocol* • Statewidedefaultand/orlocalrefinement • Integrationwith SIS/Assessment • Otherprogramcomponents • Pilotingofnoncognitivevariables* • Collecting self-reported transcript data • Collegeresponsibilities/commitments • Contingencyplanning • studentsupportservices,schedulingbuffers,late-startingclasses,earlyalert,just-in-timeintervention
Step2: ResourcePlanningandDevelopment • Enrollmentmanagement • Cohortbaselineattendance • Newenrollmentexpectationsbasedonprogram(changesin • demandbylevel) • Staffingneedsfor programelementsand/or changesin courselevel • Explorationofexternalsourcesof funding • Calendardevelopment • Backwardsmapping fromsignificantdates • Student-centeredscheduling
Step3:CommunicationPlanning • Internal:students,faculty,staff,departments, committees • Overallsummary • Additional/supportinginformation • Resources • Programelements • Schedules/Timelines • Keycontacts • Ongoinginformation/updates • Progresssummaries • Methods • Webhome/portal • Campuslistserv/mailinglists • Outreachevents/activities • Community awarenesscampaign
Step3:CommunicationPlanning • External/Outreach:K-12 students,parents,counselors,teachers, • community,businesses/funders • Similarcategories/methodsbutalso • Intersegmentalrelationship/trustdevelopment(esp.faculty) • Introduceevidence based placementandwhatit means forstudents/districts • Communicationofneedstodistrict • Work withCal-PASSPLUS toverifyand updateMOUsanddatauploads • https://www.calpassplus.org/calpass/join/members • Identifyprimaryoutreach contactfromcollegetoworkwithCalPASS. • Name,college,role,contactinfotoVictoria Pluim, vpluim@edresults.org
Step3:ProfessionalDevelopmentPlanning • Unlearning&replacingprevious narratives • Disciplinefaculty: • Transferlevelcourses(e.g.,first-yearstudentsin firstyear Englishagain) • Developmental courses(e.g.,beststudentsatappropriate higherlevel) • Trainingsupportfor facultytransitioningbetweenlevels • Studentsupportservicesfacultyandstaff • Howtointerpretandincorporateadditionalevidence ofstudent capacityintoeducationplanning andotherstudentcontact • Institutional researchandITskillsandcapacity
Step4:FinalizingImplementation • Localresearch&customizationofrulesets • Placementexecutionand communication • Evaluationplan • Completingtimelinewithbackwardmapping • Trackingexecution
Step4:Localresearchand rulesetcustomization • Completelocalversion ofstatewide retrospectiveanalysisandrulesetdevelopment(ifdesired) • CollegeandDistrictspecificversionsofthestatewidedatasetare nowavailable forlocal/districtinstitutionalresearchdepartmentincluding • DED • SPSSVariables • R • Contactdlamoree@edresults.org • Provide name,institution,role,andcontactinformation
Step 4:Placementexecution&communication • Methodofexecutionofplacement • CalPASSPlus usingstatewidemodel • CalPASSPlus usinglocal customization • Local data-sharing &localcustomization • Otheralternatives(alsousefulfor backupplanforhelpingstudentswith missingdata) • self-report • transcriptreview • Methodofcommunicationtostudents • – Additional pointofcontactmaybeneeded • Processfor setting/updatingplacementsandclearingprerequisites • Clear documentation/recordofmultiplemeasuresrule sets&studentstowhomapplied
Step 4: PlacementrecommendationsusingCalPASSPlus • College staffcreatesa fileofincomingstudent/applicantsbeingassessedfor • placementinfall2015 • Uploadfiletocalpassplus.org • Receivebacka fileorfileswith: • indicatorsshowingrecommendedplacements(CB21levels)based onstudents’highschooldata • additional recommendations for eligibility for different transfer-level courses in math • variablesindataset(paralleltoretrospectivedata)toallowforlocalcustomization(ifneeded)* • Translatetolocalcourses/placements* • Import/integrationinto: • assessmentdatabase/SIS* • studentsupportservicesprotocols • *Indicatesplaceswherethere_may_beopportunitiesforcollegestoworkcloselywithCalPASSPlus staffonlocal customizationwithinreasonablelimits
(Ifyouhaven’talready,pleasecompletethepilotcollegestatussurvey:http://goo.gl/31UJnw)(Ifyouhaven’talready,pleasecompletethepilotcollegestatussurvey:http://goo.gl/31UJnw)
(Ifyouhaven’talready,pleasecompletethepilotcollegestatussurvey:http://goo.gl/31UJnw)(Ifyouhaven’talready,pleasecompletethepilotcollegestatussurvey:http://goo.gl/31UJnw)
Step4:CompletingEvaluationPlan • Evaluate implementation • Every semester and annually • Explore disproportionate impacts • Changes in success rates at all levels • Tracks students over multiple terms when possible • Data Transparency • Share the data with the workgroup and college • Make adjustments based on findings
Step4:CompletingTimeline and Tracking Execution • Develop/reviewinstitutionaltimelineworking backwardfrom keyinstitutionaldates&deadlines • Targetcompletiondates • Must meetcompletiondates • Buildinflexibility,abilitytorespondontheground,contingency planning • Trackexecutionandwatchforopportunities • Whatworked • Celebratemilestones/successes! • Whatdidn’t • Whatcouldhave worked/willworknextyear • Documentchangestoplan