1 / 11

Evaluation of social projects and programs in Russia: history and current situation

Evaluation of social projects and programs in Russia: history and current situation. Vladimir Balakirev President of the Association of Specialists in Program and Policy Evaluation , program evaluator « Process Consulting » company June 7, 2019. Content.

seitzj
Download Presentation

Evaluation of social projects and programs in Russia: history and current situation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation of socialprojectsandprogramsin Russia: historyandcurrentsituation Vladimir Balakirev President of theAssociation of SpecialistsinProgramandPolicyEvaluation, programevaluator «ProcessConsulting» company June 7, 2019

  2. Content Twoparallelstories of theprogramevaluationdevelopmentin Russia: Programevaluationbythenon-governmentalprograms, evaluation of thegovernmentprograms EvaluationMethodology for governmentprograms Evaluability Pertinentmethodologicalissues Non-governmentalorganizations – leadersinprogramevaluationdevelopment 2

  3. Until 1990: Onethesis “Evaluation of GovernmentProgramsinthe USA” andthemonograph “Models of ExperimentsinSocialPsychologyandAppliedResearch” by D. Campbell; • 1997-1999, keymilestones: firstevaluationprojectsdone for foreignandinternationaldonorsfundingsocialprograms; twoevaluationschools: Siberian Center for civil initiatives (SCCI, Novosibirsk), Center for Development of Non-governmentalOrganizations (Moscow); a conferencedeliveredinNovosibirsk; firstpublicationsinRussian; • 2000-2015: InternationalProgramEvaluationNetwork (IPEN) establishedby 5 foundingcountries; annualconferencesdelivered; approximately 400 participantsfrom Russia; expertcapacitydevelopmentsupport; • 2000-2011: firstRussiancustomers of evaluationservices; NGOs, corporateandprivatecharities; trainings for NGO staff; firsttrainingprograms for universities (The Moscow School of Social and Economic Sciences (MSSES), HSE, NSU); firstjointpublicationonevaluationinRussian 'ProgramEvaluation: MethodologyandPractice'; evaluationschoolsat the Siberian Center for civil initiatives; • 2008-2012: expansion of participationintheevaluation; establishment of the Center 'Crossroads' MGPPU (byPodushkin T.), 'EvolutionandPhilanthropy' (byEvdokimov O.), NABO (byKhananashvili N.), TimchenkoFoundation (by E. Garifullina), CharityFoundation 'DownsideUp'; initiative 'Evaluation of programsinthefield of childwelfare' launched; group of organizationsfromSt. Petersburg (byOdinokov V.) andCherepovets (by A Bogomolov A.) established. History of the evaluation development in Russia 1997-2012 3

  4. 2015: creation of theAssociation of SpecialistsinProgramandPolicyEvaluation (ASPPE), a group of specialistsworkinginthe NGO sector, Center for RegulatoryImpactAssessment (byTsygankov D.); firstnationalconferencedelivered; InternationalYear of Evaluation (by UN EvaluationGroup); annualconferencesdelivered. Evaluation Development SupportProgram 'GarantFoundation' (Arkhangelsk) established. • АSPPE developed 'Evaluationprinciples' andprofessionalstandards 'Specialist insocialprogramsandprojectsevaluation' • 'Design, monitoringandevaluation' journallaunched; newbooksonevaluationinRussianpublished. • Continuedinternationalcooperation; IOCE (InternationalOrganization for CooperationinEvaluation), creation of Eurasianalliance of nationalevaluationassociations, internationalinitiative 'Parlamentarians for evaluation'; jointprojectsondevelopment of learningcoursesandevaluationtools. • Specilizedtraining for NGO staffonevaluation; Charity Foundation “Culture of Childhood”, “Evolution and Philanthropy”, Internet resources, websites “Basics on evaluation”, the program “PION”, online courses, webinars, pages in social networks; • First MA programon design andevaluation. CharityFoundation 'Culture of childhood' atMoscow State University of Psychology and Education (MSUPE). • Inclusion of evaluationinthegrantsystem for NGOs;development of theevaluationsystem; 'SakhalinEnergy' company, TimchenkoFoundation, Presidential Grants Fundestablished; socialprogramsimplementedby 'Metalloinvest' company. History of evaluation development in Russia 2012-2019 4

  5. 1991-2004. Creation of a legalframework for theprogrammaticapproachtopublicadministration. Thenecessaty of an 'evaluation of theeffectiveness of thetargetedprogramsresultsandimpact' wasonlyhighlightedin 2003; • 2004-2010. Programsareone of theresults-basedbudgetingtools. Budgetingprocessreformenvisagestransitionfromcostmanagementtoresultsmanagement. Requirementtothefederaltargetprogramincludeavailability of anindicatorsystemrequiredtomeasureresults. A unifiedmethodology for evaluatinggovernmentprograms of theRussianFederationwasdeveloped. • 2012-2019. Introduction of a programmaticapproachinRussianpublicadministrationandeffectivenessevaluation of thegovernmentprograms. Consolidatedannualreportsonimplementationandeffectivenessevaluation of thegovernmentalprograms of theRussianFederation (Ministry of Economic Development of theRussianFederation) delivered. TheProjectManagement Center launchedasaneducationalandmethodicalplatformbasedatthe RANEPA. Thenewstrategy of thejointventurein Russia: plans for evaluatingfederalprojectsandprograms Evaluation of governmentprograms 5

  6. Resolution of the Government of the Russian Federation of August 2, 2010 No. 588; • Theindicatorapproachtoevaluationisusedexclusively. Efficiencyisthelevel of achievement of values ​​of theestablished indicators. • “Ifthereare no indicators, itisnotpossibletoevaluatetheprogrameffectiveness” (Ministry of Economic Development of theRussianFederation). • “Theplannedindicatorshavebeenachieved, nevertheless ...” Thetheory of changesisnotdisclosed, it’sunclearwhetherthegoalortheintermediateprogramresultsareachieved. • Mostindicatorsareprocessindicators, notresults. • Publicly, thereasons for "lowefficiency" arenotdisclosed. Methodology for evaluatinggovernmentprograms 6

  7. Respondtoquestionsaboutcurrentandemergingissuesduringprogramimplementation;Respondtoquestionsaboutcurrentandemergingissuesduringprogramimplementation; • Provideinformationabouttheprogramanditscontributiontotheimplementation of thepolicy “serviced” bythisprogram; • Trackimplementation of intermediatetasksandachievement of theprogramobjective; • Providetimelyfeedbacktodecisionmakers; • Increasetheprogramimplementationtransparencyandensureaccountability; • Cutcosts, increaseefficiency; • Improveprogramsandpolicies Programevaluationfunctions 7

  8. Supportimportantdecisionsonprogrammodification, itsexpansionortermination; • Determineprogramperformanceandfailurereasons; • Identifyefficientinnovativesolutionsandconditions of theiruse; • Informontheneedtodevelopnewprograms; • Explorerequirementsandconditions for scalingupsuccessfulprograms; • Provideinformationoneffectiveprogrammanagementpractices; • Revisetheprogramrelevanceandeffectiveness Program evaluation functions 8

  9. Howdoweunderstandtheprogramevaluationpurpose, objectivesandopportunities? Whatterminologydoweuse? Howdoweunderstandthemeaning of basicconcepts? • Whatshouldbeevaluated: achievement of theplannedvalues ​​of theindicatorsorimplementation of thetheory of change, thelogic of theprogram? • Whatdistinguishestheprogramevaluationfromtherelatedevaluationdisciplines: policyevaluation, audit, assessment of theservicequality, sociologicalsurveysandtargetaudiencessatisfactionestimates, scientificresearch? • Whatistheevaluationmethodology? Whatmethodsandinwhatcasesshouldbeused? • Common “bestpractices” orstandards for evaluation? Whoisthecustomer of evaluationservices? Whoisresponsible for developingevaluationmethodology, whoconductsevaluation? • Whatarethesimilaritiesanddifferencesbetweentheprogramevaluationandtheprogrammonitoringsystem? Methodologicalquestions 9

  10. commissionandimplementprojectandprogramevaluation; • developandimplementmethods for projectappraisalingrantcompetitions; • delivershort-termcoursesandacademicprograms, advancedtrainingcourses; • createopenmethodologicalassociations, developmanuals, guidelines, methodologicalmaterials, assessmentmethods; • advisetherepresentativesandstaff of stateauthoritiesandorganizationsworkinginthesocialsphere, offeringevaluationsystems for socialprojectsandprograms (FIGs, localandregionalauthorities); • carryouttheirownspecialprojectsontheprogramevaluationimplementation • leaddiscussionsinspecializedpublications; • holdthematicconferences; • createaccessibleonlineresources; • participateininternationalconferencesandprojects; • developprofessionalassociations (ASPPE), developevaluationprinciplesandstandards RussianNGOs, privateandcorporatecharityfoundations, corporatesocialprogramsarethemainengines for programevaluationin Russia today 10

  11. Conference «Competencies, resourcesandmotivation: whatisrequired for successfulevaluation of socialprojects», organizedbytheAssociation of SpecialistsinProgramandPolicyEvaluationonOctober15-17, 2019 in Moscow • https://www.eval.ru/conferences/9 Invitation 11

More Related