Evaluation in the education systems. Political resorts and effects on the professions of education

1. RégisMalet, Research Group in Comparative education Laboratoire “Cultures Education & Sociétés” – LACES EA4140 regis.malet@iufm.u-bordeaux4.fr educationcomparee@hotmail.fr 4th International Conference Education, Economy & Society Paris, Hôtel Concorde La Fayette – July 24th 2012 Evaluation in the education systems. Political resorts and effects on the professions of education

2. Collective book in preparation (Dupriez & Malet 2013): meaning, place and crossed effects on teachers’ and leaders’ work and professionnalism of new evaluation and assessment procedures within the education systems+ BQR project 2012-15 new frames of teachers professionnality – comparative study groupI. Historicalresorts • The use of evaluation/assessment –beyond the pupils’ assessment– is not new within the education systems : principle of evaluation historically embedded in school national systems. • The institution of the right to education and the ‘systematization’ of education during XIXth century, the formal unification of the modern education systems.. • First movement of globalization and also a reciprocal nations’ interest into the organization of their respective education systems (Malet 2010) • Forms of evaluation very much linked to national administrative and political references, which guide the public action in education: internal or external, processive or sommative, qualitative or quantitative…(Dupriez & Mons 2011)

3. I. Historical resorts - 2 • Since Bidwell works (1965), the sociology of organizations has identified a double logic -bureaucratic et professional- which prevailed in the regulation of the education systems all along 20th century. • Bureaucratic logic of evaluation refers to standardization processes which organize school systems: declination of the systems through school levels and years and school subjects. The organization is supposed to guarantee, formally, thje homogeneity of the education service. • These specificities draw what Maroy (2006) calls bureaucratic-professional regulation. Resorts of such a regulation model are: • The intervention through rules and norms ; • The mobilization of a (semi-)professional logic based upon a recognition of the expertise and knowledge of which teachers are depository. The more and more extensive and systematic mobilization of evaluation is now a key-instrument of public action and policy. This move participates into a slow erosion of a « bureaucratic-professional » model of regulation of schools systems and professions.

4. II. Political resorts - 1 From an evaluation of regulation towards an avaluation of intervention • Priority responsability of the public powers: an evaluation which has a strong moral and political meaning, refering to the obligation for public administrations to be accountable for their organisation to the democratic instances from which they emerge (Maroy 2011, Malet, à paraître). But… • New forms and new tools of evaluation, inherited of the market economy appear: from a mandate logic to a contract logic (Dupriez & Mons 2011; Dupriez & Malet, forthcoming 2013) • From a classical function aiming to guarantee the organisation of the education system (for instance the selection and the recruitment of the school system, their career and the adequation to professionnal norms and prescriptions…) towards a more systematic measurement of school performances, which affects both schools and teachers. • Benchmarking and indicators of performances used in various ways from one country to another are accompanied with procedures wich tend either to support schools professionnal communities or to affect them with injonctions for change and performances, sometimes with possible pressure for professionnals and school closure (Dulude, à paraître; Malet 2012; Moreau, 2012)

5. II. Political resorts - 2 From an evaluation of regulation towards an avaluation of intervention • Classical function of a regulatory evaluation of the education offer though internal information is now complexified by a non less crucial objective of external information. • The external use of evaluation –for instance through international comparison of performances and results – expresses a progressive inflexion which tends to reinforce the capacity of evaluation to sustain school reforms. • Observable and international tendancies, with various administrative references and intensity : • From a principle of classicaprofessional accountability model to market logic procedures (informing the consumers) and/or managerial logic, contract and performances driven (Ranson 2003) • d’uneévaluation des individus, celle-ci se déplaceversuneévaluation des entitésd’enseignementd’une part. • From an evaluation centered on norms and prescription adequation, to results-focused evaluation. Evaluation is about to lose a status of ponctual disposition (in direction of individuals, schools or policies) and becomes more and more a recurrent process aiming at changing professionnal practices in schools and educational systems.

6. Political resorts (3) Various forms of accountability though.. • Distinction regarding the principles (to be accountable to publics authorities, users, consumers…), regarding the levels of intervention which are chosen (teacher, group of teachers, schools, school district…) and regarding motivations or sanctions mobilized in the process (Malet & Dupriez, forthcoming 2013) • The meaning of the words can vary a lot from one country to another, or from one cultural-linguistic area to another (Malet 2012, on the idea of « professionnalisation ») • This generates a new form of deconnection or « découplage » between an international rhetoric (evaluation culture, performance-based gouvernance, professionnalisation…) and effective actions very much indexed to local cultural, linguistic and political contexts and contingencies.

7. III. Political and theoretical resorts • This diversity is very much embedded within educational systems history, and this makes the building-up of a theory of gouvernance in education through evaluation uneasy. • Pons, 2010 & Buisson-Fenet & Pons 2011, in the French case, pointed out the political limits of the regulation through evaluation: underlining the ambiguity around the political demand for evaluation, considering the uncertainties surrounding the real impact of evaluation on practices, and the absence of actions of ‘remediation’ consecutive to the evaluation process (ie standardized evaluation of pupils achievement) : this reinforces a singular mode of control of schools and teachers with the promotion of a new managerial model with no real possibility to orientate practices in schools.  • Conséquence : France is in an “entre-deux” regarding the regulation of the educational system : no longer a bureaucratic and corporatist regulation, but not yet a « post-bureaucratic » regulation (Maroy 2006). • French dilemma can be described tis way: how can one guarantee the national coherence of education policies in an educational system which tends to be more and more decentralized and « reterritorialized »? How can one go beyond the founding fiction of a state-educator within a uniform and culturally homogeneous space, and undifferenciated pedagogically school » (Mons 2007, 8).

8. III. Political and theoretical resorts (2) • Deficit of a theory of the evaluation of schools; yet one can distinguish: 2 models of intelligibility of regulation through accountability: • A first model, classical and dominant, refers to the developement in education of New public management : Based on a (triple) critique of the role of various domains of public action (cf. Affichard, 1997) and mobilize such arguments as : • decisions from central authorities through standardized caterories/criterias for the whole territory, is unadequate regarding local realities of schooling  ; • The centralized structuring is inefficient, tends to bureaucratic errements is financially non perfomant. In the perspective of NPM, the main objective is to make structures evolve in the direction of ‘deconcentration’, decentralization, and to provide local actors of schooling with larger responsibilities and initiatives.

9. III. Political and theoretical resorts (3) One have to distinguish: - market-based accountability (logique of the « informed consumers », through « league tables », « school report cards »…) (Angleterre, Pays-Bas, Etats-Unis, Chili…) versus • government-based accountability. (pressures on schools by public authorities through regulation agents as inspectors… (Harris et Herrington 2006) Thezse two logics can be combined in some countries, as England or USA (Dupriez & Mons 2011, Malet 2009 & 2012, Moreau 2012, Lessard, 2011). Critical perspective on the accountability model : From a logic of professional qualification to a logic of value or quality attribution, potentially individuel, regarding performances and results. New model of regulation expressing an extension of regulation processes and generating a potentially ‘hyper-bureaucratization of educational systems’, conducting to a close-control of education action in schools and classrooms. (Ball, 2003 & 2005, Lima 2010, Malet 2008…)

10. III. Political and theoretical resorts (4) 2 models of intelligibility of regulation through accountability: • A second model sees new modes of evaluation as an extension of procedures of work control. Boussard (2009) notes XIXè century origins the current changes, with the development of the OrganisationScientifique du Travail (OST) : • Relationships in the workplace, with the growth of industrial organizations, tend to go from a confidence and loyalty ones (community ideal), to more contractual ones, through incitation, prescription and control. • One category of worker aspires to go beyond these procedures of control: professionnals, which tend to escape from any forms of external prescription and control through inner regulation and control of the professional group: knowlegde, ethics, deontology etc. Professionnalization process. • Yet, the development of management ingeniering expands more and more to professionnal groups, in a context of the weakening of the « profession al myth » (cf. Lessard, Bourdoncle, Maroy, Malet etc. on these stakes)

11. IV. What effects on work in schools ? Can Accountability , in its application in schooling and teaching, be associated to the improvement of pupils achievement? Teachers achievement ? Reduction of unequalities in schooling? - Studies shows the positive effects of accountability on pupils achievement (Fredericksen, 1994), or on minorities youth in schools (Carnoy et Loeb, 2002), - Other studies shows on the contrary the effetcs of politics of accountability in US, with such unexpected effetcs as labelling (Amrein et Berliner, 2002). - For Hanushek et Raymond (2004), accountability does not help to reduce the differences in achievement in pupils from various social backgrounds (see also Lee et Wong 2004 & Nichols, Glass et Berliner, 2006) - Lee (2008) produced a synthesis on the subject, based on 14 longitudinal studies, pointing out key-results : accountability politics have some effects on pupils achievement, but theses performances are short-term ones and accountability on teachers and schools would be poor in reducing social and ethnic unequalites in school achievement. The summative vsprocessual dimension of the procedures and tools of such politics of regulaton are very much linked, according to Lee, to conclude on this (standards, teaching to the test, professionnal conformation etc.).

12. IV. What effects on work in schools ? (2) Echos of research on the effects of accountability regulation on teachers work, professionnality and identity (Dupriez & Malet, forthcoming 2013,…) - Mc Ness, on the English context, shows that accountability politics tend to increase pressure on teachers and learners, linked to a very performative conception of educationg and schooling. In this context, moral engagement of teachers is troubled and affected. Cattonar et Maroy , André, Moreau ( forthcoming) showed that in France, Belgique, Suisse, there was some unexpected effects on teachers identity, becoming fragmented because of new tensions between prescripted work, experienced work and individual conceptions/ideals of one as a teacher. Its seems that the recognition of teachers professionnalism ethics and expertise is often denied (or lived as such) by teachers, and this feeling increases as the performance rhetoric and tools develop.

13. IV. What effects on work in schools ? (3) Affectation of work, moral and feeling of recognitionare observed a lot in contextes where the politics of accountability grow : USA, UK (and mostly England), Sweeden, Australia : the« teaching to the test » stake (Booher-Jennings (2005 ,Jones et Egley, 2005 ; Belair, 2005) goes with a narrowed conceptiuon of school curricula and teaching professionnalism. Studies reveal (US , France, Canada) that in some contexts cognitive objectives and more conformative attempts than creative formation develop, regardless to the development of socialization, creativity, citizenship and critical argumentation (Gordon et Reese, 2000 ; Mons, 2009).  On the contrary, Draelants, Dupriez et Maroy(2003) underline, in the Belgium context, that common objectives of education based on standardized tools of evaluation can be approached as an indication of a reinforcement of a civic preoccupation where the equivalence of the conditions of schooling prevails to more qualitative but then unequal conditions, depending on schools and teams.

14. IV. What effects on work in schools ? (4) André (forthcoming), shows (Canton de Vaud in Switzerland), that the process of external evaluation and accountability is a source of miscognition of the reaaZAAl work of teacers and teaching; it nourrishes defensive behaviour from the teachers. Verdière (idem) explores, for French secondary teachers, the evolution of the normative frames of teachers work, in the context of new organization of work in schools, with a tendancy to make work more visible or ‘transparent’: it generates a feeling of depreciation of the profession, fragmentation of work, impression of fragilisation of the status, deterioration of social position and professionnal freedom. Moreau (idem) observes , in UK (England) such a move towards a strong feeling for teachers of depreciation of their work and responsibility, revealing a tension between professional identities and normative injonctions: deficit of recognition and non protective status (cf Ball, Lawn, Ozga…) See also Dulude on the district of Chicago (US) (effects of the No child left behind reform) ()Dulude, Forthcoming in Dupriez & Malet 2013)

15. Perspectives: accountability and after? Limits of the accountability politics inspired by NPM Research points out the limits of a technocratic evaluation, based on measurement and productivity and education/schooling performance (Lee 2008, par ex.) Numerous studies showed the mediation process operating between prescription and education and teachers’ work in various contexts of schools. This can be read on two main dimensions : Pertubating confrontation of the practitionners with various levels of norms and prescriptions affecting their professionnal practices : Verdière, Moreau, Dulude, André, Barrère…) The emergence of « local curricula » (Baluteau) redefinition processes which are very much inscribed in and linked to professional and educative environments and practitionners, which tends to develop professionnal micro-decisions, both individual and collective. Research reveals a quite worrying phenomena: as soon as there is some tensions between prescriptions (missions, values, curricula…) and indicators of performance and results, the decisions (arbritrage) operate often in favour of what is based on measurement. Baluteau, Verdière, Dulude show very well how professionnals, under pressure, willlpriviledge what can be visible and eventually measured, on the detriment of other dimensions at the core of what they consider at the core of educating and teaching, but which are less measurable: citizenship and moral education, socialization, education to culture & arts…).

16. Perspectives: accountability and after? • Openings and possible routes • An important task of study, both on the side of education policies and of research in education, to make new evaluation models or paradigm not additive to more ancient ones, which would (and sometimes does) result in a multiplication of norms and prescriptions, not always so convergent, often « discordant » (Ex. for France, cf. Pons & Buisson-Fenet 2011). • A field of study to develop which integrates the lessons of experiences of broad participative evaluation/audit (theorized by Monnier for example) of schools, leaders and teachers (Demailly 1998) – such studies can ponderate the systematic mobilization of and faith in quantitative indicators, often revealing a tendancy to simplification, and which generate some defiance from the actors of schools and a form of miscognition of the complexity of education work in schools (cf. André, Maroy, Dumay & Dupriez, ouMalet). • Associate the profession/s of education and teaching in the evaluating process is what could be expected of new education policies