HTA: The need for trainings

1. HTA: The need for trainings Nine years after its start: Learning from students who are learning from us Marco Marchetti European Project Director of Ulysses Policlinico A. Gemelli Università Cattolica del Sacro Cuore, Roma, Italy On behalf of the Ulysses Consortium members Ankara, Turkey, 2009

2. Acknowledgements • To all faculty members, HTA producers and academic administrators who participated to the development and implementation of Ulysses and to the training of students since 2000 • To the students and alumni

3. Agenda • Background • A need for higher training in HTA • The experience of Ulysses International Master’s Program • Description of the program • Pedagogical approach • key pedagogical lesson • LMICs survey • Scope of the survey • Structure and main results • Conclusion

4. Background • There is an important and growing need for human resources formal trained in HTA. Two drivers justify this need • The increasing emphasis on evidence-based, and cost-effective purchasing and provision of health service, together with • The emergence of International Agencies

5. Background • Despite a clear call for greater input from HTA in policy-making there are, however, very few formal graduate programs in existence. • Faculty members who teach HTA are spread over a large number of universities around the world, and very few universities possess the expertise needed to offer a complete graduate program

6. The experience of Ulysses International Master’s Program Outline • Description of the Ulysses International Master’s Program • Pedagogical approach(es) • Key pedagogical lessons

7. Description of the Ulysses International Master’s Program

8. The Consortium • Europe (Spain and Italy) • University of Barcelona, Spain • Università Cattolica del Sacro Cuore, Italy • Canada (Quebec and Ontario) • University of Montreal • University of Toronto • (University of Ottawa) • HTA agencies in the three countries

9. Target clientele • Physicians, nurses, health scientists, ethicists, lawyers, biomedical engineers, social scientists, administrators and policy makers • The curriculum is relevant for • Evaluators who will produce HTA • Decision makers who will use HTA

10. The Program - Structure • Four intensive Modules (2 weeks) held in four cities • Two courses per Module (all mandatory) • Different “local” options defined by each university • A master’s thesis and/or • An internship (HTA agency or relevant setting) • Additional local courses and an individual research/policy analysis project

11. Understanding what is HTA about Learning how to do HTA Understanding implications of HTA Learning how to use HTA Applying what has been learned The program - Structure

12. Internship objectives Students will: • Learn how to plan, realise and present an HTA report or an HTA-based policy document • Learn how to work within a team • Apply and refine methodological tools • May have the opportunity to work in a culturally and politically different environment

13. Pedagogical approaches

14. Students’ needs? • The needs of candidates working full-time and possessing solid experience are not well met by traditional graduate programs (Mason, 1998) • Dialogue b/w HTA producers and users enables grasping the fundamental principles behind HTA, and using scientific results more systematically (Lavis et al., 2002)

15. Student cohorts vary • Number of students: • 23; 23; 13; 19; • 5th edition: on-going recruitment, aiming at 25-35 students • In a nutshell: • Training: medicine (third), engineering & health sciences (third), social sciences & policy (third) • Employment status: majority working full time in health-related institutions • Age: two thirds are over 30 years old • Gender: around a third are women

16. Up to 18 countries throughout the continents Argentina Austria Belarus Brazil Canada Columbia Denmark Germany Hong Kong Iran Italy Latvia Mexico Romania Spain Switzerland Turkey United Kingdom More international over time…

18. Courses • One-week intensive course • One lead Faculty and collaborators • On-site interactive teaching (35-39 hours) • On-site exercises and team work • Detailed course outline + Readings (prior to the Module) • Final assignments/take-home exams

19. Key pedagogical lessons

20. Diversity as a learning device? • Courses vary • Number, styles and experience of teachers • Proportion of pre-class and in-class exercises, and take homes • Students have different patterns of preferences • Ultimately, our responsibility is to remain sensitive to their preferences, while fostering the acquisition of relevant concepts, tools, skills and reasoning processes

21. Social interactions and learning • Spending a total of two months together! • Group work + meals/weekends • Cohort spirit (HTA culture and more) • International professional contacts • Social events • Life events • Long-term impacts?

22. Alumni • An association of alumni was formed • We invite “local” alumni to welcoming cocktails during Modules • A number of them have now become teachers in Ulysses • Several are working in HTA agencies, Univ. Teaching Hospitals or Ministries • They keep in touch “on their own” (e-mail, trips, Q&A, conferences)

23. Alumni (comments for the web site) • “The Ulysses programme has offered me, an opportunity to embrace a totally new and exciting career in health technology assessment. It allowed me to participate into political decision making when previously, as a clinician, I had little impact on the management of our health care system” (Pierre Dagenais) • “Together, the senior lecturers offered terrifically broad and insightful perspectives about improving healthcare. I found the program inspiring” (Michael Saginur)

24. Alumni (comments for the web site) • “The Ulysses Program has given me the most enlightening and enriching experience both on the professional and personal level. It has changed my career toward HTA and has given me a new family” (Luis Arellano) • “I really think that everyone who works in a field related to the health care system, should have this kind of HTA background. It opened my mind to evaluate the impact of a decision with a broad vision! Giving me a rational and objective method” (Lamberto Widesott)

25. What knowledge and skills…? • Learning through… • Hard work • Pleasure and pain • Dialogue and exchanges • Knowing that… • knowledge is provisional, refutable and incomplete • Doing involves… • Humbleness and cooperation • Responsibility and reflexivity

26. Other pedagogical activities • E-debates (e-mail, voice-mail) • Topics bridging the Modules • Readings • List of questions/issues • Moderated by a faculty member • Internships • Equivalent of 4 months full-time • Supervised by an HTA expert

27. Key features • Flexible format for skilled professionals • To offer flexibility to students working full-time who require a specific learning environment and timeframe • Not always easy for faculty members to adapt their regular course content • Interaction between HTA users and producers • To grasp the broader context in which evidence is being researched and translated into decisions • Stronger emphasis on methods for “producers” and on policy evaluation, management and regulation for “users”

28. Key feature • Academic challenges and international collaboration • Team teaching relevant for the whole class • It requires establishing trust, understanding the academic and cultural characteristics of each, and developing a shared vision • Partnerships with HTA agencies and other HTA suppliers • Relevant to students’ future career and reinforced by HTA agencies’ mission • Further efforts required to establish mutually beneficial collaboration with a larger group of agencies

29. Key features • Global approach to evidence-based practice • The message that HTA producers should be knowledgeable about decision-making and dissemination strategies was woven into most of our courses • It involves trade-offs in allocation of teaching space • Multidisciplinary nature • Required by real-world practice in complex health care systems • It requires time and commitment to develop mutual understanding and productive collaboration

30. Low Middle Income Countries (LMICs) survey

31. Scope of the survey • The scope of the survey is the better understanding of the potential contribution of the Ulysses Masters Program in HTA&M to students from low- and middle-income countries (LMICs). • Its goal is to identify the needs, opportunities and special challenges facing LMIC students so that we may better serve this group.

32. The survey

33. The sampling • The survey was sent on January 25th, 2009, to all Ulysses student and alumni (cohorts 1 to 4) • A majority of people where reached • A reminder was sent on February 4th,2009 • Respondents received greetings back for their participation

34. The sampling • Based on administrative data, we know that those from LMICs who have enrolled in Ulysses to date are from middle-income and transitional countries; none as yet have been from low-income countries (categorised according to the World Bank Country Classifications). • Particularly Ulysses has to date had students from : • Belarus • Brazil • Columbia • Iran • Latvia • Mexico • Romania • Turkey

35. Main results - respondents • 11 surveys were sent back (out of more than 70). Those currently enrolled in Ulysses were more likely to respond: • 6 were from cohort 4, • 2 from cohort 3, • 2 from cohort 2 and • 1 from cohort 1.

36. Main results - respondents • 6 students/alumni identified themselves as coming from LMIC background • 1 Argentina • 1 Belarus • 2 Colombia • 2 Iran • Other respondents from: • 1 Italy • 1 Denmark • 2 Canada (one of the Canadians was born in Sierra Leone) • 1 Australia (this respondent was born and trained in Romania and moved to Australia in 2009).

37. Main results – LMICs student considerations • LIMCs students see HTA as very important in their contexts but feel it faces unique challenges • Lack of infrastructure • Lack of supportive political or cultural climate • Lack of career pathways in HTA

38. Main results – LMICs student major barrier • LMIC student survey showed that: • all students from LMICs felt that there were major financial barriers to study in Ulysses due to a lack of resources and scholarships available in their countries • Several had paid the entire costs from their pockets, including travel and lodging, despite earning in a weaker currency

39. Main results – suggestions • To adapt Ulysses to LMIC students they suggested • Emphasis on contextualization to address the issue of how to capitalize on evidence produced in developed countries and systematically adapt it to developing countries, as a component of each course • Enhancement of internship or thesis experience (sometimes challenging at a distance) • No segregated courses • Distance learning useful but not a replacement for face-to-face experience

40. Main results – suggestions • In addition, they suggested a need for: • Better supervision and mentoring • Career placement services, especially important for LMIC students • Ongoing learning activities of various kinds

41. Conclusion • Managing such a program is far from simple, but extremely rewarding! • Collaborative and learning processes must be carefully cultivated (for both faculty members and students) • Navigating the cultural, professional and disciplinary diversity is key