Current Trends for Quality Assurance in Radiotherapy

1. Current Trends for Quality Assurance in Radiotherapy Tommy Knöös, Sweden

2. Current Trends for Quality Assurance in Radiotherapy Tommy Knöös, Sweden

3. Objectives Can we improve the efficency of our quality assurance? Quality assuranceefforts are increasing with new modalites and technology Canwechangeour approach? Can we learn from other organisations? Dublin

4. Background • The scientific world is full of • Suspension/Tolerance levels • Action levels • Time intervals • For testing a huge amount of parameters • Treatment units • CT scanners • Treatment planning systems • Especially technical and physical ones • Focus of current QM/QA guidelines • measure functional performances of radiotherapy equipment by measurable parameters • set tolerances at desirable but achievable values Dublin

5. What is on a cancer patient’s mind? • Safety • Radiation therapy accident article published in NY Times Saiful Huq - New Paradigms for QM in RT

6. How to avoid these? Lisa Norris Brian Lara Cancer Centre Trinidad and Tobago Dublin

7. Can current QM standards prevent such events and meet the safety standards of new clinical challenges? Main goal of QM: • Patients will receive the prescribed treatment correctly Failure of QM: • Insignificant errors in dose to the target • Major events that can injure or kill patients Saiful Huq - New Paradigms for QM in RT

8. Examples of current QA paradigms: Annual calibration of linac • that is it could have been spent checking things with a higher likelihood of failure • The annual QA takes several days to complete • If everything checks out, the effort was mostly wasted • If some problem was found, how long had it been wrong? • If the problem was significant then a daily check should be in place to prevent it Saiful Huq - New Paradigms for QM in RT

9. Xample: IMRT QC • We measure fluence maps for IMRT QA • This is certainly something that we would want to have correct • However, if the system has been commissioned, and then used correctly, why would this be wrong for the patient? • If you worry about the leaf movement, even if the fluence map indicates correct movement for the first day of treatment, why do you think it might be correct in a week? • Are there problems found this way? Almost never • Do we need QA for leaf movement, maybe daily??? Saiful Huq - New Paradigms for QM in RT

10. Problems with current QA paradigm • These examples suggest that QA as performed currently might not be the optimal use of a physicist’s time • Just performing all the recommended QA might not provide protection against events Saiful Huq - New Paradigms for QM in RT

11. Implementation of industrial tools

12. Quality management tools • Process mapping • Processes and their sub processes • Identify the processes which are more error prone • FMEA (Failure Mode and Error Analysis) • RCA (Root Cause Analysis) • Identify measures for control • SPC (Statistical Process Control) • Measure the quality of the measures/control points • Score actual and potential deviations (e.g. ROSIS) Dublin

13. Quality management tools • Process mapping • Processes and their sub processes • Identify the processes which are more error prone • FMEA (Failure Mode and Error Analysis) • RCA (Root Cause Analysis) • Identify measures for control • SPC (Statistical Process Control) • Measure the quality of the measures/control points • Score actual and potential deviations (e.g. ROSIS) Dublin

14. Process maps • Process map is a visual demonstration of work processes • The map shows how inputs, outputs, and tasks are linked together • Identifies the major steps in the process • Who performs the steps • Provides a visual picture of the process • Identifies and understand process deviations and vulnerabilities, breakdowns, mistakes, delays - detects where improvements may be made • A.k.a flow charting • Numerous formats or approaches exist - As-is and/or To-be ICARO 2009

15. Work plan • 1. Establish process boundaries • 2. Develop the data gathering plan • 3. Interview the process participants • 4. Generate the process map • 5. Analyze and use the map ICARO 2009

16. From Ford et al IJROBP, Vol. 74, No. 3, pp. 852–858, 2009 ICARO 2009

17. Dublin

18. Process mapping S Huq et al. (Red Journal QA issue) Dublin

19. Example of “swim lane” flow chart: Patient to have an diagnostic x-ray • From “Japanese Association of Healthcare Information Systems Industry“ ICARO 2009

20. Quality management tools • Process mapping • Processes and their sub processes • Identify the processes which are more error prone • FMEA (Failure Mode and Error Analysis) • RCA (Root Cause Analysis) • Identify measures for control • SPC (Statistical Process Control) • Measure the quality of the measures/control points • Score actual and potential deviations (e.g. ROSIS) Dublin

21. Use process maps to identify critical steps in the process and/or when investigating incidents and accidents

22. Identifying hazards and error prone sub-processes • FMEA – Failure Mode and Error Analysis • During the FMEA, which is a proactive method, the following questions have to be answered for each step in each sub-process; • a) what could possibly go wrong (potential failure mode), • b) how could that happen (i.e., what are the causes that result in a failure mode) and finally • c) what effects would this failure mode produce (potential effects of failure) • Suitable for creating Fault Trees ICARO 2009

23. FMEA • For each failure mode, estimate: • The probability the failure occurs (“O”) • The severity of it’s effects (“S”) • The probability that the failure will be undetected (“D”) • Form the “Risk Priority Number” RPN: • RPN=O*S*D • O,S,D range [1,10] • Concentrate on the highest RPN ICARO 2009

24. FMEA For a given sub-process: • RPN = O x S x D [ 1 ≤ RPN ≤ 1000 ] AAPM Summer School 2011 Saiful Huq - New Paradigms for QM in RT 24

25. O, S, and D values used in FMEA

26. FMEA

27. Example of a FMEA analysis – Setting up a patient and deliver a treatment at an incorrect position [1]The value was chosen based on that this do not happens to often. Suggestion is that the data is missing once per 2000 cases. In Lund we have about 2600 patients annually and it seems to be an accurate number that this data is missing not more than once a year. [2] If this will be undetected though out the full treatment the ranking of 10 maybe be adequate. [3]This number is incredible hard to set since it depends very much on the clinical environment. If for example it is possible to acquire the patient position (treatment position) by a simple action by the therapist this can be added to the setup and the patient will be treated “correctly” at each fraction. On the other hand since the data is missing one may step back in the process to get the correct data. Based on the latter process a low number is assigned. [4]This probably occurs much more often than the above case. [5] The delectability is definitely lower in this case compared to the first failure situation. [6]Having a lower severity for this failure mode seems appropriate since there are chances to detect this at the following treatment sessions. ICARO 2009

28. Quality management tools • Process mapping • Processes and their sub processes • Identify the processes which are more error prone • FMEA (Failure Mode and Error Analysis) • RCA (Root Cause Analysis) • Identify measures for control • SPC (Statistical Process Control) • Measure the quality of the measures/control points • Score actual and potential deviations (e.g. ROSIS) Dublin

29. From Todd Pawlicki Dublin

30. From Todd Pawlicki Dublin

31. Statistical Process Control - SPC • An effective method of monitoring a process through the use of control charts • Control charts distinguish background variation from events of significance based on statistical techniques • Random variation in the data is de-emphasized by the way the process behaviour limits are constructed • Using process behaviour charts is an interactive procedure that requires process knowledge and interpretation by the user Dublin

32. Example of SPC for IMRT verification Clin tolerance SD UCL LCL SD Clin tolerance Todd Pawlicki et al. Dublin

33. Example of SPC for independent monitor unit check Private communication F Nordström Dublin

34. Four states can be identified • State I • in control and within specifications (IN-IN) • Action required - maintain process control by continuing to monitor the process • State II • in control and out of specifications (IN-OUT) • Action required - change the equipment and/or procedures of the process • State III • out of control and within specifications (OUT-IN) • Action required - identify and remove systematic errors in the process • State IV • out of control and out of specifications (OUT-OUT) • Action required - identify/remove systematic errors and/or change the equipment and/or procedures of the process Dublin

35. From Todd Pawlicki Dublin

36. QC of IMRT Γ - analysis Prostate Initial model Optimized model 3% of dose difference (%ΔD) or 2 mm of distance to agreement (DTA) and an inclusion threshold (%Th) of 10% SunNuclearMapCheck Paraspinal SRT Initial model Optimized model From Létourneau et al 2010 in Med Phys Dublin

37. Simple rules to analyse control charts ESTRO Anniversary

38. Quality management tools • Process mapping • Processes and their sub processes • Identify the processes which are more error prone • FMEA (Failure Mode and Error Analysis) • RCA (Root Cause Analysis) • Identify measures for control • SPC (Statistical Process Control) • Measure the quality of the measures/control points • Score actual and potential deviations (e.g. ROSIS) Dublin

39. Check sheets Probably suitable for registration the efficiency in various defence layers/ in your QA system We have performed 140 000 MU checks with an error frequency of 0.003% • What • A pre-defined format for data collection • Categories are created in advance but my be added • A mark is added for each example • When • To keep track of events/parameters in a process • Number of correct/incorrect outcome of certain processes • Number of errors in certain important parameters ESTRO Anniversary

40. What we need? • What errorsorrurs? Or • What incidents? • The frequency • One have to remember – low probability • Learn from each other!!! Collect, compile, classify, conclude ICARO 2009

41. A radiotherapy example O – Organisation H – Human T – Technical P – Patient Start here Take these later Thanks Petra Reijnders-Thijssen for the data ESTRO Anniversary

42. ROSIS - SAFRON www.rosis.info ICARO 2009

43. Summary Using a process characterization scheme, the control chart approach provides better discrimination of process performance than the standard deviation approach. Control charts also have implications for clinical trials QA where they may be used to appropriately categorize process performance, minimize protocol variations and guide QA process improvements. A LEAN approach Dublin

44. The Problem!!! • RT QA guidance is focused on equipment performance even though most RT events have resulted from human performance failures rather than equipment failure • Lack of adequate guidance for resource allocation • The premise: If it can be checked, check it • That is…..everything • Lack of adequate personnel Dublin

45. The Problem (cont) Pressure on medical physicists to implement new technologies Lack of guidance on how to implement these technologies Delays in publishing QA protocols or guidelines for emerging technologies Dublin

46. Where are the resources and time to do it all?What should a time starved health worker do? What to Do? Saiful Huq - New Paradigms for QM in RT