Clinical Registries John McNeil Chris Reid Peter Cameron Belinda Gabbe Louise Phillips Sue Evans Dept of Epidemio

1. Clinical Registries John McNeil Chris ReidPeter CameronBelinda GabbeLouise PhillipsSue Evans Dept of Epidemiology& Preventive Medicine (DEPM) 

2. The extent of the information provided by disease & procedure registries is more complete if they are population based�i.e. if they collect data from all cases occurring in a defined population. Population based registers typically collect many more cases and therefore have the additional statistical power required to analyse sub-populations. � � With population based registers it is possible to gain information about: Epidemiology: ie the incidence of a disease, trends in the incidence, absolute numbers and costs of procedures, absolute numbers of complications �-Access to care: comparisons may be made of the frequency of treatments or procedures amongst different subgroups of the population or between different geographical areas. �Registries with a more limited coverage are still capable of providing: � �Benchmarking of outcomes: Registries provide the ideal source of data to measure outcomes and compare results amongst different institutions, within Australia and overseas. �Interpretable results require that case ascertainment is complete (from the participating institutions), valid outcome data is obtained and adequate risk adjustment undertaken. � Outcome predictors: Registries may help identify previously unsuspected factors that exert a measurable impact on outcomes. � Volume quality assessment: Registry data has been valuable for defining the numbers of procedures required by individuals and units to achieve optimal results. This data has impacted on training requirements, credentialing etc. � Safety: Systematic collection of outcome data may identify previously unsuspected risks of therapy. For example data from the Australian Joint Replacement registry identified high failure rates with certain joint prostheses. � Compliance with guidelines: Registries that collect information on aspects of disease management are useful for assessing compliance with published treatment guidelines. They provide valuable information to allow variations in practice to be investigated. � Cost-effectiveness: Data from registries may be useful for collecting information about current management approaches and their costs. � Prevention: Cases of a disease collected from registries may be matched with data from matched controls in a case control study to identify potential causal factors for the disease in question. These factors may provide information useful for disease prevention. � �Infrastructure for clinical trials: Registries also provide an infrastructure on which intervention studies can be established with relative ease. The extent of the information provided by disease & procedure registries is more complete if they are population based�i.e. if they collect data from all cases occurring in a defined population. Population based registers typically collect many more cases and therefore have the additional statistical power required to analyse sub-populations. � � With population based registers it is possible to gain information about: Epidemiology: ie the incidence of a disease, trends in the incidence, absolute numbers and costs of procedures, absolute numbers of complications �-Access to care: comparisons may be made of the frequency of treatments or procedures amongst different subgroups of the population or between different geographical areas. �Registries with a more limited coverage are still capable of providing: � �Benchmarking of outcomes: Registries provide the ideal source of data to measure outcomes and compare results amongst different institutions, within Australia and overseas. �Interpretable results require that case ascertainment is complete (from the participating institutions), valid outcome data is obtained and adequate risk adjustment undertaken. � Outcome predictors: Registries may help identify previously unsuspected factors that exert a measurable impact on outcomes. � Volume quality assessment: Registry data has been valuable for defining the numbers of procedures required by individuals and units to achieve optimal results. This data has impacted on training requirements, credentialing etc. � Safety: Systematic collection of outcome data may identify previously unsuspected risks of therapy. For example data from the Australian Joint Replacement registry identified high failure rates with certain joint prostheses. � Compliance with guidelines: Registries that collect information on aspects of disease management are useful for assessing compliance with published treatment guidelines. They provide valuable information to allow variations in practice to be investigated. � Cost-effectiveness: Data from registries may be useful for collecting information about current management approaches and their costs. � Prevention: Cases of a disease collected from registries may be matched with data from matched controls in a case control study to identify potential causal factors for the disease in question. These factors may provide information useful for disease prevention. � �Infrastructure for clinical trials: Registries also provide an infrastructure on which intervention studies can be established with relative ease.

4. Background & Rationale

6. Registries are repositories of systematically collected data from patients diagnosed with a particular disease or undergoing a specific procedure.� Its purpose is to provide researchers with information which can be used to study variation in outcomes and assist improving treatment and management of diseases. The fundamental requirement of a registry is that the data is collected in a systematic and planned fashion, using identical definitions and modes of collection. Successful registries typically restrict their data collection to a limited number of key variables that are epidemiologically sound and relatively simple and inexpensive to collect.� They should be regarded as a data-spine (which other specific studies can be attached to) rather than a comprehensive data collection suitable for answering a large range of possible future questions. �Epidemiologically sound refers to data that is consistently available and reproducible, typically items such as date of birth, blood pressure etc which require little in the way of clinical judgement or interpretation and can be collected by a non-expert. �Registries provide data that is well respected by clinicians and highly relevant to clinical practice Registries are repositories of systematically collected data from patients diagnosed with a particular disease or undergoing a specific procedure.� Its purpose is to provide researchers with information which can be used to study variation in outcomes and assist improving treatment and management of diseases. The fundamental requirement of a registry is that the data is collected in a systematic and planned fashion, using identical definitions and modes of collection. Successful registries typically restrict their data collection to a limited number of key variables that are epidemiologically sound and relatively simple and inexpensive to collect.� They should be regarded as a data-spine (which other specific studies can be attached to) rather than a comprehensive data collection suitable for answering a large range of possible future questions. �Epidemiologically sound refers to data that is consistently available and reproducible, typically items such as date of birth, blood pressure etc which require little in the way of clinical judgement or interpretation and can be collected by a non-expert. �Registries provide data that is well respected by clinicians and highly relevant to clinical practice

7. �Registries require personal medical information to be collected and pooled from large numbers of individuals. Ideally this should only be undertaken with specific permission in advance from each individual whose data is involved.� Some ethics committees have equated participation in registries to participation in clinical trials and insisted that advance permission be sought before any patient�s data is included. �The downside of seeking individual permission is that participation rates are likely to be low. This has been well documented in overseas studies (quote NEJM 2003). When the participation rate is low there is strong likelihood that the resultant participants will be unrepresentative and the data therefore unsuited for quality improvement or benchmarking. In particular it would not be acceptable for benchmarking of performance. �To overcome the problems associated with low participation rates registries typically request approval for an �opt out� approach for data collection. Under this approach patients are provided with a leaflet explaining the purposes and procedures of the registry, and told that their identity and some specific clinical information will be passed to the registry unless they contact the registry to register their objection. These procedures provide potential participants with a means for rejecting participation but experience shows that this option is rarely exercised. � Registries also require publication policies (needs expansion) �Registries require personal medical information to be collected and pooled from large numbers of individuals. Ideally this should only be undertaken with specific permission in advance from each individual whose data is involved.� Some ethics committees have equated participation in registries to participation in clinical trials and insisted that advance permission be sought before any patient�s data is included. �The downside of seeking individual permission is that participation rates are likely to be low. This has been well documented in overseas studies (quote NEJM 2003). When the participation rate is low there is strong likelihood that the resultant participants will be unrepresentative and the data therefore unsuited for quality improvement or benchmarking. In particular it would not be acceptable for benchmarking of performance. �To overcome the problems associated with low participation rates registries typically request approval for an �opt out� approach for data collection. Under this approach patients are provided with a leaflet explaining the purposes and procedures of the registry, and told that their identity and some specific clinical information will be passed to the registry unless they contact the registry to register their objection. These procedures provide potential participants with a means for rejecting participation but experience shows that this option is rarely exercised. � Registries also require publication policies (needs expansion)

8. Collecting �identifying information� Name, DOB, address Required for delayed outcomes tracking Data-security

9. Opt-in consent invalidates most registries

10. Ethics Committee �tests� (S95a) Public health significance Rationale for using identified data Rationale for avoiding opt-in consent Credentials of researchers Sensitivity of data Security measures Planned use of data (including linkage, contact with participants)

11. Part of the �science� of registries is to choose the data elements with great care and skill.� There is a trade off between data that is epidemiologically sound and easily accessible with that necessary to adequately describe the condition and/or its treatment and outcome. Identifying information Typically it is necessary to include identifying information in registries in order to: avoid duplicates, � track individuals whose treatment may span several institutions, to allow follow-up to ascertain outcomes, and to allow further data to be collected to answer additional questions.� In the absence of a unique health identification number, it is typically necessary to include names and addresses (in such cases special arrangements must be incorporated to maintain privacy- usually by maintaining a file containing the identifying information separately under high security). � When unique identifiers are not available it is necessary to collect information to minimise losses to follow-up. As with clinical trials this typically requires the collection of information that will allow follow-up ie names of the closest relative, closes friend and family doctor. Basic demography and essential clinical information Although all cases require key information about age, sex, key dates etc the precise nature of the clinical information collected will vary according to the purpose of the registries. �For example a typical registry will wish to collect specific details of the condition being treated (including its severity) together with details of the treatment given.. Part of the �science� of registries is to choose the data elements with great care and skill.� There is a trade off between data that is epidemiologically sound and easily accessible with that necessary to adequately describe the condition and/or its treatment and outcome. Identifying information Typically it is necessary to include identifying information in registries in order to: avoid duplicates, � track individuals whose treatment may span several institutions, to allow follow-up to ascertain outcomes, and to allow further data to be collected to answer additional questions.� In the absence of a unique health identification number, it is typically necessary to include names and addresses (in such cases special arrangements must be incorporated to maintain privacy- usually by maintaining a file containing the identifying information separately under high security). � When unique identifiers are not available it is necessary to collect information to minimise losses to follow-up. As with clinical trials this typically requires the collection of information that will allow follow-up ie names of the closest relative, closes friend and family doctor. Basic demography and essential clinical information Although all cases require key information about age, sex, key dates etc the precise nature of the clinical information collected will vary according to the purpose of the registries. �For example a typical registry will wish to collect specific details of the condition being treated (including its severity) together with details of the treatment given..

12. Much of the potential value of registries is gained only if information is collected about the outcome of a treatment or procedure. �In some cases the outcome may be��immediate� such as when death or surgery is used as the outcome after coronary angioplasty. In other instances a meaningful outcome can be assessed only after a delay of some weeks to months. For example the outcome of severe trauma may not be stable and assessable for 6-12 months after the injury is sustained. In these cases information may need to be gained by contacting patients by phone or in person. The key is that the information is collected from all�cases in a systematic way and at a defined interval. In collecting outcome information it is often necessary to make a trade off between the time elapsed and the ability to access all patients. For example in the �severe �trauma registry the ability to maintain contact with all patients declines rapidly with time and a trade off must be made between allowing time for maximum clinical recovery and minimising loss to follow-up. Hence the follow-up has been undertaken at 6 months rather than the ideal 12 months of elapsed time. �Since outcome information is commonly collected by phone, a detailed protocol is often developed to allow this information to be collected most efficiently. Stratification of patients into useful subgroups can be achieved with a small number of key questions. An example is the �four question test� used by Warlow (2001) to classify outcomes after stroke. These allow subjects to be subdivided according to whether they are dead, confined to bed, requiring assistance with toileting, or independent. � Much of the potential value of registries is gained only if information is collected about the outcome of a treatment or procedure. �In some cases the outcome may be��immediate� such as when death or surgery is used as the outcome after coronary angioplasty. In other instances a meaningful outcome can be assessed only after a delay of some weeks to months. For example the outcome of severe trauma may not be stable and assessable for 6-12 months after the injury is sustained. In these cases information may need to be gained by contacting patients by phone or in person. The key is that the information is collected from all�cases in a systematic way and at a defined interval. In collecting outcome information it is often necessary to make a trade off between the time elapsed and the ability to access all patients. For example in the �severe �trauma registry the ability to maintain contact with all patients declines rapidly with time and a trade off must be made between allowing time for maximum clinical recovery and minimising loss to follow-up. Hence the follow-up has been undertaken at 6 months rather than the ideal 12 months of elapsed time. �Since outcome information is commonly collected by phone, a detailed protocol is often developed to allow this information to be collected most efficiently. Stratification of patients into useful subgroups can be achieved with a small number of key questions. An example is the �four question test� used by Warlow (2001) to classify outcomes after stroke. These allow subjects to be subdivided according to whether they are dead, confined to bed, requiring assistance with toileting, or independent. �

13. Typical outcomes Dead/alive Typical brief 4 question functional assessment via phone Dead/alive Bedridden/ambulant Requiring assistance with ADL/no assistance Specific items Eg thrombotic episodes with rF7a Interpretation may be difficult Controls: often inadequate Confounding by underlying conditions, routinely co-administered therapy

14. Risk adjustment: A continuing and obsessive focus on data quality is a fundamental part of a registry. Collection of data from widely dispersed sites is a well-established risk factor for poor quality data (and sometimes fraudulent) data collection. �Furthermore the increasing use of registry data for performance measurement is may create legal issues if a unit�s reputation or an individual�s career is tarnished by inaccurate information. For these reasons alone the data from registries must be the subject of rigorous quality checks. Typically such checks involve a focus on: �Completeness For most purposes the data from registries is rendered virtually useless if units are allowed to �cherry pick� i.e. to select which patients they will report to a registry and which they will omit. Under such circumstances suspicion of manipulation of the data provided, eg by reporting only those cases with a good outcome, is unavoidable. ��The proportion of eligible cases referred is therefore a fundamental quality measure �which must be ascertained by comparing reported cases to eligible cases by inspection of appropriate hospital logs. �Accuracy Data collectors should be subject to a routine review of perhaps 5% of cases by reference to source data records. More intensive targeting should be directed at units where the error rate or missing data rate is high or where there are delays in receiving data. Percentages of errors should be rigorously monitored and remedial training provided to those whose accuracy rate is below an acceptable rate Timeliness The impact of much of the performance data gained from registries is lessened by delayed reporting. A key performance indicator is therefore a measure of the delay involved in obtaining complete data sets from individual data collectors. Analysis Inaccuracies can often occur once the data has been received because of inaccurate data entry or poor data processing methods. Performance in these aspects� requires systematic �assessment through range checks, consistency checks etc Risk adjustment: A continuing and obsessive focus on data quality is a fundamental part of a registry. Collection of data from widely dispersed sites is a well-established risk factor for poor quality data (and sometimes fraudulent) data collection. �Furthermore the increasing use of registry data for performance measurement is may create legal issues if a unit�s reputation or an individual�s career is tarnished by inaccurate information. For these reasons alone the data from registries must be the subject of rigorous quality checks. Typically such checks involve a focus on: �Completeness For most purposes the data from registries is rendered virtually useless if units are allowed to �cherry pick� i.e. to select which patients they will report to a registry and which they will omit. Under such circumstances suspicion of manipulation of the data provided, eg by reporting only those cases with a good outcome, is unavoidable. ��The proportion of eligible cases referred is therefore a fundamental quality measure �which must be ascertained by comparing reported cases to eligible cases by inspection of appropriate hospital logs. �Accuracy Data collectors should be subject to a routine review of perhaps 5% of cases by reference to source data records. More intensive targeting should be directed at units where the error rate or missing data rate is high or where there are delays in receiving data. Percentages of errors should be rigorously monitored and remedial training provided to those whose accuracy rate is below an acceptable rate Timeliness The impact of much of the performance data gained from registries is lessened by delayed reporting. A key performance indicator is therefore a measure of the delay involved in obtaining complete data sets from individual data collectors. Analysis Inaccuracies can often occur once the data has been received because of inaccurate data entry or poor data processing methods. Performance in these aspects� requires systematic �assessment through range checks, consistency checks etc

15. Finally, the governance of a register. It is important to sort this out at an early stage because in the past disputes have arisen about issues such as registry ownership and access to data.��� My suggestions (that I think are in keeping with policies being developed elsewhere) � -��������� a register should be established in a strong clinical research environment, independent of� providers of care � -��������� the� Director of the register should report to a steering committee comprising senior clinicians and representatives of the funding body � ����������� -���� the register should be established in an environment capable of meeting highest standards of data quality monitoring and maintenance of privacy Finally, the governance of a register. It is important to sort this out at an early stage because in the past disputes have arisen about issues such as registry ownership and access to data.��� My suggestions (that I think are in keeping with policies being developed elsewhere) � -��������� a register should be established in a strong clinical research environment, independent of� providers of care � -��������� the� Director of the register should report to a steering committee comprising senior clinicians and representatives of the funding body � ����������� -���� the register should be established in an environment capable of meeting highest standards of data quality monitoring and maintenance of privacy

16. Drug & device registries: special issues Monitor safety, not efficacy Trials: limited population, determine efficacy Registries: broad population, determine safety Size Not seen in 3000, excludes 1/1000 Completeness of follow-up Close to 100% Interpretation May be difficult without adequate controls

17. Research approaches Case-control studies Registries supply source of cases and controls Cohort studies Multiple outcomes can be studied Frequently safety related Clinical trials Health services research Access, appropriateness, quality

19. ASCTS Database Project Australian Rheumatology Association Database Bosentan Patient Registry (finished) Melbourne Interventional Group (MIG) Interventional Cardiology Registry Melbourne Vascular Surgeons Association Registry REACH Registry National burns registry Surveillance of workplace Based Respiratory Events (SABRE) Victorian Cardiac Arrest Registry (managed by Ambulance Victoria) Victorian Orthopaedic Trauma Registry (VOTOR) Victorian State Trauma Registry (VSTORM) Haemostasis Registry Registries associated with DEPM

20. Developments Registry standards Registry special interest group Contact Dr Sue Evans (sue.evans@med.monash.edu.au) Sustainable funding model

21. Key points Era of registry development Minimalistic data Epidemiologically sound Ethical constraints being loosened Quality control important