2000 Immunization Registry Conference Using Site Visits to Assess Immunization Registry Progress

1. 1  2000 Immunization Registry Conference Using Site Visits to Assess Immunization Registry Progress Cathy Stout Program Analyst Registry Support Team, Systems Development Branch Data Management, NIP Phone: (404) 639-8816 E-Mail: CStout@cdc.gov March 27, 2000 Introduction of self Purpose of talk: To present an overview of a standardized site visit protocol that will be used during site visits and discuss the process used in developing the protocol.Introduction of self Purpose of talk: To present an overview of a standardized site visit protocol that will be used during site visits and discuss the process used in developing the protocol.

2. 2 Using Site Visits to Assess Immunization Registry Progress BACKGROUND: January 12, 1999, NVAC Report: Development of Community- and State-Based Immunization Registries January 2000: Healthy People 2010 Objectives Background As way of background, 2 important documents were published that influenced the decision to develop a standardized site visit protocol. NVAC 2010Background As way of background, 2 important documents were published that influenced the decision to develop a standardized site visit protocol. NVAC 2010

3. 3 Using Site Visits to Assess Immunization Registry Progress January 12, 1999, NVAC Report: Development of Community- and State-Based Immunization Ensure appropriate protections of privacy and confidentiality for individuals and security for information included in the registry Ensure participation of all immunization providers and recipients Ensure appropriate functioning of registries Ensure sustainable funding of registries CDC should monitor the level of provider and recipient participation; the implementation of registries; and provide technical assistance. 1/12/99: NVAC issued report supporting the development of state- and community based registries. Specifically identified 4 registry objectives NVAC also Recommended that CDC monitor and provide assistance 1/12/99: NVAC issued report supporting the development of state- and community based registries. Specifically identified 4 registry objectives NVAC also Recommended that CDC monitor and provide assistance

4. 4 Using Site Visits to Assess Immunization Registry Progress January 2000: Healthy People 2010 Objectives Objective14-26: Increase the proportion of children who participate in fully operational population-based immunization registries. Target: 95% of all children under age 6 years. January 2000: Healthy People 2010 objectives Self explanatoryJanuary 2000: Healthy People 2010 objectives Self explanatory

5. 5 Using Site Visits to Assess Immunization Registry Progress Site visits to immunization registry projects Purpose: Review project’s progress toward meeting the Healthy People 2010 Objective and facilitate registry implementation. Discuss successes and key issues regarding registry implementation Provide recommendations to projects to identify gaps and help focus future activities In order to meet the NVAC recommendations and the HP2010 objectives, the Registry Support Team proposed a standardized approach for monitoring progress by conducting regular site visits to our grantees, with the purpose being: Review slide In order to meet the NVAC recommendations and the HP2010 objectives, the Registry Support Team proposed a standardized approach for monitoring progress by conducting regular site visits to our grantees, with the purpose being: Review slide

6. 6 Using Site Visits to Assess Immunization Registry Progress OUTCOMES: Confirmation of registry functionality Data to support the certification process Accurate baseline data to measure future progress Publications supporting immunization registries Some of the proposed outcomes that will enable CDC to monitor and facilitate progress include:: confirmation of registry functionality collection of data to support registry certification the collection of baseline data that can be used to measure future progress; and information for future journal publications documenting immunization registry use and implementation. Some of the proposed outcomes that will enable CDC to monitor and facilitate progress include:: confirmation of registry functionality collection of data to support registry certification the collection of baseline data that can be used to measure future progress; and information for future journal publications documenting immunization registry use and implementation.

7. 7 Using Site Visits to Assess Immunization Registry Progress OUTCOMES: Document registry achievements lessons learned and shared marketing support state initiatives future funding options Facilitate solutions Other outcomes that will offer direct benefits to the grantees include: Our ablity to document registry achievements, enabling us to: share experiences, lessons learned, and other activities with all registry projects document registry successes that can be used to develop marketing strategies provide feedback to projects that can be used to support state initiatives offer possible options for future funding sources since Federal immunization funds are limited and assist you in supporting future funding initiatives by providing data necessary to identify underfunded activities and support future funding requests In addition, this information will allow us to facilitate solutions: We will be able to document common problems and offer various methods of resolution. Other outcomes that will offer direct benefits to the grantees include: Our ablity to document registry achievements, enabling us to: share experiences, lessons learned, and other activities with all registry projects document registry successes that can be used to develop marketing strategies provide feedback to projects that can be used to support state initiatives offer possible options for future funding sources since Federal immunization funds are limited and assist you in supporting future funding initiatives by providing data necessary to identify underfunded activities and support future funding requests In addition, this information will allow us to facilitate solutions: We will be able to document common problems and offer various methods of resolution.

8. 8 Using Site Visits to Assess Immunization Registry Progress ACTIVITIES: DMD Registry Support Team staff assigned as state consultants to monitor immunization registry activities Site Visit Protocol Working Group formed Standardized site visit protocol drafted Site visits to Federally-funded immunization registries Post-visit trip report with recommendations -- to be developed In order to accomplish the proposed goals and objectives, several activities were initiatied: slide bullets -- summarize each separately In order to accomplish the proposed goals and objectives, several activities were initiatied: slide bullets -- summarize each separately

9. 9 Using Site Visits to Assess Immunization Registry Progress DMD Registry Support Team staff assigned as project consultants: Dontanette Cohill: IA, MI, MN, MT, NE, ND, SD, WI Phyllis Harmon: AL, FL, GA, KY, MS, NYC, PR, VI Gail Horlick: DE, DC, ME, MD, NJ, PA, Phil, VA Alton Reddick: AR, Houston, IN, LA, NH, NM, NC, OH, OK, TN, TX, San Antonio, SC, VT, WV Angela Salazar: AK, AZ, ID, NV, NY, OR, UT, WA Cathy Stout: Chic, CO, CT, IL, KS, MO, RI, WY, AS, CA, Guam, HI, Mars. Isl, MA, Micro, NMI, Palau, The first step was to assign staff members responsibility for working with specific states in order to provide one contact point for advice and assistance. Each of the above staff members will be providing assistance to the states noted. A listing of the RST and ISD consultant assignments is included in your handouts. The first step was to assign staff members responsibility for working with specific states in order to provide one contact point for advice and assistance. Each of the above staff members will be providing assistance to the states noted. A listing of the RST and ISD consultant assignments is included in your handouts.

10. 10 Using Site Visits to Assess Immunization Registry Progress Site Visit Protocol Work Group formed Systems Development Branch (SDB), DMD: Cathy Stout, Chair; Jim Harrison, Angela Salazar, John Woodfin, Julie Gamez, Dontanette Cohill, Gary Urquhart, Nancy Koughan, Ali Rashidee Immunization Services Division (ISD): Marie Mitchell, Mehran Massoudi Vaccines for Children Program: Laurie Johnson Public Health Practice Program Office (PHPPO): Brian Mahoney All Kids Count (AKC): Kristen Saarlas, KC Edwards (NIP) Additional NIP Division and Branch members Next, a work group was formed to draft a standardized protocol. The initial document development led by SDB Partners were invited to review and provide feedback to the protocol Next, a work group was formed to draft a standardized protocol. The initial document development led by SDB Partners were invited to review and provide feedback to the protocol

11. 11 Using Site Visits to Assess Immunization Registry Progress Standardized site visit protocol package (draft) Contents Site visit guidelines for RST members Memo to Program Manager Draft Agenda Pre-visit materials The protocol is still in the developmental stage. It is a dynamic document that will be refined prior to regular use. Three pilot site visits are being scheduled next month for the purpose of testing and revising the actual protocol. We would also like your input into this process, and there will be time for questions and discussion after the presentations. It contains a number of documents, outlined here. A content page listing all documents in package Guidelines: This document outlines the process to be followed by our Team members and will be given to the Program Manager to provide insight into our process. Memo: A confirmation memo will be sent to each program manager that confirms the proposed date of the site visit and explains the purpose of the visit, the documentation needed prior to the visit, the reasoning behind our requests, and activities that should occur during the visit. A complete site protocol package will be sent to the Program Manager with the memo. Also included is a Draft Agenda . List of pre-visit materials. The list is long and may seem intimidating, but many of these documents may already be available. If not available, they will be discussed during the site visit. The protocol is still in the developmental stage. It is a dynamic document that will be refined prior to regular use. Three pilot site visits are being scheduled next month for the purpose of testing and revising the actual protocol. We would also like your input into this process, and there will be time for questions and discussion after the presentations. It contains a number of documents, outlined here. A content page listing all documents in package Guidelines: This document outlines the process to be followed by our Team members and will be given to the Program Manager to provide insight into our process. Memo: A confirmation memo will be sent to each program manager that confirms the proposed date of the site visit and explains the purpose of the visit, the documentation needed prior to the visit, the reasoning behind our requests, and activities that should occur during the visit. A complete site protocol package will be sent to the Program Manager with the memo. Also included is a Draft Agenda . List of pre-visit materials. The list is long and may seem intimidating, but many of these documents may already be available. If not available, they will be discussed during the site visit.

12. 12 Using Site Visits to Assess Immunization Registry Progress Standardized site visit protocol package List of functions for demonstration (public and private sites) List of documentation to be collected on site Annual Report Assessment Tool Minimum functional standards of a registry List of functions: This is a list of actual functions we would like to see demonstrated.. It will give us an opportunity to see how the registry is being used and will also substantiate the functionality of the registry (e.g., locate records of patients with appointments that day, identify immunizations due that day, enter vaccines administered, enter historical information, query other sytems, etc.). We would like to see demonstrations at both the central site and at 1-2 public/private sites. During demonstrations, discussions will take place focusing on how well the registry is received and used. Samples of different records or reports will be collected at each site (e.g., list of all patients past due, official immunization history, coverage reports). The ability to produce these reports will verify the functionality of the registry. There are two key documents that will guide the site visit: the Annual Report and the Assessment Tool. An Immunization Registry Annual Report is required for all immunization grantees. Since a report is due each year, the RST will complete as many fields as possible, based on prior reports, before sending out a new one to each project to complete. Since the information is self-reported, confirming the data will be a main focus of the site visit. If a project has not submitted a report, one will be requested prior to the site visit. (A copy is in your handouts.) RST Consultants will review the latest report available prior to the site visit. Assessment Tool: This document was developed to guide the site visit. It includes many of the questions included in the annual report along with other programmatic issues to be addressed. It’s main focus, though, is on the minimum functional standards of a registry. Prior to the site visit, the AT will be completed as much as possible based on information provided in the annual report. The agenda will then be modified accordingly. (A copy is in your handouts.) Finally, the package will include the minimum functional standards as described by Julie. List of functions: This is a list of actual functions we would like to see demonstrated.. It will give us an opportunity to see how the registry is being used and will also substantiate the functionality of the registry (e.g., locate records of patients with appointments that day, identify immunizations due that day, enter vaccines administered, enter historical information, query other sytems, etc.). We would like to see demonstrations at both the central site and at 1-2 public/private sites. During demonstrations, discussions will take place focusing on how well the registry is received and used. Samples of different records or reports will be collected at each site (e.g., list of all patients past due, official immunization history, coverage reports). The ability to produce these reports will verify the functionality of the registry. There are two key documents that will guide the site visit: the Annual Report and the Assessment Tool. An Immunization Registry Annual Report is required for all immunization grantees. Since a report is due each year, the RST will complete as many fields as possible, based on prior reports, before sending out a new one to each project to complete. Since the information is self-reported, confirming the data will be a main focus of the site visit. If a project has not submitted a report, one will be requested prior to the site visit. (A copy is in your handouts.) RST Consultants will review the latest report available prior to the site visit. Assessment Tool: This document was developed to guide the site visit. It includes many of the questions included in the annual report along with other programmatic issues to be addressed. It’s main focus, though, is on the minimum functional standards of a registry. Prior to the site visit, the AT will be completed as much as possible based on information provided in the annual report. The agenda will then be modified accordingly. (A copy is in your handouts.) Finally, the package will include the minimum functional standards as described by Julie.

13. 13 Using Site Visits to Assess Immunization Registry Progress Site visits to immunization registry projects Process: Request, from projects, documentation to be submitted prior to visit RST review of project’s latest annual report, saturation measures, and materials received RST completion of assessment tool Finalize site visit agenda focusing on gaps in reporting and key issues needing discussion In preparing for the site visit, the RST Consultants will follow a process consisting of the following steps: Review each step as noted: Request, Review, Complete Finalize the agenda. Discussion should focus on gaps in the annual report and other key programmatic issues. In preparing for the site visit, the RST Consultants will follow a process consisting of the following steps: Review each step as noted: Request, Review, Complete Finalize the agenda. Discussion should focus on gaps in the annual report and other key programmatic issues.

14. 14 Using Site Visits to Assess Immunization Registry Progress Site visits to immunization registry projects Process: Conduct actual site visit Review annual report Discuss the assessment tool, focusing on: minimum functional standards of a registry saturation measures programmatic activities other key issues Once the pre-visit activities (and pilot site visits) have been completed, it’s time to conduct the actual site visit: During the site visit, the following activities will take place: (Review the agenda items as listed: Review AR, discuss AT)Once the pre-visit activities (and pilot site visits) have been completed, it’s time to conduct the actual site visit: During the site visit, the following activities will take place: (Review the agenda items as listed: Review AR, discuss AT)

15. 15 Using Site Visits to Assess Immunization Registry Progress Site visits to immunization registry projects Process: Visit central registry and 1-2 public and private provider sites: demonstration of functionality discussion on ease and use of registry and benefits Exit interview A visit to the central registry and 1-2 public or private provider sites should be scheduled.. During the provider visits, registry functionality should be demonstrated. The specific items to be demonstrated were provided on the list that was included in the site visit protocol package. Talk with the provider staff about: the ease and use of the registry; how often data is submitted and retrieved from the registry consent, opt out or privacy issues; training and support; any benefits or barriers to use; other billing or patient management systems used. Exit interview: The last afternoon should be focused on summarizing the visit and providing suggestions or recommendations for future action. A visit to the central registry and 1-2 public or private provider sites should be scheduled.. During the provider visits, registry functionality should be demonstrated. The specific items to be demonstrated were provided on the list that was included in the site visit protocol package. Talk with the provider staff about: the ease and use of the registry; how often data is submitted and retrieved from the registry consent, opt out or privacy issues; training and support; any benefits or barriers to use; other billing or patient management systems used. Exit interview: The last afternoon should be focused on summarizing the visit and providing suggestions or recommendations for future action.

16. 16 Using Site Visits to Assess Immunization Registry Progress Post-visit trip report with recommendations Standardized report including: purpose and summary of visit current activities key issues suggestions/recommendations After the trip is completed, the project will be sent a trip report. This will be written following a standardized format and include: Review above list After the trip is completed, the project will be sent a trip report. This will be written following a standardized format and include: Review above list

17. 17 Using Site Visits to Assess Immunization Registry Progress In closing, we hope these site visits will: enable CDC to meet the recommendations specified in the NVAC report provide projects with information to: identify gaps and support future activities realize their own success Read slide Reminds me of Marty LaVenture’s Mountain climbing analogy at AKC conferenceRead slide Reminds me of Marty LaVenture’s Mountain climbing analogy at AKC conference