National Public Health Performance Standards Program NPHPSP

1. National Public Health Performance Standards Program (NPHPSP) At the conclusion of this power point, participants will be able to: Navigate the CDC NPHPSP data entry website Retrieve reports of results from CDC NPHPSP website Use reports to identify challenges and opportunities Integrate results from the local public health system assessment in a larger MAPP-based process At the conclusion of this power point, participants will be able to: Navigate the CDC NPHPSP data entry website Retrieve reports of results from CDC NPHPSP website Use reports to identify challenges and opportunities Integrate results from the local public health system assessment in a larger MAPP-based process

2. Entering Your Data Obtain User ID and Password Contact Ms. Ali Gilmore at Ali_Gilmore@doh.state.fl.us or by phone at 850-245-4444 ext. 2038 Go to CDC website www.phppo.cdc.gov/takesurvey Click on “Begin Survey” Enter the survey number and password Survey number: 780 Survey password: 780 Hit “Continue” and proceed to the next screen When the local public health system assessment is complete, you will need to enter your data (that is, the consensus responses to the survey instrument questions). The first step is obtain your User Id and password from Ms. Ali Gilmore of the DOH’s Office of Planning, Evaluation and Data Analysis. Her email is Ali_Gilmore@doh.state.fl.us or you may call her at 850-245-4444 ext. 2038 or SunCom 205-4444 ext. 2038. Next, go the CDC’s limited access website at www.phppo.cdc.gov/takesurvey “Enter the Survey” will appear on the page. Click on the “Begin Survey” link.When the local public health system assessment is complete, you will need to enter your data (that is, the consensus responses to the survey instrument questions). The first step is obtain your User Id and password from Ms. Ali Gilmore of the DOH’s Office of Planning, Evaluation and Data Analysis. Her email is Ali_Gilmore@doh.state.fl.us or you may call her at 850-245-4444 ext. 2038 or SunCom 205-4444 ext. 2038. Next, go the CDC’s limited access website at www.phppo.cdc.gov/takesurvey “Enter the Survey” will appear on the page. Click on the “Begin Survey” link.

3. Entering Your Data (cont.) Use your mouse to click on the drop-down menu to select the section (Essential Service) you want to answer Sections need not be answered in numerical order Before final submission of data, users must complete Demographics questionnaire Evaluation survey Using your mouse, click on the drop-down menu to select the section you would like to answer. You may complete the survey in numerical order, beginning with Essential Service #1 or you may begin with any other Essential Service. Before final submission of data, users will be asked to complete a brief demographics questionnaire and an evaluation survey.Using your mouse, click on the drop-down menu to select the section you would like to answer. You may complete the survey in numerical order, beginning with Essential Service #1 or you may begin with any other Essential Service. Before final submission of data, users will be asked to complete a brief demographics questionnaire and an evaluation survey.

4. Entering Your Data (cont.) When entering responses to Essential Services Have your hard copy in front of you Save responses frequently Validation (system check for missed questions) only happens after hitting “Fully Submit” When entering responses in Essential Public Health Services 1-10 sections, please remember Have a full set of hard copy responses in front of you which you will transfer to the on-line system Save responses at any time and save frequently Validation (when the system checks for missed questions) is only performed when you press the “Fully Submit” button Tip! Once a page is displayed, you have 20 minutes to enter your responses, save the information and/or select a section before being automatically logged out of the system. When 18 minutes have expired, a warning message will pop up. You can log back in at any time by clicking on the “Log In” link on the left navigation menu. A pop up window will appear for you to enter your log in information. Your answers will not be lost.When entering responses in Essential Public Health Services 1-10 sections, please remember Have a full set of hard copy responses in front of you which you will transfer to the on-line system Save responses at any time and save frequently Validation (when the system checks for missed questions) is only performed when you press the “Fully Submit” button Tip! Once a page is displayed, you have 20 minutes to enter your responses, save the information and/or select a section before being automatically logged out of the system. When 18 minutes have expired, a warning message will pop up. You can log back in at any time by clicking on the “Log In” link on the left navigation menu. A pop up window will appear for you to enter your log in information. Your answers will not be lost.

5. Entering Your Data (cont.) When entering data in the Demographics and Evaluation sections: Fill in a response for each question Validation is performed each time you “save” Here are two helpful tips. First, after saving your responses, you must wait until the system returns to the “Section Selection” screen before proceeding. This may take a few moments. Be patient! Second, if your response to a question is Yes, High Partially or Low Partially and the question has sub-questions, responses are required for each sub-question. If your response to a question is No and there are sub-questions, please do not select a response for the sub-questions. When entering responses in the Demographics and Evaluation sections Fill in a response for each question Validation is performed during any attempt to save. Each time you save, you will have 20 additional minutes to enter information before the system times out. Save frequently. The demographics questionnaire asks for information such as population size of the jurisdiction, basic characteristics of the county health department and the participants involved in the performance assessment process. The evaluation survey asks several questions about how the assessment was conducted, suggestions for improvement, and plans for using results.Here are two helpful tips. First, after saving your responses, you must wait until the system returns to the “Section Selection” screen before proceeding. This may take a few moments. Be patient! Second, if your response to a question is Yes, High Partially or Low Partially and the question has sub-questions, responses are required for each sub-question. If your response to a question is No and there are sub-questions, please do not select a response for the sub-questions. When entering responses in the Demographics and Evaluation sections Fill in a response for each question Validation is performed during any attempt to save. Each time you save, you will have 20 additional minutes to enter information before the system times out. Save frequently. The demographics questionnaire asks for information such as population size of the jurisdiction, basic characteristics of the county health department and the participants involved in the performance assessment process. The evaluation survey asks several questions about how the assessment was conducted, suggestions for improvement, and plans for using results.

6. Entering Your Data (cont.) Automated process begins to generate report of results Reports available within 48 hours When all sections are completed, press the “Survey Fully Completed” button May get error message if any questions missed Once you have completed all sections, press the “Survey Fully Completed” button. At this time you may receive an error message reminding you of any questions that you may not have answered or if you answered a question in a way that confuses the system. Tip! Keep the hard copies of all your responses handy so that you can correct any errors that the system identifies after you have hit the “Survey Fully Completed” button. Once you press the final “submit” button, an automated process will begin to generate a report of results. In most cases, within 48 hours you will be able to download your final reportOnce you have completed all sections, press the “Survey Fully Completed” button. At this time you may receive an error message reminding you of any questions that you may not have answered or if you answered a question in a way that confuses the system. Tip! Keep the hard copies of all your responses handy so that you can correct any errors that the system identifies after you have hit the “Survey Fully Completed” button. Once you press the final “submit” button, an automated process will begin to generate a report of results. In most cases, within 48 hours you will be able to download your final report

7. Accessing Reports of Results Log in using Survey number 780 Survey password 780 User ID Self-selected password To locate your final report to the following URL approximately 2-3 business days after you finish submitting your data: www.phppo.cdc.gov/takesurvey/reports/login.asp Please note! CDC will not contact you to tell you your reports are ready; it is your responsibility to check the website. On the website you will see prompts to input the survey login and password and your User ID and self-selected password. To locate your final report to the following URL approximately 2-3 business days after you finish submitting your data: www.phppo.cdc.gov/takesurvey/reports/login.asp Please note! CDC will not contact you to tell you your reports are ready; it is your responsibility to check the website. On the website you will see prompts to input the survey login and password and your User ID and self-selected password.

8. Accessing Reports of Results Reports Narrative Summary data Scores in two formats Raw data After inputting this information, you’ll see a screen that provides four options for downloading documents. They are: A report narrative that provides an over of the report, data limitations, and some information for using results for quality improvement A summary of the data, along with key information displayed in charts and graphs Two files that allow the user to access their scores in two different formats. These files are useful for those who are interested in conducting further analysis A file with all of the raw data. After inputting this information, you’ll see a screen that provides four options for downloading documents. They are: A report narrative that provides an over of the report, data limitations, and some information for using results for quality improvement A summary of the data, along with key information displayed in charts and graphs Two files that allow the user to access their scores in two different formats. These files are useful for those who are interested in conducting further analysis A file with all of the raw data.

9. Sample Reports This bar chart will be included in the automated report to respondents. With just a quick glance, the responding system can see that Essential Services 2 and 6 have the highest performance and the most improvement may be needed in Essential Services 5, 7, and 9. This bar chart will be included in the automated report to respondents. With just a quick glance, the responding system can see that Essential Services 2 and 6 have the highest performance and the most improvement may be needed in Essential Services 5, 7, and 9.

10. Sample Reports (cont.) This is another summary report. This shows, by Essential Service, the scores for each indicator.This is another summary report. This shows, by Essential Service, the scores for each indicator.

11. Sample Reports (cont.) This bar graph depicts the summary scores for each essential service in descending order (strengths being the higher scores)This bar graph depicts the summary scores for each essential service in descending order (strengths being the higher scores)

12. Sample Reports (cont.) This bar graph shows the summary scores for each essential service.This bar graph shows the summary scores for each essential service.

13. Sample Reports (cont.) Another of the reports you will have access to this summary of performance by model standards. For each model standard you can quickly see where your score falls.Another of the reports you will have access to this summary of performance by model standards. For each model standard you can quickly see where your score falls.

14. Sample Reports (cont.) This pie chart depicts overall performance on model standards. Blue represents the percentage of model standards that are “fully met,” pink are those “substantially met,” yellow represents those “partially met,” and show in red is the remaining percentage that is “not met.” There is another pie chart that shows an overhead view of these data (not shown here).This pie chart depicts overall performance on model standards. Blue represents the percentage of model standards that are “fully met,” pink are those “substantially met,” yellow represents those “partially met,” and show in red is the remaining percentage that is “not met.” There is another pie chart that shows an overhead view of these data (not shown here).

15. Sample Reports (cont.) This chart shows, for each measure, the contribution of the system and contribution of the county health department. These data depict the responses to the summary question for each indicatorThis chart shows, for each measure, the contribution of the system and contribution of the county health department. These data depict the responses to the summary question for each indicator

16. Identifying Challenges and Opportunities Partners discuss results Through consensus, categorize Success (maintain effort) Success (cut back resources) Challenge (increased activity) Challenge (increased coordination) This step is perhaps the most important because it is at this stage that your local public health system partners will discuss the results and identify challenges and opportunities. Through discussion, participants should be able to categorize the indicators (from the performance measures instrument) into a list of challenges and opportunities. Through consensus discussions, it may be helpful to categorize the activities (indicators) using these groups: Success – maintain effort: Activity is being done well. We should maintain current level of effort Success – cut back resources: This activity is being done well but can be cut back (i.e., has reached maintenance level, decreasing demand). We can withdraw some resources from this activity to devote to some other priority area. Challenge – requires increased activity: This activity requires improvement. More attention is needed in this area. Challenge – requires increased coordination: This activity requires improvement. Better coordination among partners should occur Remember to use the recorder’s notes from the NPHPSP local process to remind participants of their previous discussions. These notes are a rich source of information. The list should be comprehensive enough to include the issues identified in the assessment but short enough (i.e., 10-15 items) for the local public health system to address many of them. This step is perhaps the most important because it is at this stage that your local public health system partners will discuss the results and identify challenges and opportunities. Through discussion, participants should be able to categorize the indicators (from the performance measures instrument) into a list of challenges and opportunities. Through consensus discussions, it may be helpful to categorize the activities (indicators) using these groups: Success – maintain effort: Activity is being done well. We should maintain current level of effort Success – cut back resources: This activity is being done well but can be cut back (i.e., has reached maintenance level, decreasing demand). We can withdraw some resources from this activity to devote to some other priority area. Challenge – requires increased activity: This activity requires improvement. More attention is needed in this area. Challenge – requires increased coordination: This activity requires improvement. Better coordination among partners should occur Remember to use the recorder’s notes from the NPHPSP local process to remind participants of their previous discussions. These notes are a rich source of information. The list should be comprehensive enough to include the issues identified in the assessment but short enough (i.e., 10-15 items) for the local public health system to address many of them.

17. Strategies (continued) Small Group Activities Work groups or subcommittees Advantages Allows for expertise, as needed Less intense Drawbacks Less cross-learning Less consistency in response development Second, many sites have split up the instrument so that responses are developed by small groups or subcommittees (e.g., each is responsible for three ES). If this is done, some attention should be given to maximizing cross-learning and consistency in responses. For example, consider having the same facilitators and recorders or a “core group” that participates in all subcommittees. Second, many sites have split up the instrument so that responses are developed by small groups or subcommittees (e.g., each is responsible for three ES). If this is done, some attention should be given to maximizing cross-learning and consistency in responses. For example, consider having the same facilitators and recorders or a “core group” that participates in all subcommittees.

18. Strategies (continued) Series of Meetings (e.g., 5 meetings completing 2 ES at each) Advantages Allows for expertise, as needed Gets work done in small pieces Drawbacks Participation changes with meeting attendance Process may seem to “drag on” A third option is to conduct a series of meetings (e.g., 5 meetings at which 2 ES are discussed at each). This allows for a core group to participate, as well as experts to be brought in, as needed. The work is also done in small pieces, which can be seem more manageable. However, conveners need to explore how to minimize the drawbacks: attendance at meetings will change with participant availability, and the process can also seem to “drag on” Once you have selected the meeting format you will use, focus your efforts on identifying recorders and facilitators.A third option is to conduct a series of meetings (e.g., 5 meetings at which 2 ES are discussed at each). This allows for a core group to participate, as well as experts to be brought in, as needed. The work is also done in small pieces, which can be seem more manageable. However, conveners need to explore how to minimize the drawbacks: attendance at meetings will change with participant availability, and the process can also seem to “drag on” Once you have selected the meeting format you will use, focus your efforts on identifying recorders and facilitators.

19. Facilitator and Recorder Roles Recorder Track responses Capture discussion points including Priority issues Barriers Solutions Documented points to be used when selecting priorities and making action plans May have two or more recorders To keep the process flowing smoothly, a facilitator and a recorder should be designated so that discussions do not get bogged down. This is especially true when consensus-building around the responses needs to occur. The Recorder has a critical role in the process’s success. In addition to tracking the responses, so that they can be inputted on the NPHPS website and submitted to CDC, the recorder should keep track of ideas, solutions, and comments that arise during discussions. Comments on priority areas, possible solutions, barriers, and new ideas for system coordination often arise from these discussions. These comments will be very useful in the action planning discussions that will take place later. You may want to consider having two recorders – one to track responses (or votes) and a second to track comments. A strong facilitator can help keep everyone on track and ensure the discussion moves along. The facilitator should also inform participants of time commitments and keep to those commitments. The facilitator should focus the discussion on the overall model standard and measures in order for the group to come to a common understanding of the system’s activities and develop consensus responses. Both the facilitator and the recorder roles should ensure that the group stays focused and gets through the process in a timely manner. To keep the process flowing smoothly, a facilitator and a recorder should be designated so that discussions do not get bogged down. This is especially true when consensus-building around the responses needs to occur. The Recorder has a critical role in the process’s success. In addition to tracking the responses, so that they can be inputted on the NPHPS website and submitted to CDC, the recorder should keep track of ideas, solutions, and comments that arise during discussions. Comments on priority areas, possible solutions, barriers, and new ideas for system coordination often arise from these discussions. These comments will be very useful in the action planning discussions that will take place later. You may want to consider having two recorders – one to track responses (or votes) and a second to track comments. A strong facilitator can help keep everyone on track and ensure the discussion moves along. The facilitator should also inform participants of time commitments and keep to those commitments. The facilitator should focus the discussion on the overall model standard and measures in order for the group to come to a common understanding of the system’s activities and develop consensus responses. Both the facilitator and the recorder roles should ensure that the group stays focused and gets through the process in a timely manner.

20. Facilitator and Recorder Roles Recorder Track responses Capture discussion points including Priority issues Barriers Solutions Documented points to be used when selecting priorities and making action plans May have two or more recorders To keep the process flowing smoothly, a facilitator and a recorder should be designated so that discussions do not get bogged down. This is especially true when consensus-building around the responses needs to occur. The Recorder has a critical role in the process’s success. In addition to tracking the responses, so that they can be inputted on the NPHPS website and submitted to CDC, the recorder should keep track of ideas, solutions, and comments that arise during discussions. Comments on priority areas, possible solutions, barriers, and new ideas for system coordination often arise from these discussions. These comments will be very useful in the action planning discussions that will take place later. You may want to consider having two recorders – one to track responses (or votes) and a second to track comments. A strong facilitator can help keep everyone on track and ensure the discussion moves along. The facilitator should also inform participants of time commitments and keep to those commitments. The facilitator should focus the discussion on the overall model standard and measures in order for the group to come to a common understanding of the system’s activities and develop consensus responses. Both the facilitator and the recorder roles should ensure that the group stays focused and gets through the process in a timely manner. To keep the process flowing smoothly, a facilitator and a recorder should be designated so that discussions do not get bogged down. This is especially true when consensus-building around the responses needs to occur. The Recorder has a critical role in the process’s success. In addition to tracking the responses, so that they can be inputted on the NPHPS website and submitted to CDC, the recorder should keep track of ideas, solutions, and comments that arise during discussions. Comments on priority areas, possible solutions, barriers, and new ideas for system coordination often arise from these discussions. These comments will be very useful in the action planning discussions that will take place later. You may want to consider having two recorders – one to track responses (or votes) and a second to track comments. A strong facilitator can help keep everyone on track and ensure the discussion moves along. The facilitator should also inform participants of time commitments and keep to those commitments. The facilitator should focus the discussion on the overall model standard and measures in order for the group to come to a common understanding of the system’s activities and develop consensus responses. Both the facilitator and the recorder roles should ensure that the group stays focused and gets through the process in a timely manner.

21. Facilitator and Recorder Roles (continued) Facilitator Key to successful process Must be neutral Ensures Discussion stays on track Time limits are honored Common understanding and development of consensus responses May have more than one facilitator For consistency try to have the same facilitators A strong facilitator is essential to an effective process. The facilitator should not only be skilled but neutral. CHD staff should avoid facilitating their own assessment meetings to prevent bias. It is important to identify the facilitator early in the planning so that he/she can familiarize themselves with the instrument and the process. It is strongly recommended to use a facilitator who is familiar with your community and the mission of public health. The facilitator’s key responsibility is to keep everyone on track and ensure the discussion moves along. The facilitator should also inform participants of time commitments and keep to those commitments. The facilitator should focus the discussion on the overall model standard and measures in order for the group to come to a common understanding of the system’s activities and develop consensus responses. You may choose to have more than one facilitator or to “team” facilitate. For consistency, have the same facilitators at each meeting (if doing a series) and that the co-facilitators use the same method of moving through the instrument and reaching consensus. Both the facilitator and the recorder roles should ensure that the group stays focused and gets through the process in a timely manner. There is a detailed facilitator’s guide in the materials on the COMPASS website. Tips from the field ……. A strong facilitator is essential to an effective process. The facilitator should not only be skilled but neutral. CHD staff should avoid facilitating their own assessment meetings to prevent bias. It is important to identify the facilitator early in the planning so that he/she can familiarize themselves with the instrument and the process. It is strongly recommended to use a facilitator who is familiar with your community and the mission of public health. The facilitator’s key responsibility is to keep everyone on track and ensure the discussion moves along. The facilitator should also inform participants of time commitments and keep to those commitments. The facilitator should focus the discussion on the overall model standard and measures in order for the group to come to a common understanding of the system’s activities and develop consensus responses. You may choose to have more than one facilitator or to “team” facilitate. For consistency, have the same facilitators at each meeting (if doing a series) and that the co-facilitators use the same method of moving through the instrument and reaching consensus. Both the facilitator and the recorder roles should ensure that the group stays focused and gets through the process in a timely manner. There is a detailed facilitator’s guide in the materials on the COMPASS website. Tips from the field …….

22. Moving Through the Instrument Walk through questions one by one Discuss model standard with facilitator/recorder judgment on response Discuss model standard with follow up voting Determining Responses Think about the focus of the questions Dispersion through program areas Participation among many system partners Frequency of activity Answers should reflect system’s capacity All responses should be consensus Regardless of how participation is structured, participants will need to agree on a process for discussing the assessment instrument and identifying responses. Consensus responses should be developed through dialogue among system partner organizations. There are several possible approaches that can be considered, as you design your process: Walk through the instrument and questions one by one – this will allow for a very methodical and thorough process. However, it can seem tedious to participants. Discuss the model standards with facilitator/recorder judgment on responses – Provide participants with only the model standards. Participants can discuss each model standard for a set period of time (e.g., 10 minutes) during which the facilitator ensures that the discussion hits all of the key points addressed in the model standard. The facilitator and recorders make judgments on the responses to the questions (asking follow-up questions as needed) based on the discussion. This keeps the interest of participants’ high, since the discussion is focused more on content than the process of identifying a response. As a drawback, the final responses can be greatly impacted by the facilitator’s and recorder’s perceptions of activities. Discuss the model standards with follow-up voting – Provide participants with the full instruments. Participants discuss each model standard for a set period of time (e.g., 10 minutes) similar to what is described above. After the model standard has been fully discussed, participants vote (using color-coded cards or raised hands) on the response to each question. Further discussion can occur where there is disparity in responses. As stated, all responses should be developed through a consensus process or by coming to a common understanding. Don’t hand out the instruments and ask people to respond individually and then aggregate the responses – then there will be no cross-learning about what activities are currently going on and no new ideas about how activities can be improved. The process will generally require some intensive and lengthy discussions. Also, the group should be sure to think carefully about the focus of the question. For example, for a question about health education activities, there might be a great deal of activities by a few partners, but not by others. Or there may be a lot of health education occurring in certain program areas – like tobacco use – but not in others. And you may need to consider the extent or frequency to which the activity is done. Think about these questions in the context of each measure and be sure that all participants have the same understanding. Regardless of how participation is structured, participants will need to agree on a process for discussing the assessment instrument and identifying responses. Consensus responses should be developed through dialogue among system partner organizations. There are several possible approaches that can be considered, as you design your process: Walk through the instrument and questions one by one – this will allow for a very methodical and thorough process. However, it can seem tedious to participants. Discuss the model standards with facilitator/recorder judgment on responses – Provide participants with only the model standards. Participants can discuss each model standard for a set period of time (e.g., 10 minutes) during which the facilitator ensures that the discussion hits all of the key points addressed in the model standard. The facilitator and recorders make judgments on the responses to the questions (asking follow-up questions as needed) based on the discussion. This keeps the interest of participants’ high, since the discussion is focused more on content than the process of identifying a response. As a drawback, the final responses can be greatly impacted by the facilitator’s and recorder’s perceptions of activities. Discuss the model standards with follow-up voting – Provide participants with the full instruments. Participants discuss each model standard for a set period of time (e.g., 10 minutes) similar to what is described above. After the model standard has been fully discussed, participants vote (using color-coded cards or raised hands) on the response to each question. Further discussion can occur where there is disparity in responses. As stated, all responses should be developed through a consensus process or by coming to a common understanding. Don’t hand out the instruments and ask people to respond individually and then aggregate the responses – then there will be no cross-learning about what activities are currently going on and no new ideas about how activities can be improved. The process will generally require some intensive and lengthy discussions. Also, the group should be sure to think carefully about the focus of the question. For example, for a question about health education activities, there might be a great deal of activities by a few partners, but not by others. Or there may be a lot of health education occurring in certain program areas – like tobacco use – but not in others. And you may need to consider the extent or frequency to which the activity is done. Think about these questions in the context of each measure and be sure that all participants have the same understanding.