Journal Club. 29/11/13 Jessica Griffith. Chosen Paper. ‘Measuring the quality of anaesthesia from a patient’s perspective: development, validation, and implementation of a short questionnaire’ Hocking et al Br J Anaesth 2013 Dec; 111(6) 979-89. Background.
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Journal Club 29/11/13 Jessica Griffith
Chosen Paper • ‘Measuring the quality of anaesthesia from a patient’s perspective: development, validation, and implementation of a short questionnaire’ • Hocking et al • Br J Anaesth 2013 Dec; 111(6) 979-89
Background • An important aspect of the quality of anaesthetic care is the satisfaction of the patient with their care • Need a suitable instrument to measure this • Simple and easily administered • Validated
Quality Of health care Through the patient’s Eyes (QuOTE) • Series of studies performed in Netherlands Institute of Health Services Research • Items generated and ranked for importance by patients • No input from experts
Validity • Definition: degree to which a research study measures what it intends to measure • Compare with existing psychometric tool • Use statistics e.g. Cronbach’sα
Novel Test of Validity “a valid satisfaction instrument should contain information about the concept that is being measured – the patient’s perception of the quality of anaesthesia. If this is true, effective feedback of this information to anaesthetists might be expected to change the patient’s perception of quality of anaesthesia”
Aims of Study • Develop and validate a short instrument the Perception of Quality in Anaesthesia (PQA) • Identify the most important aspects of anaesthesia quality from the patient’s perspective
Methods • Ethical approval • Single centre - Sir Charles Gairdner Hospital, Perth – university affiliated tertiary institution • October 2008 – Oct 2009
Generating of Items • Asked patients and members of the public to describe any attributes they consider important for ‘high quality anaesthesia’ • Face-to-face and by e-mail • Recruitment continued until no new attributes were generated
Ranking of Items • Attributes were printed onto a pack of cards • A different set of participants ranked them in order of importance • 12 top ranked attributes used in pilot study • Pilot study assessed wording, understanding and practicality
Perception of Quality in Anaesthesia Questionnaire • If you had a general anaesthetic, do you remember anything during surgery? • If you did not have a general anaesthetic (had a nerve block or epidural, etc.) did you feel any pain during the operation? • How would you rate the management of your pain after the operation? • Did you have any nausea (feeling sick) or vomiting after the operation?
Did your anaesthetist address any concerns you had regarding your anaesthetic? • Did your anaesthetist talk to you in a way you could understand? • Did you feel confident in the ability of your anaesthetist? • How would you rate the amount of information your anaesthetist gave you? • How gentle was your anaesthetist? • How would you rate your anaesthetist’s technical skills? (needles, drips etc.) • How friendly was your anaesthetist? • How would you rate the amount of time you had with your anaesthetist?
Comparative Questions • Overall satisfaction: • How would you rate your anaesthetic experience overall? • Would you recommend your anaesthetist to a good friend? • Overall anxiety and understanding of risk: • Were you nervous about having this anaesthetic? • What level of risk did you think you were taking in undergoing anaesthesia?
Validating • Included 714 patients over 4 week period • Participants invited to suggest additional questions more important than those on questionnaire • Test-retest reliability assessed on 100 patients • Content validity study performed with separate sample of post-op patients and medical experts
Execution of Questionnaire • Interview conducted on day after surgery by one of 4 trained interviewers • Inclusion criteria: • adult patients undergoing elective and emergency surgery requiring anaesthetic care • Excluded: • patients ventilated post-op • intellectual impairment requiring permanent care
Scoring • Each question has a five point scale e.g. very poor to very good • Unsatisfactory patient response defined as score ≤ 3 • Performance Score = proportion of patients with unsatisfactory response • Importance Score = mean rank for each attribute • Quality Index = importance score x performance score
Additional Data • Data forms filled in by anaesthetist included: • Patient characteristics • Anaesthetic details • Perioperative details • Data forms cross-checked against anaesthetic record and operating theatre management systems
Feedback • 48 week before and after study • To determine if providing anaesthetists with information from the PQA was associated with any change in patient perception of quality of anaesthesia • 6 month period of individualised feedback • After this PQA performance scores were re-measured
Statistical Analysis • Principle component analysis • Used in exploratory data analysis to determine the structure of correlations within data • Content validity study – Aiken V coefficients were calculated • Poisson regression – to see if responses to PQA questions were associated with various aspects of medical care and patient characteristics
Generation and Ranking • 100 participants generated 52 items • Ranked in order of importance by further 20 participants • Top 12 items included
Patient Validation of the PQA • 714 patients completed questionnaire out of 799 eligible (89% response rate) • Questionnaire completed within 3 mins • Pearson correlation coefficient of test-retest was 0.88 • Highest quality index questions: • PONV question (2.3) • Pain management question (1.04) • Communication with anaesthetist (0.3)
Content Validity Study • 30 post-op patients • Mean Aiken V coefficient of 0.84 (P<0.0001) • 54 medical experts • Mean Aiken V coefficient of 0.72 (P<0.0001) • Indicate satisfactory content validity • Cronbach’sα for all questions was 0.7
Pre-feedback group • Studied weeks 1 – 14 • 2359 patients eligible and 2046 consented (87% response rate) • Unsatisfactory responses • ≥ 1 = 45% • ≥ 2 = 12% • ≥ 3 = 3%
Feedback group • Studied in weeks 15-39 • 4251 patients returned questionnaire
Post-feedback group • Weeks 40-49 • 1721 eligible with 1421 returning questionnaires (83% response rate) • Unsatisfactory responses: • ≥ 1 = 35% (45%) • ≥ 2 = 6% (12%) • ≥ 3 = 1% (3%)
Results Summary • Similar pre- and post-feedback differences were observed if the PQA completed by interview, or left at bedside • Patients in pre-feedback group had 45% higher rate of unsatisfactory responses • Differences in anaesthetic management in post-feedback group: • Increased use of 5HT3 anti-emetics • More inductions of anaesthesia where consultant assisted trainee • Reduced use of morphine as intra-op opioid
Conclusions • “We have developed a short instrument to measure the patient’s perception of the quality of anaesthesia and demonstrated that it is valid and reliable” • “Individualised, repeated feedback of patient satisfaction-related performance to health practitioners can improve the patient’s experience”
Anaesthesia Quality • Communication with anaesthetist • Gentleness and attention • Information and confidence • Addressing the patient’s concerns • Avoidance of PONV • Post-op pain management
Discussion • Validity? • Difficult to determine if any psychometric instrument truly measures what is intended • No gold standard as reference • Conventional methods used • In addition tested PQA in before and after study
Limitations • No power calculation used before starting the investigation • No control group for before and after study (parallel group of patients treated by anaesthetists not exposed to feedback) • Confounded by patient characteristics and surgical differences between pre-and post-feedback groups
Additional Factors Affecting PQA • Patients less likely to give an adverse rating of their experience if they: • underwent shorter surgery • male • older age • less pre-operative anxiety
How is this tool different? • Several instruments already developed to measure patient satisfaction • Aim to measure all aspects of satisfaction – opinions of patients and experts • Lengthy to perform • Factors often included: • Waiting/delays • Recovery nursing care • Continuity of care • Emotional support