Volume 6 no 3

Measuring physician engagement in quality improvement: a pilot study

Back to Index

Measuring physician engagement in quality improvement: a pilot study

Tyrone Perreira, PhD, MEd, Melissa Prokopy, LLB, Adalsteinn Brown, DPhil, Anna Greenberg, MPP, James Wright, MD, MPH, Christine Shea, PhD, MEd, and Julie Simard, PhD

https//doi.org/10.37964/cr24717

 

The term “physician engagement” is overused and often misunderstood. It is believed that system transformation requires physician engagement in quality improvement (QI); however, no tool exists to accurately measure this. The purpose of this study was to develop an instrument that could be used to evaluate physician engagement in QI and then pilot it with a small sample of physicians and physician leaders. An electronic survey was developed using a series of focused literature searches and a modified Delphi panel of QI experts. Cognitive debriefing was performed with a group of physicians and physician leaders. The survey was then administered to 37 physicians working in Ontario hospitals. Descriptive analyses were carried out. This short, easy to administer survey allows for the collection of baseline data on facilitators of physician engagement, as well as training and participation in QI. Construct reliabilities are promising with Cronbach’s alphas between 0.75 and 0.96.

 

KEY WORDS:  physician engagement, quality improvement, hospitals, evaluation, survey

 

Despite the notion that physician engagement in quality improvement (QI) is critical to health system transformation, the concept remains poorly defined, measured, and researched.1 The Ontario Hospital Association’s (OHA’s) Physician Provincial Leadership Council (PPLC), which comprises senior physician leaders from across the province, identified an important need for additional work in this area and supporting evidence identifying facilitators of physician engagement.1 Top

 

The health care literature suggests that the following antecedents are necessary to enhance physician engagement: accountability2-8; communication2,5-29; incentives30 (both financial3,31,32 and non-financial33); and good interpersonal relations between physicians and administrators, with alignment of goals,34-37 values,7,37-40 and beliefs.41 There must be trust14,15,27,40,41 and respect,6,8,38 such that physicians feel supported by their organizations.2,42 The work environment must promote teamwork,14,17,38,43-45 relationship building,46,47 and the development of strategic partnerships,48 whether intergroup18 or peer.47 The environment must provide opportunities to partake in and be involved in leadership8,9,47,49 and decision-making,2,6,8,15,22,46,50-53 and allow for assessment and suggestions for possible improvements12 and improvement projects.54 Finally, there must be opportunities for education, training, and support,3,6,10,22,26,42,54-66 including training in how to use data effectively.67 Vital to all of this is protected time to participate in these activities.49,68-72

 

No data currently exist with respect to the number of Ontario physicians formally trained or participating in QI. There is a need to better understand facilitators and barriers to involvement as well as perceptions of significance. Thus, this study had three objectives: to develop an instrument that could be used to evaluate physician engagement in QI; to pilot the instrument with a small sample of physicians and physician leaders; and to identify facilitators and barriers to physician engagement in QI. Top

 

Methods

 

Part 1. Survey development

No single tool examined all of the facilitators of engagement identified in the literature. Instruments, such as the Well-being Index73 and Culture of Care Barometer,74 include only select components. Others, such as the Medical Engagement Survey,75 are broken down into other well-established, distinct constructs, such as “empowerment” and “satisfaction,” perhaps contributing to the ambiguity of the term, engagement. Top

 

Response burden was also a concern. Instruments were quite lengthy, even though the literature suggested that fewer questions would suffice. For example, two single-item questions to represent depersonalization (I have become more callous) and emotional exhaustion (I feel burned out) demonstrated results consistent with those based on the 22-item Maslach Burnout Inventory.76

 

As a result, two robust, comprehensive literature reviews were conducted and published.1,77,78 The first was a scoping review to identify factors associated with, and tools used to measure, physician engagement.1,78 The second was a conceptual analysis to study and clarify the term “physician engagement.”77 Based on these exhaustive reviews, five key constructs were identified that enhance physician engagement: well-being, interpersonal relationships, opportunities, work environment, and incentives.1,77 A modified Delphi technique was then used to finalize key areas of focus and corresponding questions.79-81 Top

 

Sample: Convenience sampling was used to recruit participants from Ontario, Canada, for the modified Delphi technique. The panel consisted of senior leaders from the Ontario Hospital Association (2), the Ontario Medical Association (3), Ontario Health (formerly Health Quality Ontario) (2), and faculty at the Dalla Lana School of Public Health (4), two of whom are quality improvement experts. Each of these organizations works closely with, and obtains feedback from, a pool of frontline physicians from a variety of clinical settings.

Data collection: Potential Delphi participants (n = 11) were contacted via email and in person. All agreed to participate. The panel was then sent an email that contained an Excel file (Microsoft, Redmond, Wash., USA) with constructs and sample questions. Participants were asked to rank questions on a Likert scale from one (not at all important) to five (very important) and to suggest additional indicators. Items included in a second round were determined by the first round.82 The questions were then revised and recirculated to the team via email and an Excel spreadsheet. Questions with an average score of less than 3 were removed. Top

 

Questions were then distributed to the PPLC, and feedback was obtained in person at its quarterly meeting. Cognitive debriefing was conducted with this group of physicians and physician leaders to ensure that the questions resonated with them, were actionable, were worded appropriately (e.g., not too negative or abrasive), and that respondent burden was minimized.

 

The survey was constructed using Checkbox 7 (Checkbox 7, Watertown, Mass., USA) online survey platform.

 

Part 2. Pilot study

The study design was cross-sectional. The rationale for a pilot study can be grouped into several broad classifications: process (e.g., assess feasibility of steps required), resources (e.g., assess time and budget), management (e.g., human and data optimization/management), and scientific (e.g., assessment of treatments).83 The purpose of this pilot study was to assess feasibility of the email distribution method, assess the amount of time it takes to complete the survey, and assess data management. Top

 

Sample: Convenience sampling was used to recruit physicians from across Ontario, who were representative of the physician population at which the survey was aimed. An email invitation was sent from the OHA to members of their PPLC to ensure variety in hospital type (i.e., community, small/rural, academic teaching, mental health, and complex continuing care/rehabilitation). Those interested in providing feedback were asked to contact the research team. Respondents were also asked to forward the link to individuals on their medical advisory committee who would complete the survey, critically assess the instrument, and provide feedback. In total, the link was distributed to 49 physicians. Based on the Canadian 2014 National Physician Survey, a 16% response rate was expected.84

 

Data collection: Potential participants were sent an information email containing a link to the online survey. This afforded an inexpensive method that allowed for rapid surveying of a large, geographically distributed sample across the province.85 The survey was administered through Checkbox. Once participants clicked on the link, they were directed to an introduction page, which explicitly stated that by completing and submitting the survey, they were consenting to participate in this study. Top

 

Following the initial invitation, participants were sent two follow-up reminders at 1-week intervals. All questions on the survey were mandatory; thus, participants were required to answer all questions on each page before proceeding to the next page of questions. Once the survey was completed, participants had the opportunity to provide additional free text and general comments.

 

Data: All data were categorical. They were imported from Checkbox into Excel and then directly into SPSS v. 23.

 

Analysis: Descriptive analyses were performed to generate frequency distributions for each variable. Negative survey items were reverse-coded and included as new variables in the data set. Cronbach’s alphas were calculated for each construct to test reliability. In the literature, the ratio of sample size to number of free parameters ranges from as low as five participants per observed variable to 10–20:1.86,87 Top

 

Ethics: Approval was obtained from the Research Ethics Board at the University of Toronto.

 

Results

 

Characteristics of respondents

Of the 49 physicians contacted, 37 completed the survey for a response rate of 75.5%. This sample included 15 specialties from seven sites, with variation in hospital type. To avoid potential identification of participants, details related to hospital type and specialty are not reported. On average, it took five minutes and 43 seconds to complete the survey.

Respondents were 59% (n = 22) male, with 73% (n = 27) born between 1965 and 1995 (Table 1). Over 76% (n = 28) were in formal leadership roles, and 62% (n = 23) had been practising medicine for over 10 years and had been with their organizations longer than five years.  Top

 

Constructs

Well-being: Over 91% (n = 34) of respondents agreed and strongly agreed that they felt they were having a positive impact on people’s lives through their work, and 95% (n = 35) felt the work they do is meaningful to them (Table 2). Five (13%) felt they had become more callous toward people since they started their current job, with nine (24%) unable to decide. Nine (24%) agreed or strongly agreed that they felt burned out, with 13 (35%) unable to decide. Almost 46% (n = 17) felt their schedules afforded them enough time for their personal life and families, and 68% (n = 25) felt their organization had a positive workplace culture. Top

 

Perceptions of senior leadership and co-workers: With 76% of respondents holding formal leadership roles, it was not unexpected to find that over 73% agreed or strongly agreed that they trusted their senior leadership and that their senior leadership listened to their views, took their concerns seriously, supported and respected them (Table 3). However, only 59% (n = 22) agreed or strongly agreed that senior leadership provided constructive feedback. Regarding co-workers, 92% (n = 34) agreed or strongly agreed that they felt respected, and 89% (n = 33) felt their interprofessional teams functioned well together. Top

 

Opportunities and work environment: Just over 80% (n = 30) of respondents agreed or strongly agreed that they have opportunities to be involved in decision-making and opportunities for leadership (Table 4). Almost 90% (n = 33) felt they had opportunities to suggest improvements; however, only 62% (n = 23) felt they had opportunities for training and education.

 

Approximately 65% (n = 24) agreed or strongly agreed that they had the resources they needed to do a good job. Only about 60% (n = 22) felt that unacceptable behaviour was consistently tackled. Over 80% (n = 30) of respondents felt well informed about what was happening in their organization, that two-way communication existed with the organization’s administration, and that there was alignment between their goals and those of the organization. Only 62% (n = 23) agreed or strongly disagreed that they were held accountable for achieving results.

 

Scale reliabilities

All Cronbach’s alphas were greater than 0.7 and were considered acceptable (Table 5).88 Top

 

Incentives

Approximately 84% (n = 31) of respondents reported that their organization did not use any form of incentive to obtain outcomes (Table 6).

 

Quality improvement

Fewer than 14% (n = 5) of respondents were formally trained in QI at their organization (Table 7). Of the five people trained, four received intermediate training (e.g., the application of basic tools in small projects) and the fifth received introductory training (e.g., basic concepts and tools). All five “agreed” that the training received prepared them to participate effectively in QI projects. Top

 

Regardless of training, 57% (n = 21) of respondents had participated in QI projects: 49% (n = 18) at the organization level, 40.5% (n = 15) at the patient level, and only 19% (n = 7) at the system level. Approximately 70% (n = 26) “did not know” or “disagreed” that useful data on their own performance to support QI were available.

 

When asked if their organization made it easy to participate in QI, 68% (n = 25) responded “yes” and identified “provision of organizational support” (n = 17) and “making QI part of their job” (n = 14) as the main facilitators. The remaining 32% (n = 12) that felt their organization did not make it easy to participate and identified “no training offered” (n = 7), “never asked” (n = 6), and “not enough time” (n = 5) as the main barriers. Top

 

Approximately 60% “don’t know” (n = 21) or “disagree” (n = 1) when asked if resources dedicated to QI are producing positive results. Respondents felt that the

QI projects their organization participates in result in services that are safe (n = 25), patient-centred (n = 25), effective (n = 13), efficient (n = 10), timely (n = 9), and equitable (n = 5).

 

Additional questions identified for inclusion

It was suggested that Schaufeli’s nine-item work engagement scale,89 which is valid and reliable, be added to determine the level of overall “work engagement” and to establish a baseline for physicians. It was also suggested that an additional single question be added to determine whether an individual received training in QI external to their organization.

 

Discussion

 

The purpose of this pilot study was to assess the feasibility of the email distribution method, the amount of time it takes to complete the survey, and data management. No concerns with our methods were identified. All participants were able to open the information email and use the link to the survey. Completion time was short, approximate five minutes. Finally, no concerns with our data management were identified; data were easily and securely transferred between Checkbox, Excel, and SPSS software. Top

 

This short survey identifies key facilitators of physician engagement and can quickly highlight opportunities for both senior leadership and policymakers. It is promising that all scale reliabilities were found to be acceptable. This level of psychometric and formative evaluation is not present with other surveys in the engagement literature.1 This is important and one of the reasons that such a rigorous approach to the development of this survey was taken.

 

The literature suggests that a dedicated effort is required by all health care workers to achieve and sustain high performance.90 This instrument helps to identify an opportunity for formal QI training. Only a small percentage of our participants were formally trained in QI at their organization; none received advanced training, an interesting finding considering that over half participated in QI projects. Top

 

This tool helped to reveal that approximately a third of the organizations made it challenging for physicians to participate in QI, the main barriers being no training offered, no formal invitation to participate, and lack of time. Given that Ontario’s Excellent Care For All Act requires hospitals to link executive compensation to the achievement of targets set out in the QI plan,91 it is interesting to see that only a small number of organizations used incentives to drive outcomes within their organizations.

 

In conjunction with participation in QI, feedback11 and assigned accountability have also been identified as important.3,4 Feedback related to clinical performance is critical to QI.92 This instrument helped to show that a large proportion of respondents were unaware or confirmed that they did not receive useful data on their own performance to support QI. Many respondents reported a lack of constructive feedback, which may relate to over a third of respondents undecided with respect to whether they were held accountable for achieving results. In addition, many respondents, almost two thirds, did not know whether resources dedicated to QI were producing positive results. Top

 

Using Health Quality Ontario’s six defining elements of quality care,93 our survey helped to show that there may be opportunities for greater promotion of project results and additional QI projects focused on equitable, timely, efficient, and/or effective services. To create a high-performing health care system, a system-wide perspective is needed.94 This instrument helps to identify a potential need for, or lack of, system-level QI projects. This is the first time this type of data has been captured and examined in Ontario. Results clearly indicated that just over half of our sample group participated in QI projects, of which the majority were at the organization and patient levels, with only a few at the system level. Top

 

Finally, the Canadian Medical Association recently released a report that one in four Canadian physicians report burnout.95 Based on a single question, our survey found that in this small Ontario sample, approximately one in four respondents expressed burnout, supporting the use of single-item questions when possible to reduce respondent burden.76

 

This work has the potential to create opportunities for future research that can substantiate or refute common organizational theories about motivation, culture, and performance in relation to physicians. By collecting accurate, valid, and reliable longitudinal data, we can move beyond the simple association of variables and start identifying causation, which could help health care leaders make evidence-informed decisions and focus resources in areas proven to have the greatest impact.  Top

 

Limitations

 

Our survey population was small and made up, predominantly, of individuals in hospital leadership roles. However, the purpose was not to generalize results, but to develop and test an instrument that could be used by health care leadership in Ontario to quickly evaluate key areas, suggested in the literature to impact engagement in QI within their organizations.

 

Conclusion

 

A short, easy to administer survey was developed to help Ontario hospital leaders obtain baseline data on facilitators of physician engagement, participation, and training in QI. This instrument was able to help leaders quickly evaluate key actionable areas linked to physician engagement. A larger sample is warranted for accurate validity and reliability testing. This tool could prove extremely valuable in enhancing physician engagement in QI initiatives. Top

 

References

1.Perreira TA, Perrier L, Prokopy M. Hospital physician engagement: a scoping review. Med Care 2018;56(12):969-75. DOI: 10.1097/MLR.0000000000000983

2.Lepore SJ, Nair RG, Davis SN, Wolf RL, Basch CE, Thomas N, et al. Patient and physician factors associated with undisclosed prostate cancer screening in a sample of predominantly immigrant black men. J Immigr Minor Health 2017;19(6):1343-50. DOI: 10.1007/s10903-016-0468-1

3.Calayag J. Physician engagement: strengthening the culture of quality and safety. Healthc Exec 2014;29(2):28-30.

4.Erlandson E, Ludeman K. Physician engagement and shared accountability. Buzzwords, dilemma or choice? Mich Health Hosp 2003;39(6):28-9.

5.Spaulding A, Gamm L, Menser T. Physician engagement: strategic considerations among leaders at a major health system. Hosp Top 2014;92(3):66-73.

6.Stark R. Increasing physician engagement: start with what’s important to physicians. J Med Pract Manage 2014;30(3):171-5.

7.Studer Q, Hagins Jr M, Cochrane BS. The power of engagement: creating the culture that gets your staff aligned and invested. Healthc Manage Forum 2014;27(1 Suppl):S79-97. DOI: 10.1016/j.hcmf.2014.01.008

8.Whitlock DJ, Stark R. Understanding physician engagement — and how to increase it. Physician Leadersh J 2014;1(1):8-12.

9.Buell JM. Achieving financial success through improved physician engagement: revenue enhancements can be realized with stronger relationships. Healthc Exec 2009;24(1):22-4, 26, 28-9.

10.Daly R. Putting physicians in the lead for cost containment. Healthc Financ Manage 2013;67(12):52-9.

11.Garvin D, Worthington J, McGuire S, Burgetz S, Forster AJ, Patey, et al. Physician performance feedback in a Canadian academic center. Leadersh Health Serv 2017;30(4):457-74. DOI: 10.1108/LHS-08-2016-0037

12.George AE, Frush K, Michener JL. Developing physicians as catalysts for change. Acad Med 2013;88(11):1603-5.

13.Grace SM, Rich J, Chin W, Rodriguez HP. Flexible implementation and integration of new team members to support patient-centered care. Healthc (Amst) 2014;2(2):145-51. DOI: 10.1016/j.hjdsi.2014.02.003 Top

14.Jarousse L. Physician engagement (getting your docs on board!). Hosp Health Netw 2014;88(4):41-7.

15.Kaissi A. Enhancing physician engagement: an international perspective. Int J Health Serv 2014;44(3):567-92. DOI: 10.2190/HS.44.3.h

16.Kim Y, Winner M, Page A, Tisnado DM, Martinez KA, Buettner S, et al. Patient perceptions regarding the likelihood of cure after surgical resection of lung and colorectal cancer. Cancer 2015;121(20):3564-73. DOI: 10.1002/cncr.29530

17.Friedman EL, Chawla N, Morris PT, Castro KM, Carrigan AC, Prabhu Das I, et al. Assessing the development of multidisciplinary care: experience of the National Cancer Institute Community Cancer Centers Program. J Oncol Pract 2015;11(1):e36-43. DOI: 10.1200/JOP.2014.001535

18.Kreindler SA, Larson BK, Wu FM, Gbemudu JN, Carluzzo KL, Struthers A, et al. The rules of engagement: physician engagement strategies in intergroup contexts. J Health Organ Manag 2014;28(1):41-61. DOI: 10.1108/JHOM-02-2013-0024

19.Lee VS, Miller T, Daniels C, Paine M, Gresh B, Betz AL. Creating the exceptional patient experience in one academic health system. Acad Med 2016;91(3):338-44. DOI: 10.1097/ACM.0000000000001007

20.Manary M, Staelin R, Kosel K, Schulman KA, Glickman SW. Organizational characteristics and patient experiences with hospital care: a survey study of hospital chief patient experience officers. Am J Med Qual 2015;30(5):432-40. DOI: 10.1177/1062860614539994

21.Pantaleoni JL, Stevens LA, Mailes ES, Goad BA, Longhurst CA. Successful physician training program for large scale EMR implementation. Appl Clin Inform 2015;6(1):80-95. DOI: 10.4338/ACI-2014-09-CR-0076

22.Pariser P, Pus L, Stanaitis I, Abrams H, Ivers N, Baker GR, et al. Improving system integration: the art and science of engaging small community practices in health system innovation. Int J Fam Med 2016;2016:5926303. DOI: 10.1155/2016/5926303

23.Puri AK, Bhaloo T, Kirshin T, Mithani A. A comprehensive approach to effectively engage physicians during a hospital closure: using the physician engagement model. Healthc Manage Forum 2006;19(4):34-9. DOI: 10.1016/S0840-4704(10)60244-X

24.Rangachari P. Role of social knowledge networking technology in facilitating meaningful use of electronic health record medication reconciliation. J Hosp Admin 2016;5(3):98-106. DOI: 10.5430/jha.v5n3p98

25.Ricottone M. Reducing alternate level of care days at Winchester District Memorial Hospital. Healthc Manage Forum 2015;28(5):190-4. DOI: 10.1177/0840470415588668

26.Rosenstein AH. Strategies to enhance physician engagement. J Med Pract Manage 2015;31(2):113-6.

27.Sondheim SE, Patel DM, Chin N, Barwis K, Werner J, Barclay A, et al. Governance practices in an era of healthcare transformation: achieving a successful turnaround. J Healthc Manage 2017;62(5):316-26. DOI: 10.1097/JHM-D-15-00036

28.Townsend CS, McNulty M, Grillo-Peck A. Implementing huddles improves care coordination in an academic health center. Prof Case Manag 2017;22(1):29-35. DOI: 10.1097/NCM.0000000000000200

29.Wei AC, Sandhu L, Devitt KS, Gagliardi AR, Kennedy ED, Urbach DR, et al., Practice patterns for the management of hepatic metastases from colorectal cancer: a mixed methods analysis. Ann Surg Oncol 2013;20(5):1567-74. DOI: 10.1245/s10434-012-2698-3

30.Akosa AN. Physician engagement is critical to the success of any accountable care organization. J Manage Care Med 2013;16(3):67-76.

31.Klugman R, Gitkind MJ, Walsh KE. The physician quality officer model: 5-year follow-up. Am J Med Qual, 2015;30(5):454-8. DOI: 10.1177/1062860614536221

32.Skillman M, Cross-Barnet C, Singer RF, Ruiz S, Rotondo C, Ahn R, et al., Physician engagement strategies in care coordination: findings from the Centers for Medicare & Medicaid Services’ Health Care Innovation Awards Program. Health Serv Res 2017;52(1):291-312. DOI: 10.1111/1475-6773.12622

33.Lee TH. Financial versus non-financial incentives for improving patient experience. J Patient Exp 2015;2(1):4-6. DOI: 10.1177/237437431500200102

34.Bunkers B, Koch M, McDonough B, Whited B. Aligning physician compensation with strategic goals. Healthc Financ Manage 2014;68(7):38-45.

35.Walsh KE, Ettinger WH, Klugman RA. Physician quality officer: a new model for engaging physicians in quality improvement. Am J Med Qual 2009;24(4):295-301. DOI: 10.1177/1062860609336219

36.Rinne ST, Rinne TJ, Olsen K, Wiener RS, Balcezak TJ, Dardani W, et al. Hospital administrators’ perspectives on physician engagement: a qualitative study. J Hosp Med 2018;13(3):179-81. DOI: 10.12788/jhm.2880 Top

37.Rice JA, Sagin T. New conversations for physician engagement. Five design principles to upgrade your governance model. Healthc Exec 2010;25(4):66-70.

38.Scott CG, Thériault A, McGuire S, Samson A, Clement C, Worthington JR. Developing a physician engagement agreement at The Ottawa Hospital: a collaborative approach. Healthc Q 2012;15(3):50-3. DOI: 10.12927/hcq.2013.23020

39.Weiss R. The quest for physician engagement: Physician relationships are crucial in today’s changing environment. Marketing Healthc Serv 2011;31(2):29-31.

40.Denis JL, van Gestel N. Medical doctors in healthcare leadership: theoretical and practical challenges. BMC Health Serv Res 2016;16(Suppl 2):158. DOI: 10.1186/s12913-016-1392-8 Top

41.Gosfield AG, Reinertsen JL. Finding common cause in quality: confronting the physician engagement challenge. Physician Exec 2008;34(2):26-31.

42.Marsden J, van Dijk M, Doris P, Krause DP, Cochrane D. Improving care for British Columbians: the critical role of physician engagement. Healthc Q 2012;15(Spec no.):51-5. DOI: 10.12927/hcq.2012.23163

43.Strasser DC, Burridge AB, Falconer JA, Uomoto JM, Herrin J. Toward spanning the quality chasm: an examination of team functioning measures. Arch Phys Med Rehabil 2014;95(11):2220-3. DOI: 10.1016/j.apmr.2014.06.013

44.Swensen S, Kabcenell A, Shanafelt T. Physician-organization collaboration reduces physician burnout and promotes engagement: the Mayo Clinic experience. J Healthc Manag 2016;61(2):105-27.

45.Ahnfeldt-Mollerup P, dePont Christensen R, Halling A, Kritensen T, Lykkegaard J, Nexøe J, et al. Medical engagement and organizational characteristics in general practice. Fam Pract 2015;33(1):69-74. DOI: 10.1093/fampra/cmv085

46.Cox ED, Nackers KA, Young HN, Moreno MA, Levy JF, Mangione-Smith RM. Influence of race and socioeconomic status on engagement in pediatric primary care. Patient Educ Couns 2012;87(3):319-26. DOI: 10.1016/j.pec.2011.09.012

47.Duberman T, Hemker B, Solomon L. Innovative strategies for physician partnerships. One health system shares the steps it took to strengthen physician engagement. Healthc Exec 2015;30(3):84-6.

48.Kraft S. Changes in acquisition patterns. Health systems rely on due diligence, physician engagement to ensure successful alignment. MGMA Connex 2015;15(2):40-3.

49.Engelman D, Benjamin EM. Physician engagement: the “secret sauce” to success in bundled health care. Am J Med Quality 2018;33(1):100-2. DOI: 10.1177/1062860617703730

50.Patty B, Svendsen CA. A proven approach to physician engagement. Physician Exec 2011;37(4):92-93.

51.Brémault-Phillips SC, Parmar J, Friesen S, Rogers LG, Pike A, Sluggett B. An evaluation of the decision-making capacity assessment model. Can Geriatr J 2016;19(3):83-96. DOI: 10.5770/cgj.19.222

52.Leone FT, Evers-Casey S, Graden S, Schnoll R. Behavioral economic insights into physician tobacco treatment decision-making. Ann Am Thorac Soc 2015;12(3):364-9. DOI: 10.1513/AnnalsATS.201410-467BC

53.Engaging clinicians in the new NHS. London: NHS Alliance; 2003.

54.Donaghy G, McKeever K, Flanagan C, O’Kane D, McQuillan B, Cash J, et al. Helping doctors in training to STEP-UP: a leadership and quality improvement programme in the Belfast Health and Social Care Trust. Ulster Med J 2018;87(2):112-6.

55.Hettema JE, Sorensen JL, Uy M, Jain S. Motivational enhancement therapy to increase resident physician engagement in substance abuse education. Subst Abus 2009;30(3):244-7. DOI: 10.1080/08897070903041210

56.Patterson CJ. Best practices in specialty pharmacy management. J Manag Care Pharm 2013;19(1):42-8. DOI: 10.18553/jmcp.2013.19.1.42

57.Gray D, Nussle R, Cruz A, Kane G, Toomey M, Bay C, et al. Effects of a catheter-associated urinary tract infection prevention campaign on infection rate, catheter utilization, and health care workers’ perspective at a community safety net hospital. Am J Infect Control 2016;44(1):115-6. DOI: 10.1016/j.ajic.2015.08.011

58.Wilson KC, Merli GJ. Performance measures for improving the prevention of venous thromboembolism: achievement in clinical practice. J Thromb Thrombolysis 2011;32(3):293-302. DOI: 10.1007/s11239-011-0605-6

59.Hockey PM, Bates DW. Physicians’ identification of factors associated with quality in high- and low-performing hospitals. Jt Comm J Qual Patient Saf 2010;36(5):217-23. DOI: 10.1016/s1553-7250(10)36035-1 Top

60.Krein SL, Kowlaski CP, Harrod M, Forman J, Saint S. Barriers to reducing urinary catheter use: a qualitative assessment of a statewide initiative. JAMA Intern Med 2013;173(10):881-6. DOI: 10.1001/jamainternmed.2013.105

61.McFadyen C, Lankshear S, Divaris D, Berry M, Hunter A, Srigley J, et al. Physician level reporting of surgical and pathology performance indicators: a regional study to assess feasibility and impact on quality. Can J Surg 2015;58(1):31-40. DOI: 10.1503/cjs.004314

62.Tuttle JC. Cutting CAUTIs in critical care. J Clin Outcomes Manag 2017;24(6):267-72.

63.Wynn JD, Draffin E, Jones A, Reida L. The Vidant Health quality transformation. Jt Comm J Qual Patient Saf 2014;40(5):212-8. DOI: 10.1016/s1553-7250(14)40028-x

64.Nordstrom BR, Saunders EC, McLeman B, Meier A, Xie H, Lambert-Harris C, et al. Using a learning collaborative strategy with office-based practices to increase access and improve quality of care for patients with opioid use disorders. J Addict Med 2016;10(2):117-23. DOI: 10.1097/ADM.0000000000000200

65.Sears NJ. Managing margins through physician engagement. Healthc Financ Manage 2012;66(7):44-7.

66.Snell AJ, Briscoe D, Dickson G. From the inside out: the engagement of physicians as leaders in health care settings. Qual Health Res 2011;21(7):952-67. DOI: 10.1177/1049732311399780

67.Croft GP. Engaging clinicians in improving data quality in the NHS. Swansea, UK: Centre for Health Information, University of Wales Swansea; 2006.

68.Greysen SR, Detsky AS. Solving the puzzle of posthospital recovery: what is the role of the individual physician? J Hosp Med 2015;10(10):697-700. DOI: 10.1002/jhm.2421

69.Law TJ, Leistikow N, Hoofring L, Krumm S, Neufeld K, Needham D. A survey of nurses’ perceptions of the intensive care delirium screening checklist. Dynamics (Pembroke, Ont.) 2012;23(4):18-24.

70.Silver SA, Harel Z, McQuillan R, Weizman AV, Thomas A, Chertow GM, et al. How to begin a quality improvement project. Clin J Am Soc Nephrol 2016;11(5):893-900. DOI: 10.2215/CJN.11491015

71.Taitz JM, Lee TH, Sequist TD. A framework for engaging physicians in quality and safety. BMJ Qual Saf 2012;21(9):722-8. DOI: 10.1136/bmjqs-2011-000167

72.Uy V, May SG, Tietbohl C, Frosch DL. Barriers and facilitators to routine distribution of patient decision support interventions: a preliminary study in community-based primary care settings. Health Expect 2014;17(3):353-64. DOI: 10.1111/j.1369-7625.2011.00760.x

73.Dyrbye, L.N.,Cross-Barnet C, Singer RF, Ruiz S, Rotondo C, Ahn R, et al., Development and preliminary psychometric properties of a well-being index for medical students. BMC Med Educ , 2010;10(1):8.

74.Rafferty AM, Philippou J, Fitzpatrick JM, Pike G, Ball J. Development and testing of the ‘Culture of Care Barometer’(CoCB) in healthcare organisations: a mixed methods study. BMJ Open 2017;7(8):e016677. DOI: 10.1136/bmjopen-2017-016677

75.Spurgeon P, Barwell F, Mazelan P. Developing a medical engagement scale (MES). Int J Clin Leadersh 2008;16(4):213-23.

76.West CP, Dyrbye LN, Satele DV, Sloan JA, Shanafelt TD. Concurrent validity of single-item measures of emotional exhaustion and depersonalization in burnout assessment. J Gen Intern Med 2012;27(11):1445-52. DOI: 10.1007/s11606-012-2015-7 Top

77.Perreira TA, Perrier L, Prokopy M, Neves-Mera L, Persaud DD. Physician engagement: a concept analysis. J Healthc Leadersh 2019;11:101-13. DOI: 10.2147/JHL.S214765

78.Perreira T, Perrier L, Prokopy M, Jonker A. Physician engagement in hospitals: a scoping review protocol. BMJ Open 2018;8(1):e018837. DOI: 10.1136/bmjopen-2017-018837

79.Fink A, Kosecoff J, Chassin M, Brook RH. Consensus methods: characteristics and guidelines for use. Am J Public Health 1984;74(9):979-83. DOI: 10.2105/ajph.74.9.979

80.Mullen PM. Delphi: myths and reality. J Health Organ Manag 2003;17(1):37-52. DOI: 10.1108/14777260310469319

81.Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PloS One 2011;6(6):e20476. DOI: 10.1371/journal.pone.0020476

82.Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs 2000;32(4):1008-15.

83.Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol 2010;10(1):1. DOI: 10.1186/1471-2288-10-1

84.2014 response rates. In National Physician Survey. Mississauga, Ont.: National Physician Survey; 2019. Available: https://tinyurl.com/uzhqm94

85.Dillman DA, Smyth JD, Christian LM. Internet, phone, mail, and mixed-mode surveys: the tailored design method. Hoboken, N.J.: John Wiley & Sons; 2014.

86.Chou CP, Bentler PM. Estimates and tests in structural equation modeling. In Hoyle RH (editor). Structural equation modeling: concepts, issues, and applications. Thousand Oaks, Calif.: Sage; 1995:37-55.

87.Weston R, Gore Jr PA, Chan F, Catalano D. An introduction to using structural equation models in rehabilitation psychology. Rehabil Psychol 2008;53(3):340. DOI: 10.1037/a0013039

88.Field A. Discovering statistics using IBM SPSS statistics (4th edition). Thousand Oaks, Calif.: Sage; 2013.

89.Schaufeli WB, Salanova M, González-Romá V, Bakker AB. The measurement of engagement and burnout: A two sample confirmatory factor analytic approach. J Happiness Stud 2002;3(1):71-92.

90.Brown A, Baker GR, Closson T, Sullivan T. The journey toward high performance and excellent quality. Healthc Q 2012;15(sp):6-9. DOI: 10.12927/hcq.2012.23152

91.About the Excellent Care for All Act. Toronto: Ministry of Health and Long-Term Care; 2019. Available: https://tinyurl.com/mg9pogs

92.Kaye AD, Okanlawon OJ, Urman RD. Clinical performance feedback and quality improvement opportunities for perioperative physicians. Adv Med Educ Pract 2014;5:115-23. DOI: 10.2147/AMEP.S62165

93.Quality matters: realizing excellent care for all. Toronto: Health Quality Ontario; 2015. Available: https://tinyurl.com/tloqrp6

94.Baker GR, Axler R. Creating a high performing healthcare system for Ontario: evidence supporting strategic changes in Ontario. Ottawa: Ontario Hospital Association; 2015. Available: https://tinyurl.com/ubtd4s3

95.CMA National Physician Health Survey: a national snapshot. Ottawa: Canadian Medical Association; 2018. Available: https://tinyurl.com/ubtd4s3

 

Acknowledgements

We thank Elizabeth Carlton, Dara Laxer, Lee Fairclough, Monique Herbert, Gillian Elliott, and Ross Baker for their support and contribution to this project. Each contributed to the research design and development of the survey instrument. We also thank the Ontario Hospital Association’s Physician Provincial Leadership Council and those physicians who participated in this pilot study. Top

 

Authors

Tyrone Perreira, PhD, MEd, is an assistant professor at the University of Toronto’s Institute of Health Policy Management and Evaluation, Dalla Lana School of Public Health and a research scientist at the Ontario Hospital Association.

 

Melissa Prokopy, LLB, is director of Legal, Policy and Professional Issues at the Ontario Hospital Association and adjunct faculty at the University of Toronto’s Institute for Health Policy, Management and Evaluation.

 

Adalsteinn Brown, DPhil, AB, is dean of the Dalla Lana School of Public Health at the University of Toronto.

 

Anna Greenberg, MPP, is president of Ontario Health’s business unit focused on quality.

 

James Wright, MD, MPH, is chief, Economics, Policy and Research at the Ontario Medical Association.

 

Christine Shea, PhD, MEd, is program director of Quality Improvement and Patient Safety at the University of Toronto’s Institute of Health Policy Management and Evaluation, Dalla Lana School of Public Health.

 

Julie Simard is a doctoral student at the University of Toronto’s Institute of Health Policy Management and Evaluation, Dalla Lana School of Public Health.

 

Author attestation

In addition to participating in the research design and development of the survey instrument, all authors contributed significantly to article preparation. Tyrone Perreira and Melissa Prokopy conceptualized the article. Adalsteinn Brown, Anna Greenberg, James Wright, Christine Shea, and Julie Simard assisted with organization and revisions of the article. All authors approved the final version.

 

Conflict of interest: The authors declare no conflict of interest.

 

Correspondence to:

ty.perreira@utoronto.ca

 

This article has been peer reviewed.

 

Top

Measuring physician engagement in quality improvement: a pilot study

Tyrone Perreira, PhD, MEd, Melissa Prokopy, LLB, Adalsteinn Brown, DPhil, Anna Greenberg, MPP, James Wright, MD, MPH, Christine Shea, PhD, MEd, and Julie Simard, PhD

https//doi.org/10.37964/cr24717

 

The term “physician engagement” is overused and often misunderstood. It is believed that system transformation requires physician engagement in quality improvement (QI); however, no tool exists to accurately measure this. The purpose of this study was to develop an instrument that could be used to evaluate physician engagement in QI and then pilot it with a small sample of physicians and physician leaders. An electronic survey was developed using a series of focused literature searches and a modified Delphi panel of QI experts. Cognitive debriefing was performed with a group of physicians and physician leaders. The survey was then administered to 37 physicians working in Ontario hospitals. Descriptive analyses were carried out. This short, easy to administer survey allows for the collection of baseline data on facilitators of physician engagement, as well as training and participation in QI. Construct reliabilities are promising with Cronbach’s alphas between 0.75 and 0.96.

 

KEY WORDS:  physician engagement, quality improvement, hospitals, evaluation, survey

 

Despite the notion that physician engagement in quality improvement (QI) is critical to health system transformation, the concept remains poorly defined, measured, and researched.1 The Ontario Hospital Association’s (OHA’s) Physician Provincial Leadership Council (PPLC), which comprises senior physician leaders from across the province, identified an important need for additional work in this area and supporting evidence identifying facilitators of physician engagement.1 Top

 

The health care literature suggests that the following antecedents are necessary to enhance physician engagement: accountability2-8; communication2,5-29; incentives30 (both financial3,31,32 and non-financial33); and good interpersonal relations between physicians and administrators, with alignment of goals,34-37 values,7,37-40 and beliefs.41 There must be trust14,15,27,40,41 and respect,6,8,38 such that physicians feel supported by their organizations.2,42 The work environment must promote teamwork,14,17,38,43-45 relationship building,46,47 and the development of strategic partnerships,48 whether intergroup18 or peer.47 The environment must provide opportunities to partake in and be involved in leadership8,9,47,49 and decision-making,2,6,8,15,22,46,50-53 and allow for assessment and suggestions for possible improvements12 and improvement projects.54 Finally, there must be opportunities for education, training, and support,3,6,10,22,26,42,54-66 including training in how to use data effectively.67 Vital to all of this is protected time to participate in these activities.49,68-72

 

No data currently exist with respect to the number of Ontario physicians formally trained or participating in QI. There is a need to better understand facilitators and barriers to involvement as well as perceptions of significance. Thus, this study had three objectives: to develop an instrument that could be used to evaluate physician engagement in QI; to pilot the instrument with a small sample of physicians and physician leaders; and to identify facilitators and barriers to physician engagement in QI. Top

 

Methods

 

Part 1. Survey development

No single tool examined all of the facilitators of engagement identified in the literature. Instruments, such as the Well-being Index73 and Culture of Care Barometer,74 include only select components. Others, such as the Medical Engagement Survey,75 are broken down into other well-established, distinct constructs, such as “empowerment” and “satisfaction,” perhaps contributing to the ambiguity of the term, engagement. Top

 

Response burden was also a concern. Instruments were quite lengthy, even though the literature suggested that fewer questions would suffice. For example, two single-item questions to represent depersonalization (I have become more callous) and emotional exhaustion (I feel burned out) demonstrated results consistent with those based on the 22-item Maslach Burnout Inventory.76

 

As a result, two robust, comprehensive literature reviews were conducted and published.1,77,78 The first was a scoping review to identify factors associated with, and tools used to measure, physician engagement.1,78 The second was a conceptual analysis to study and clarify the term “physician engagement.”77 Based on these exhaustive reviews, five key constructs were identified that enhance physician engagement: well-being, interpersonal relationships, opportunities, work environment, and incentives.1,77 A modified Delphi technique was then used to finalize key areas of focus and corresponding questions.79-81 Top

 

Sample: Convenience sampling was used to recruit participants from Ontario, Canada, for the modified Delphi technique. The panel consisted of senior leaders from the Ontario Hospital Association (2), the Ontario Medical Association (3), Ontario Health (formerly Health Quality Ontario) (2), and faculty at the Dalla Lana School of Public Health (4), two of whom are quality improvement experts. Each of these organizations works closely with, and obtains feedback from, a pool of frontline physicians from a variety of clinical settings.

Data collection: Potential Delphi participants (n = 11) were contacted via email and in person. All agreed to participate. The panel was then sent an email that contained an Excel file (Microsoft, Redmond, Wash., USA) with constructs and sample questions. Participants were asked to rank questions on a Likert scale from one (not at all important) to five (very important) and to suggest additional indicators. Items included in a second round were determined by the first round.82 The questions were then revised and recirculated to the team via email and an Excel spreadsheet. Questions with an average score of less than 3 were removed. Top

 

Questions were then distributed to the PPLC, and feedback was obtained in person at its quarterly meeting. Cognitive debriefing was conducted with this group of physicians and physician leaders to ensure that the questions resonated with them, were actionable, were worded appropriately (e.g., not too negative or abrasive), and that respondent burden was minimized.

 

The survey was constructed using Checkbox 7 (Checkbox 7, Watertown, Mass., USA) online survey platform.

 

Part 2. Pilot study

The study design was cross-sectional. The rationale for a pilot study can be grouped into several broad classifications: process (e.g., assess feasibility of steps required), resources (e.g., assess time and budget), management (e.g., human and data optimization/management), and scientific (e.g., assessment of treatments).83 The purpose of this pilot study was to assess feasibility of the email distribution method, assess the amount of time it takes to complete the survey, and assess data management. Top

 

Sample: Convenience sampling was used to recruit physicians from across Ontario, who were representative of the physician population at which the survey was aimed. An email invitation was sent from the OHA to members of their PPLC to ensure variety in hospital type (i.e., community, small/rural, academic teaching, mental health, and complex continuing care/rehabilitation). Those interested in providing feedback were asked to contact the research team. Respondents were also asked to forward the link to individuals on their medical advisory committee who would complete the survey, critically assess the instrument, and provide feedback. In total, the link was distributed to 49 physicians. Based on the Canadian 2014 National Physician Survey, a 16% response rate was expected.84

 

Data collection: Potential participants were sent an information email containing a link to the online survey. This afforded an inexpensive method that allowed for rapid surveying of a large, geographically distributed sample across the province.85 The survey was administered through Checkbox. Once participants clicked on the link, they were directed to an introduction page, which explicitly stated that by completing and submitting the survey, they were consenting to participate in this study. Top

 

Following the initial invitation, participants were sent two follow-up reminders at 1-week intervals. All questions on the survey were mandatory; thus, participants were required to answer all questions on each page before proceeding to the next page of questions. Once the survey was completed, participants had the opportunity to provide additional free text and general comments.

 

Data: All data were categorical. They were imported from Checkbox into Excel and then directly into SPSS v. 23.

 

Analysis: Descriptive analyses were performed to generate frequency distributions for each variable. Negative survey items were reverse-coded and included as new variables in the data set. Cronbach’s alphas were calculated for each construct to test reliability. In the literature, the ratio of sample size to number of free parameters ranges from as low as five participants per observed variable to 10–20:1.86,87 Top

 

Ethics: Approval was obtained from the Research Ethics Board at the University of Toronto.

 

Results

 

Characteristics of respondents

Of the 49 physicians contacted, 37 completed the survey for a response rate of 75.5%. This sample included 15 specialties from seven sites, with variation in hospital type. To avoid potential identification of participants, details related to hospital type and specialty are not reported. On average, it took five minutes and 43 seconds to complete the survey.

Respondents were 59% (n = 22) male, with 73% (n = 27) born between 1965 and 1995 (Table 1). Over 76% (n = 28) were in formal leadership roles, and 62% (n = 23) had been practising medicine for over 10 years and had been with their organizations longer than five years.  Top

 

Constructs

Well-being: Over 91% (n = 34) of respondents agreed and strongly agreed that they felt they were having a positive impact on people’s lives through their work, and 95% (n = 35) felt the work they do is meaningful to them (Table 2). Five (13%) felt they had become more callous toward people since they started their current job, with nine (24%) unable to decide. Nine (24%) agreed or strongly agreed that they felt burned out, with 13 (35%) unable to decide. Almost 46% (n = 17) felt their schedules afforded them enough time for their personal life and families, and 68% (n = 25) felt their organization had a positive workplace culture. Top

 

Perceptions of senior leadership and co-workers: With 76% of respondents holding formal leadership roles, it was not unexpected to find that over 73% agreed or strongly agreed that they trusted their senior leadership and that their senior leadership listened to their views, took their concerns seriously, supported and respected them (Table 3). However, only 59% (n = 22) agreed or strongly agreed that senior leadership provided constructive feedback. Regarding co-workers, 92% (n = 34) agreed or strongly agreed that they felt respected, and 89% (n = 33) felt their interprofessional teams functioned well together. Top

 

Opportunities and work environment: Just over 80% (n = 30) of respondents agreed or strongly agreed that they have opportunities to be involved in decision-making and opportunities for leadership (Table 4). Almost 90% (n = 33) felt they had opportunities to suggest improvements; however, only 62% (n = 23) felt they had opportunities for training and education.

 

Approximately 65% (n = 24) agreed or strongly agreed that they had the resources they needed to do a good job. Only about 60% (n = 22) felt that unacceptable behaviour was consistently tackled. Over 80% (n = 30) of respondents felt well informed about what was happening in their organization, that two-way communication existed with the organization’s administration, and that there was alignment between their goals and those of the organization. Only 62% (n = 23) agreed or strongly disagreed that they were held accountable for achieving results.

 

Scale reliabilities

All Cronbach’s alphas were greater than 0.7 and were considered acceptable (Table 5).88 Top

 

Incentives

Approximately 84% (n = 31) of respondents reported that their organization did not use any form of incentive to obtain outcomes (Table 6).

 

Quality improvement

Fewer than 14% (n = 5) of respondents were formally trained in QI at their organization (Table 7). Of the five people trained, four received intermediate training (e.g., the application of basic tools in small projects) and the fifth received introductory training (e.g., basic concepts and tools). All five “agreed” that the training received prepared them to participate effectively in QI projects. Top

 

Regardless of training, 57% (n = 21) of respondents had participated in QI projects: 49% (n = 18) at the organization level, 40.5% (n = 15) at the patient level, and only 19% (n = 7) at the system level. Approximately 70% (n = 26) “did not know” or “disagreed” that useful data on their own performance to support QI were available.

 

When asked if their organization made it easy to participate in QI, 68% (n = 25) responded “yes” and identified “provision of organizational support” (n = 17) and “making QI part of their job” (n = 14) as the main facilitators. The remaining 32% (n = 12) that felt their organization did not make it easy to participate and identified “no training offered” (n = 7), “never asked” (n = 6), and “not enough time” (n = 5) as the main barriers. Top

 

Approximately 60% “don’t know” (n = 21) or “disagree” (n = 1) when asked if resources dedicated to QI are producing positive results. Respondents felt that the

QI projects their organization participates in result in services that are safe (n = 25), patient-centred (n = 25), effective (n = 13), efficient (n = 10), timely (n = 9), and equitable (n = 5).

 

Additional questions identified for inclusion

It was suggested that Schaufeli’s nine-item work engagement scale,89 which is valid and reliable, be added to determine the level of overall “work engagement” and to establish a baseline for physicians. It was also suggested that an additional single question be added to determine whether an individual received training in QI external to their organization.

 

Discussion

 

The purpose of this pilot study was to assess the feasibility of the email distribution method, the amount of time it takes to complete the survey, and data management. No concerns with our methods were identified. All participants were able to open the information email and use the link to the survey. Completion time was short, approximate five minutes. Finally, no concerns with our data management were identified; data were easily and securely transferred between Checkbox, Excel, and SPSS software. Top

 

This short survey identifies key facilitators of physician engagement and can quickly highlight opportunities for both senior leadership and policymakers. It is promising that all scale reliabilities were found to be acceptable. This level of psychometric and formative evaluation is not present with other surveys in the engagement literature.1 This is important and one of the reasons that such a rigorous approach to the development of this survey was taken.

 

The literature suggests that a dedicated effort is required by all health care workers to achieve and sustain high performance.90 This instrument helps to identify an opportunity for formal QI training. Only a small percentage of our participants were formally trained in QI at their organization; none received advanced training, an interesting finding considering that over half participated in QI projects. Top

 

This tool helped to reveal that approximately a third of the organizations made it challenging for physicians to participate in QI, the main barriers being no training offered, no formal invitation to participate, and lack of time. Given that Ontario’s Excellent Care For All Act requires hospitals to link executive compensation to the achievement of targets set out in the QI plan,91 it is interesting to see that only a small number of organizations used incentives to drive outcomes within their organizations.

 

In conjunction with participation in QI, feedback11 and assigned accountability have also been identified as important.3,4 Feedback related to clinical performance is critical to QI.92 This instrument helped to show that a large proportion of respondents were unaware or confirmed that they did not receive useful data on their own performance to support QI. Many respondents reported a lack of constructive feedback, which may relate to over a third of respondents undecided with respect to whether they were held accountable for achieving results. In addition, many respondents, almost two thirds, did not know whether resources dedicated to QI were producing positive results. Top

 

Using Health Quality Ontario’s six defining elements of quality care,93 our survey helped to show that there may be opportunities for greater promotion of project results and additional QI projects focused on equitable, timely, efficient, and/or effective services. To create a high-performing health care system, a system-wide perspective is needed.94 This instrument helps to identify a potential need for, or lack of, system-level QI projects. This is the first time this type of data has been captured and examined in Ontario. Results clearly indicated that just over half of our sample group participated in QI projects, of which the majority were at the organization and patient levels, with only a few at the system level. Top

 

Finally, the Canadian Medical Association recently released a report that one in four Canadian physicians report burnout.95 Based on a single question, our survey found that in this small Ontario sample, approximately one in four respondents expressed burnout, supporting the use of single-item questions when possible to reduce respondent burden.76

 

This work has the potential to create opportunities for future research that can substantiate or refute common organizational theories about motivation, culture, and performance in relation to physicians. By collecting accurate, valid, and reliable longitudinal data, we can move beyond the simple association of variables and start identifying causation, which could help health care leaders make evidence-informed decisions and focus resources in areas proven to have the greatest impact.  Top

 

Limitations

 

Our survey population was small and made up, predominantly, of individuals in hospital leadership roles. However, the purpose was not to generalize results, but to develop and test an instrument that could be used by health care leadership in Ontario to quickly evaluate key areas, suggested in the literature to impact engagement in QI within their organizations.

 

Conclusion

 

A short, easy to administer survey was developed to help Ontario hospital leaders obtain baseline data on facilitators of physician engagement, participation, and training in QI. This instrument was able to help leaders quickly evaluate key actionable areas linked to physician engagement. A larger sample is warranted for accurate validity and reliability testing. This tool could prove extremely valuable in enhancing physician engagement in QI initiatives. Top

 

References

1.Perreira TA, Perrier L, Prokopy M. Hospital physician engagement: a scoping review. Med Care 2018;56(12):969-75. DOI: 10.1097/MLR.0000000000000983

2.Lepore SJ, Nair RG, Davis SN, Wolf RL, Basch CE, Thomas N, et al. Patient and physician factors associated with undisclosed prostate cancer screening in a sample of predominantly immigrant black men. J Immigr Minor Health 2017;19(6):1343-50. DOI: 10.1007/s10903-016-0468-1

3.Calayag J. Physician engagement: strengthening the culture of quality and safety. Healthc Exec 2014;29(2):28-30.

4.Erlandson E, Ludeman K. Physician engagement and shared accountability. Buzzwords, dilemma or choice? Mich Health Hosp 2003;39(6):28-9.

5.Spaulding A, Gamm L, Menser T. Physician engagement: strategic considerations among leaders at a major health system. Hosp Top 2014;92(3):66-73.

6.Stark R. Increasing physician engagement: start with what’s important to physicians. J Med Pract Manage 2014;30(3):171-5.

7.Studer Q, Hagins Jr M, Cochrane BS. The power of engagement: creating the culture that gets your staff aligned and invested. Healthc Manage Forum 2014;27(1 Suppl):S79-97. DOI: 10.1016/j.hcmf.2014.01.008

8.Whitlock DJ, Stark R. Understanding physician engagement — and how to increase it. Physician Leadersh J 2014;1(1):8-12.

9.Buell JM. Achieving financial success through improved physician engagement: revenue enhancements can be realized with stronger relationships. Healthc Exec 2009;24(1):22-4, 26, 28-9.

10.Daly R. Putting physicians in the lead for cost containment. Healthc Financ Manage 2013;67(12):52-9.

11.Garvin D, Worthington J, McGuire S, Burgetz S, Forster AJ, Patey, et al. Physician performance feedback in a Canadian academic center. Leadersh Health Serv 2017;30(4):457-74. DOI: 10.1108/LHS-08-2016-0037

12.George AE, Frush K, Michener JL. Developing physicians as catalysts for change. Acad Med 2013;88(11):1603-5.

13.Grace SM, Rich J, Chin W, Rodriguez HP. Flexible implementation and integration of new team members to support patient-centered care. Healthc (Amst) 2014;2(2):145-51. DOI: 10.1016/j.hjdsi.2014.02.003 Top

14.Jarousse L. Physician engagement (getting your docs on board!). Hosp Health Netw 2014;88(4):41-7.

15.Kaissi A. Enhancing physician engagement: an international perspective. Int J Health Serv 2014;44(3):567-92. DOI: 10.2190/HS.44.3.h

16.Kim Y, Winner M, Page A, Tisnado DM, Martinez KA, Buettner S, et al. Patient perceptions regarding the likelihood of cure after surgical resection of lung and colorectal cancer. Cancer 2015;121(20):3564-73. DOI: 10.1002/cncr.29530

17.Friedman EL, Chawla N, Morris PT, Castro KM, Carrigan AC, Prabhu Das I, et al. Assessing the development of multidisciplinary care: experience of the National Cancer Institute Community Cancer Centers Program. J Oncol Pract 2015;11(1):e36-43. DOI: 10.1200/JOP.2014.001535

18.Kreindler SA, Larson BK, Wu FM, Gbemudu JN, Carluzzo KL, Struthers A, et al. The rules of engagement: physician engagement strategies in intergroup contexts. J Health Organ Manag 2014;28(1):41-61. DOI: 10.1108/JHOM-02-2013-0024

19.Lee VS, Miller T, Daniels C, Paine M, Gresh B, Betz AL. Creating the exceptional patient experience in one academic health system. Acad Med 2016;91(3):338-44. DOI: 10.1097/ACM.0000000000001007

20.Manary M, Staelin R, Kosel K, Schulman KA, Glickman SW. Organizational characteristics and patient experiences with hospital care: a survey study of hospital chief patient experience officers. Am J Med Qual 2015;30(5):432-40. DOI: 10.1177/1062860614539994

21.Pantaleoni JL, Stevens LA, Mailes ES, Goad BA, Longhurst CA. Successful physician training program for large scale EMR implementation. Appl Clin Inform 2015;6(1):80-95. DOI: 10.4338/ACI-2014-09-CR-0076

22.Pariser P, Pus L, Stanaitis I, Abrams H, Ivers N, Baker GR, et al. Improving system integration: the art and science of engaging small community practices in health system innovation. Int J Fam Med 2016;2016:5926303. DOI: 10.1155/2016/5926303

23.Puri AK, Bhaloo T, Kirshin T, Mithani A. A comprehensive approach to effectively engage physicians during a hospital closure: using the physician engagement model. Healthc Manage Forum 2006;19(4):34-9. DOI: 10.1016/S0840-4704(10)60244-X

24.Rangachari P. Role of social knowledge networking technology in facilitating meaningful use of electronic health record medication reconciliation. J Hosp Admin 2016;5(3):98-106. DOI: 10.5430/jha.v5n3p98

25.Ricottone M. Reducing alternate level of care days at Winchester District Memorial Hospital. Healthc Manage Forum 2015;28(5):190-4. DOI: 10.1177/0840470415588668

26.Rosenstein AH. Strategies to enhance physician engagement. J Med Pract Manage 2015;31(2):113-6.

27.Sondheim SE, Patel DM, Chin N, Barwis K, Werner J, Barclay A, et al. Governance practices in an era of healthcare transformation: achieving a successful turnaround. J Healthc Manage 2017;62(5):316-26. DOI: 10.1097/JHM-D-15-00036

28.Townsend CS, McNulty M, Grillo-Peck A. Implementing huddles improves care coordination in an academic health center. Prof Case Manag 2017;22(1):29-35. DOI: 10.1097/NCM.0000000000000200

29.Wei AC, Sandhu L, Devitt KS, Gagliardi AR, Kennedy ED, Urbach DR, et al., Practice patterns for the management of hepatic metastases from colorectal cancer: a mixed methods analysis. Ann Surg Oncol 2013;20(5):1567-74. DOI: 10.1245/s10434-012-2698-3

30.Akosa AN. Physician engagement is critical to the success of any accountable care organization. J Manage Care Med 2013;16(3):67-76.

31.Klugman R, Gitkind MJ, Walsh KE. The physician quality officer model: 5-year follow-up. Am J Med Qual, 2015;30(5):454-8. DOI: 10.1177/1062860614536221

32.Skillman M, Cross-Barnet C, Singer RF, Ruiz S, Rotondo C, Ahn R, et al., Physician engagement strategies in care coordination: findings from the Centers for Medicare & Medicaid Services’ Health Care Innovation Awards Program. Health Serv Res 2017;52(1):291-312. DOI: 10.1111/1475-6773.12622

33.Lee TH. Financial versus non-financial incentives for improving patient experience. J Patient Exp 2015;2(1):4-6. DOI: 10.1177/237437431500200102

34.Bunkers B, Koch M, McDonough B, Whited B. Aligning physician compensation with strategic goals. Healthc Financ Manage 2014;68(7):38-45.

35.Walsh KE, Ettinger WH, Klugman RA. Physician quality officer: a new model for engaging physicians in quality improvement. Am J Med Qual 2009;24(4):295-301. DOI: 10.1177/1062860609336219

36.Rinne ST, Rinne TJ, Olsen K, Wiener RS, Balcezak TJ, Dardani W, et al. Hospital administrators’ perspectives on physician engagement: a qualitative study. J Hosp Med 2018;13(3):179-81. DOI: 10.12788/jhm.2880 Top

37.Rice JA, Sagin T. New conversations for physician engagement. Five design principles to upgrade your governance model. Healthc Exec 2010;25(4):66-70.

38.Scott CG, Thériault A, McGuire S, Samson A, Clement C, Worthington JR. Developing a physician engagement agreement at The Ottawa Hospital: a collaborative approach. Healthc Q 2012;15(3):50-3. DOI: 10.12927/hcq.2013.23020

39.Weiss R. The quest for physician engagement: Physician relationships are crucial in today’s changing environment. Marketing Healthc Serv 2011;31(2):29-31.

40.Denis JL, van Gestel N. Medical doctors in healthcare leadership: theoretical and practical challenges. BMC Health Serv Res 2016;16(Suppl 2):158. DOI: 10.1186/s12913-016-1392-8 Top

41.Gosfield AG, Reinertsen JL. Finding common cause in quality: confronting the physician engagement challenge. Physician Exec 2008;34(2):26-31.

42.Marsden J, van Dijk M, Doris P, Krause DP, Cochrane D. Improving care for British Columbians: the critical role of physician engagement. Healthc Q 2012;15(Spec no.):51-5. DOI: 10.12927/hcq.2012.23163

43.Strasser DC, Burridge AB, Falconer JA, Uomoto JM, Herrin J. Toward spanning the quality chasm: an examination of team functioning measures. Arch Phys Med Rehabil 2014;95(11):2220-3. DOI: 10.1016/j.apmr.2014.06.013

44.Swensen S, Kabcenell A, Shanafelt T. Physician-organization collaboration reduces physician burnout and promotes engagement: the Mayo Clinic experience. J Healthc Manag 2016;61(2):105-27.

45.Ahnfeldt-Mollerup P, dePont Christensen R, Halling A, Kritensen T, Lykkegaard J, Nexøe J, et al. Medical engagement and organizational characteristics in general practice. Fam Pract 2015;33(1):69-74. DOI: 10.1093/fampra/cmv085

46.Cox ED, Nackers KA, Young HN, Moreno MA, Levy JF, Mangione-Smith RM. Influence of race and socioeconomic status on engagement in pediatric primary care. Patient Educ Couns 2012;87(3):319-26. DOI: 10.1016/j.pec.2011.09.012

47.Duberman T, Hemker B, Solomon L. Innovative strategies for physician partnerships. One health system shares the steps it took to strengthen physician engagement. Healthc Exec 2015;30(3):84-6.

48.Kraft S. Changes in acquisition patterns. Health systems rely on due diligence, physician engagement to ensure successful alignment. MGMA Connex 2015;15(2):40-3.

49.Engelman D, Benjamin EM. Physician engagement: the “secret sauce” to success in bundled health care. Am J Med Quality 2018;33(1):100-2. DOI: 10.1177/1062860617703730

50.Patty B, Svendsen CA. A proven approach to physician engagement. Physician Exec 2011;37(4):92-93.

51.Brémault-Phillips SC, Parmar J, Friesen S, Rogers LG, Pike A, Sluggett B. An evaluation of the decision-making capacity assessment model. Can Geriatr J 2016;19(3):83-96. DOI: 10.5770/cgj.19.222

52.Leone FT, Evers-Casey S, Graden S, Schnoll R. Behavioral economic insights into physician tobacco treatment decision-making. Ann Am Thorac Soc 2015;12(3):364-9. DOI: 10.1513/AnnalsATS.201410-467BC

53.Engaging clinicians in the new NHS. London: NHS Alliance; 2003.

54.Donaghy G, McKeever K, Flanagan C, O’Kane D, McQuillan B, Cash J, et al. Helping doctors in training to STEP-UP: a leadership and quality improvement programme in the Belfast Health and Social Care Trust. Ulster Med J 2018;87(2):112-6.

55.Hettema JE, Sorensen JL, Uy M, Jain S. Motivational enhancement therapy to increase resident physician engagement in substance abuse education. Subst Abus 2009;30(3):244-7. DOI: 10.1080/08897070903041210

56.Patterson CJ. Best practices in specialty pharmacy management. J Manag Care Pharm 2013;19(1):42-8. DOI: 10.18553/jmcp.2013.19.1.42

57.Gray D, Nussle R, Cruz A, Kane G, Toomey M, Bay C, et al. Effects of a catheter-associated urinary tract infection prevention campaign on infection rate, catheter utilization, and health care workers’ perspective at a community safety net hospital. Am J Infect Control 2016;44(1):115-6. DOI: 10.1016/j.ajic.2015.08.011

58.Wilson KC, Merli GJ. Performance measures for improving the prevention of venous thromboembolism: achievement in clinical practice. J Thromb Thrombolysis 2011;32(3):293-302. DOI: 10.1007/s11239-011-0605-6

59.Hockey PM, Bates DW. Physicians’ identification of factors associated with quality in high- and low-performing hospitals. Jt Comm J Qual Patient Saf 2010;36(5):217-23. DOI: 10.1016/s1553-7250(10)36035-1 Top

60.Krein SL, Kowlaski CP, Harrod M, Forman J, Saint S. Barriers to reducing urinary catheter use: a qualitative assessment of a statewide initiative. JAMA Intern Med 2013;173(10):881-6. DOI: 10.1001/jamainternmed.2013.105

61.McFadyen C, Lankshear S, Divaris D, Berry M, Hunter A, Srigley J, et al. Physician level reporting of surgical and pathology performance indicators: a regional study to assess feasibility and impact on quality. Can J Surg 2015;58(1):31-40. DOI: 10.1503/cjs.004314

62.Tuttle JC. Cutting CAUTIs in critical care. J Clin Outcomes Manag 2017;24(6):267-72.

63.Wynn JD, Draffin E, Jones A, Reida L. The Vidant Health quality transformation. Jt Comm J Qual Patient Saf 2014;40(5):212-8. DOI: 10.1016/s1553-7250(14)40028-x

64.Nordstrom BR, Saunders EC, McLeman B, Meier A, Xie H, Lambert-Harris C, et al. Using a learning collaborative strategy with office-based practices to increase access and improve quality of care for patients with opioid use disorders. J Addict Med 2016;10(2):117-23. DOI: 10.1097/ADM.0000000000000200

65.Sears NJ. Managing margins through physician engagement. Healthc Financ Manage 2012;66(7):44-7.

66.Snell AJ, Briscoe D, Dickson G. From the inside out: the engagement of physicians as leaders in health care settings. Qual Health Res 2011;21(7):952-67. DOI: 10.1177/1049732311399780

67.Croft GP. Engaging clinicians in improving data quality in the NHS. Swansea, UK: Centre for Health Information, University of Wales Swansea; 2006.

68.Greysen SR, Detsky AS. Solving the puzzle of posthospital recovery: what is the role of the individual physician? J Hosp Med 2015;10(10):697-700. DOI: 10.1002/jhm.2421

69.Law TJ, Leistikow N, Hoofring L, Krumm S, Neufeld K, Needham D. A survey of nurses’ perceptions of the intensive care delirium screening checklist. Dynamics (Pembroke, Ont.) 2012;23(4):18-24.

70.Silver SA, Harel Z, McQuillan R, Weizman AV, Thomas A, Chertow GM, et al. How to begin a quality improvement project. Clin J Am Soc Nephrol 2016;11(5):893-900. DOI: 10.2215/CJN.11491015

71.Taitz JM, Lee TH, Sequist TD. A framework for engaging physicians in quality and safety. BMJ Qual Saf 2012;21(9):722-8. DOI: 10.1136/bmjqs-2011-000167

72.Uy V, May SG, Tietbohl C, Frosch DL. Barriers and facilitators to routine distribution of patient decision support interventions: a preliminary study in community-based primary care settings. Health Expect 2014;17(3):353-64. DOI: 10.1111/j.1369-7625.2011.00760.x

73.Dyrbye, L.N.,Cross-Barnet C, Singer RF, Ruiz S, Rotondo C, Ahn R, et al., Development and preliminary psychometric properties of a well-being index for medical students. BMC Med Educ , 2010;10(1):8.

74.Rafferty AM, Philippou J, Fitzpatrick JM, Pike G, Ball J. Development and testing of the ‘Culture of Care Barometer’(CoCB) in healthcare organisations: a mixed methods study. BMJ Open 2017;7(8):e016677. DOI: 10.1136/bmjopen-2017-016677

75.Spurgeon P, Barwell F, Mazelan P. Developing a medical engagement scale (MES). Int J Clin Leadersh 2008;16(4):213-23.

76.West CP, Dyrbye LN, Satele DV, Sloan JA, Shanafelt TD. Concurrent validity of single-item measures of emotional exhaustion and depersonalization in burnout assessment. J Gen Intern Med 2012;27(11):1445-52. DOI: 10.1007/s11606-012-2015-7 Top

77.Perreira TA, Perrier L, Prokopy M, Neves-Mera L, Persaud DD. Physician engagement: a concept analysis. J Healthc Leadersh 2019;11:101-13. DOI: 10.2147/JHL.S214765

78.Perreira T, Perrier L, Prokopy M, Jonker A. Physician engagement in hospitals: a scoping review protocol. BMJ Open 2018;8(1):e018837. DOI: 10.1136/bmjopen-2017-018837

79.Fink A, Kosecoff J, Chassin M, Brook RH. Consensus methods: characteristics and guidelines for use. Am J Public Health 1984;74(9):979-83. DOI: 10.2105/ajph.74.9.979

80.Mullen PM. Delphi: myths and reality. J Health Organ Manag 2003;17(1):37-52. DOI: 10.1108/14777260310469319

81.Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PloS One 2011;6(6):e20476. DOI: 10.1371/journal.pone.0020476

82.Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs 2000;32(4):1008-15.

83.Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol 2010;10(1):1. DOI: 10.1186/1471-2288-10-1

84.2014 response rates. In National Physician Survey. Mississauga, Ont.: National Physician Survey; 2019. Available: https://tinyurl.com/uzhqm94

85.Dillman DA, Smyth JD, Christian LM. Internet, phone, mail, and mixed-mode surveys: the tailored design method. Hoboken, N.J.: John Wiley & Sons; 2014.

86.Chou CP, Bentler PM. Estimates and tests in structural equation modeling. In Hoyle RH (editor). Structural equation modeling: concepts, issues, and applications. Thousand Oaks, Calif.: Sage; 1995:37-55.

87.Weston R, Gore Jr PA, Chan F, Catalano D. An introduction to using structural equation models in rehabilitation psychology. Rehabil Psychol 2008;53(3):340. DOI: 10.1037/a0013039

88.Field A. Discovering statistics using IBM SPSS statistics (4th edition). Thousand Oaks, Calif.: Sage; 2013.

89.Schaufeli WB, Salanova M, González-Romá V, Bakker AB. The measurement of engagement and burnout: A two sample confirmatory factor analytic approach. J Happiness Stud 2002;3(1):71-92.

90.Brown A, Baker GR, Closson T, Sullivan T. The journey toward high performance and excellent quality. Healthc Q 2012;15(sp):6-9. DOI: 10.12927/hcq.2012.23152

91.About the Excellent Care for All Act. Toronto: Ministry of Health and Long-Term Care; 2019. Available: https://tinyurl.com/mg9pogs

92.Kaye AD, Okanlawon OJ, Urman RD. Clinical performance feedback and quality improvement opportunities for perioperative physicians. Adv Med Educ Pract 2014;5:115-23. DOI: 10.2147/AMEP.S62165

93.Quality matters: realizing excellent care for all. Toronto: Health Quality Ontario; 2015. Available: https://tinyurl.com/tloqrp6

94.Baker GR, Axler R. Creating a high performing healthcare system for Ontario: evidence supporting strategic changes in Ontario. Ottawa: Ontario Hospital Association; 2015. Available: https://tinyurl.com/ubtd4s3

95.CMA National Physician Health Survey: a national snapshot. Ottawa: Canadian Medical Association; 2018. Available: https://tinyurl.com/ubtd4s3

 

Acknowledgements

We thank Elizabeth Carlton, Dara Laxer, Lee Fairclough, Monique Herbert, Gillian Elliott, and Ross Baker for their support and contribution to this project. Each contributed to the research design and development of the survey instrument. We also thank the Ontario Hospital Association’s Physician Provincial Leadership Council and those physicians who participated in this pilot study. Top

 

Authors

Tyrone Perreira, PhD, MEd, is an assistant professor at the University of Toronto’s Institute of Health Policy Management and Evaluation, Dalla Lana School of Public Health and a research scientist at the Ontario Hospital Association.

 

Melissa Prokopy, LLB, is director of Legal, Policy and Professional Issues at the Ontario Hospital Association and adjunct faculty at the University of Toronto’s Institute for Health Policy, Management and Evaluation.

 

Adalsteinn Brown, DPhil, AB, is dean of the Dalla Lana School of Public Health at the University of Toronto.

 

Anna Greenberg, MPP, is president of Ontario Health’s business unit focused on quality.

 

James Wright, MD, MPH, is chief, Economics, Policy and Research at the Ontario Medical Association.

 

Christine Shea, PhD, MEd, is program director of Quality Improvement and Patient Safety at the University of Toronto’s Institute of Health Policy Management and Evaluation, Dalla Lana School of Public Health.

 

Julie Simard is a doctoral student at the University of Toronto’s Institute of Health Policy Management and Evaluation, Dalla Lana School of Public Health.

 

Author attestation

In addition to participating in the research design and development of the survey instrument, all authors contributed significantly to article preparation. Tyrone Perreira and Melissa Prokopy conceptualized the article. Adalsteinn Brown, Anna Greenberg, James Wright, Christine Shea, and Julie Simard assisted with organization and revisions of the article. All authors approved the final version.

 

Conflict of interest: The authors declare no conflict of interest.

 

Correspondence to:

ty.perreira@utoronto.ca

 

This article has been peer reviewed.

 

Top

Using Health Quality Ontario’s six defining elements of quality care,93 our survey helped to show that there may be opportunities for greater promotion of project results and additional QI projects focused on equitable, timely, efficient, and/or effective services. To create a high-performing health care system, a system-wide perspective is needed.94 This instrument helps to identify a potential need for, or lack of, system-level QI projects. This is the first time this type of data has been captured and examined in Ontario. Results clearly indicated that just over half of our sample group participated in QI projects, of which the majority were at the organization and patient levels, with only a few at the system level. Top

Our survey population was small and made up, predominantly, of individuals in hospital leadership roles. However, the purpose was not to generalize results, but to develop and test an instrument that could be used by health care leadership in Ontario to quickly evaluate key areas, suggested in the literature to impact engagement in QI within their organizations.

4.Erlandson E, Ludeman K. Physician engagement and shared accountability. Buzzwords, dilemma or choice? Mich Health Hosp 2003;39(6):28-9.