In an era of near-complete adoption of electronic health records (EHRs) and coalescence of health data across departments and institutions, a growing recognition of practice variation has emerged. Perioperative care is no exception, with recent studies demonstrating wide institution-level variation in practices such as anesthetic techniques employed,¹ medications administered,^2,3 and operating room staffing models used.⁴ In some cases, practice variation is warranted—as explained by factors such as subspecialty training, local health resource constraints, and informed expectations of patients. Yet, in other cases variation is unexplained or unwarranted, and possibly attributable to a lack of practice benchmarking, suboptimal hospital resource allocation, or lack of precision care tailored to individual patient needs.^5,6

In some cases, such practice variation may be associated with worse outcomes, including anesthesia professional staffing ratio practice patterns,⁴ hospital level compliance with safety practices,⁷ and failure to rescue rates.⁸

To address unexplained or unwarranted variation, modern quality improvement (QI) and research initiatives increasingly seek out multicenter learning-health systems approaches, integrating comparative effectiveness evidence drawn from practice variation across centers to develop performance benchmarks and quality measures.^9,10 With strategic multicenter infrastructures in place, such benchmarks and quality measures can in turn be disseminated across participating institutions to rapidly iterate upon evolving best practices and enhance patient safety and health care value.^11,12 One learning health system infrastructure relevant to perioperative care is the Multicenter Perioperative Outcomes Group (MPOG), which we cover in this article to illustrate (i) approaches necessary for integrating perioperative EHRs for research and quality improvement (QI); (ii) big data tools which can be used to effectively harness large volumes of perioperative health data amassed; and (iii) the value proposition of creating community sharing research and quality measure outputs to advance perioperative care and patient safety. Finally, with the rise of artificial intelligence and machine learning approaches offering new opportunities for enhancing health information gathering and clinical decision-making, we describe core challenges to successful, sustained implementation of artificial intelligence/machine learning methods and approaches to address such challenges.

PRINCIPLES OF A LEARNING HEALTH SYSTEM GUIDED BY PERIOPERATIVE DATA: THE MULTICENTER PERIOPERATIVE OUTCOMES GROUP (MPOG)

EHR Data Are Highly Variable Across Institutions

A Learning Health System (LHS) has been defined as one “in which knowledge generation is so embedded into the core of the practice of medicine that it is a natural outgrowth and product of the health care delivery process and leads to continual improvement in care.”¹³ MPOG aspires to be a learning health system focused on perioperative care that addresses continuously rising standards for QI, research, and patient safety (Figure 1). MPOG was launched in 2008 by several academic centers interested in using their newly implemented electronic anesthesia recordkeeping systems for multicenter observational analyses. However, it soon became clear that this same dataset, with appropriate governance and collaboration, could be the foundation of a learning health system where MPOG data generates knowledge. This knowledge leads to practice change, and practice changes lead to new data. The flywheel effect of this approach has now led to the participation of nearly 100 hospitals in the MPOG group. In turn, MPOG has developed tools to extract, ingest, clean, and analyze these data for a variety of research, QI, and education-related uses. The minimum dataset submitted by each institution includes physiologic, medication, text notes, staffing, key events, and fluid input and output data during the perioperative period. These markers are all derived automatically from institutionally mapped data within existing anesthesia medical records and are largely agnostic to the specific EHR vendor being used at each institution. Additionally, preoperative history and physical information, laboratory results, and administrative data such as Current Procedural Terminology (CPT) codes, discharge diagnoses, and hospital mortality data are included.

Figure 1: Pillars of Multicenter Perioperative Outcomes Group (MPOG) Research and Quality Improvement.

EHR data are highly variable across institutions. As a result, a foundational component of MPOG is the methodology for translating EHR data across participating sites into pre-computed, validated phenotypes usable for research and QI.¹⁴ This rigorous process involves applying algorithms to integrate combinations of all the data types within MPOG to generate more reliable clinical inferences. These inferences serve as building blocks that enable both researchers to conduct analyses, and QI leaders and clinicians to understand variation in care patterns. Examples of phenotypes that are essential components of MPOG research and QI include anesthesia technique, American Society of Anesthesiologists physical status, and patients’ smoking status. In each of these cases, there are thousands of ways these data are documented across sites, and software algorithms developed by MPOG translate the data into interoperable phenotypes.

MPOG TOOLS FOR TRANSFORMING PERIOPERATIVE EHR DATA INTO KNOWLEDGE AND ACTION FOR ENHANCING PATIENT SAFETY

MPOG has developed programs and tools to analyze big data

MPOG has developed programs and tools to analyze big data and enable inferences for nuanced and meaningful QI and research projects aimed at improving patient safety.

MPOG’s QI mission is governed by its Quality Committee, composed of anesthesia professional QI champions for each participating site. This committee approves and maintains quality measures reflecting the best available evidence with an established plan to revisit QI measures at regular intervals to accommodate the field’s expanding and evolving knowledge base. Ideas for new QI initiatives are generated from this committee as well as subspecialty subcommittees focused on pediatric, obstetric, geriatric, and cardiac anesthesia, each composed of quality champions and domain experts from participating institutions. These committees foster open discussions, collaboration, and the sharing of best practices and lessons learned.

In order for members to enact change at their institutions, MPOG has developed a series of programs built upon the computed phenotypes foundation. These programs include QI measure development, practice level feedback, individual provider feedback, QI toolkits, and quality collaborative meetings as described in Table 1. Further details describing all QI measures can be found at https://spec.mpog.org/Measures/Public. Individual provider performance can be tracked and feedback can be provided to individuals (Figure 2).

Table 1: Quality Improvement Programs within the Multicenter Perioperative Outcomes Group.

Figure 2: Individual Provider Feedback on Perioperative Quality: Personalized Performance Emails.

To complement its QI mission, MPOG’s research mission is governed by its Research Committee, which coordinates clinical research efforts of MPOG by reviewing submitted proposals and tracking the progress of ongoing projects. This committee, composed of MPOG principal investigators from each participating site, evaluates all MPOG research proposals, provides crucial guidance on hypotheses and methodology, and ensures the scientific appropriateness of clinical research using MPOG data prior to a project’s approval. To enable meaningful research using MPOG data, the group has built several programs and tools to leverage the Registry. These programs include regular research committee meetings and an annual MPOG Retreat, as well as software tools (e.g., DataDirect®, Ann Arbor, Michigan) to develop research cohorts and streamline research queries.

PERFORMANCE IMPROVEMENT WITHIN THE STATE OF MICHIGAN

In the state of Michigan, MPOG is part of a Blue Cross Blue Shield of Michigan funded QI program, which functions as a learning health system.¹⁷ This program funds QI groups across a range of specialties and health conditions.¹⁸ Through the mechanisms described above, unblinded performance reviews, multispecialty collaborative meetings, and payor-driven financial incentives lead to substantial improvements in care. These are evidenced by improvements in important anesthetic care domains such as glycemic and temperature management, as well as achieving more cost-effective care for hospitals participating in this program (Table 2).¹⁹

Table 2: Multicenter Perioperative Outcomes Group Examples of Quality Improvement Impact.

RESEARCH INITIATIVE: ASSESSMENTS OF MULTICENTER PRACTICE VARIATION AND PERIOPERATIVE CARE STRUCTURES

Given the breadth of perioperative practice variation across clinicians and sites, important research findings of MPOG have included studies which quantify the degree to which practice patterns are explained by the clinician or institution, rather than the patient or surgery. Such practice variation, potentially indicative of clinician training, personal practice preferences, or institution-level structures of clinical care and infrastructure, has been leveraged to study impact on patient outcomes. In some cases, practice variation—including anesthesia professional staffing ratios, hospital level compliance with safety practices,⁷ and failure to rescue rates⁸­—is associated with worse outcomes; whereas in other cases a lack of association exists with adverse outcomes, including overlapping surgeries by an attending surgeon²⁰ or surgeries in which the surgeon operated overnight the day prior.²¹

OPPORTUNITIES AND CHALLENGES INTRODUCED BY ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING IN PERIOPERATIVE CARE

Challenges Exist to Safe Adoption of Artificial Intelligence

Coinciding with the development of big data tools for processing electronic health record (EHR) data to perform multicenter research and QI, are opportunities to apply methods using artificial intelligence and machine learning to improve data quality, develop QI measures, and improve clinical care through predictive algorithm development. Given the complexities and granularity of perioperative EHR data, artificial intelligence/machine learning methods capable of handling large numbers of complex non-linear interactions across variables sometimes offer substantial advantages over classical statistical approaches. Yet, challenges exist to safe adoption of artificial intelligence/machine learning-based methods in perioperative learning health systems. These include (i) wide variations in the available clinician knowledge base regarding strengths and limitations; (ii) a need for clinical algorithm oversight and governance; (iii) the need to ensure fidelity of source data upon which artificial intelligence/machine learning algorithms are trained; and (iv) a systematic approach to recognizing and addressing biases potentially propagated in artificial intelligence/machine learning-based clinical decision support systems (Figure 3).

Figure 3: Considerations for Safe Adoption of Artificial Intelligence (AI) and Machine Learning (ML) into Perioperative Care.

Related to clinician knowledge, artificial intelligence/machine learning education is being incorporated into medical curricula and continuing medical education opportunities in health care.²² Related to algorithm governance and oversight, QI and patient safety efforts propose frameworks for committees to monitor artificial intelligence/machine learning models deployed within a health system.²³ With regard to data fidelity, approaches to diagnosing and remedying changes to EHR data quality (“dataset shift”) are proposed,²⁴ focusing on maintaining closed-loop communication between frontline clinicians and algorithm governance committees, which may enhance patient safety by promoting awareness of model under-performance and thereby educating clinicians as to clinical contexts for which the prediction model can be relied upon versus disregarded. Finally, as algorithmic bias concerns remain, opportunities to address differential model performance across varying clinical subgroups—particularly when racial, ethnic, and sex-based,²⁵—include explicitly examining artificial intelligence/machine learning model performance in such subgroups.

CONCLUSION

Opportunities are ripe for coalescing perioperative EHR data across patients, clinicians, institutions, and regions to perform comparative effectiveness research and improve the quality and safety of anesthesia care. Perioperative learning health systems equipped with big data tools with appropriate leveraging of novel artificial intelligence/machine learning-based methods provide a platform for clinician communities to share data, exchange ideas, and disseminate evolving best practices within a learning health system.

Michael R. Mathis is an associate professor of anesthesiology, and an affiliate faculty member in the Department of Computational Bioinformatics at Michigan Medicine, University of Michigan, Ann Arbor, MI.

Robert B. Schonberger is an associate professor and vice chair for academic affairs in anesthesiology at the Yale School of Medicine, New Haven, CT.

Anthony L. Edelman is an assistant professor of anesthesiology and associate chair for adult anesthesiology at Michigan Medicine, University of Michigan, Ann Arbor, MI.

Allison M. Janda, MD, is an assistant professor of anesthesiology at Michigan Medicine, University of Michigan, Ann Arbor, MI.

Douglas A. Colquhoun, MB ChB, MSc, MPH, is an assistant professor of anesthesiology at Michigan Medicine, University of Michigan, Ann Arbor, MI.

Michael L. Burns, MD, is an assistant professor of anesthesiology at Michigan Medicine, University of Michigan, Ann Arbor, MI.

Nirav J. Shah is an associate professor of anesthesiology at Michigan Medicine, University of Michigan, Ann Arbor, MI.

Michael Mathis, MD, has received research grants from the US National Institutes of Health (NHLBI, NIDDK, AHRQ) and research support paid to the University of Michigan from Chiesi, USA, unrelated to this present work. Robert Schoenberger, MD, MCDHS, reports that he owns stock in Johnson and Johnson unrelated to the present work. Anthony Edelman, MD, has received funding (paid to the University of Michigan) from the US National Institutes of Health (AHRQ) unrelated to the present work. Allison Janda, MD, has received research grant support from the US National Institutes of Health (NHLBI) and the Patient Centered Outcomes Research Institute unrelated to this present work. Douglas Colquhoun, MB ChB, MSc, MPH, has received a research grant from the US National Institutes of Health (NHLBI) and research support paid to the University of Michigan from Merck & Co and Chiesi, USA, unrelated to this present work. Michael Burns, MD, has received research grant support from Blue Cross Blue Shield of Michigan (BCBSM) and the Patient-Centered Outcomes Research Institute unrelated to this present work; and is the co-founder of Decimal Code, Inc., unrelated to the present work. Nirav Shah, MD, has received funding (paid to the University of Michigan) from the US National Institutes of Health (NLM, NIA), Patient Centered Outcomes Research Institute, Blue Cross Blue Shield Michigan, Edwards Lifesciences, and Apple, Inc., unrelated to this present work. No other relationships or activities that could appear to have influenced the submitted work.

All work and partial funding is attributed to the Department of Anesthesiology, Michigan Medicine, University of Michigan (Ann Arbor, Michigan, USA). The project described was supported in part by the US National Institutes of Health (NIDDK R01DK133226; NHLBI R01HL167790, NIA R01AG059607, NHLBI K08HL159327, NHLBI K23HL166685, Bethesda, MD). In addition, partial funding to support underlying electronic health record data collection into the Multicenter Perioperative Outcomes Group registry was provided by Blue Cross Blue Shield of Michigan/Blue Care Network as part of the Blue Cross Blue Shield of Michigan/Blue Care Network Value Partnerships program. Although Blue Cross Blue Shield of Michigan/Blue Care Network and Multicenter Perioperative Outcomes Group work collaboratively, the opinions, beliefs, and viewpoints expressed by the authors do not necessarily reflect the opinions, beliefs, and viewpoints of Blue Cross Blue Shield of Michigan/Blue Care Network or any of its employees. Additionally, the opinions, beliefs, and viewpoints expressed by the authors do not necessarily reflect the opinions, beliefs, and viewpoints of the National Institutes of Health, or any of its employees. Industry contributors have had no role in the study.

REFERENCES

1. Roberts DJ, Mor R, Rosen MN, et al. Hospital-, anesthesiologist-, surgeon-, and patient-level variations in neuraxial anesthesia use for lower limb revascularization surgery: a population-based cross-sectional study. Anesth Analg. 2022;135:1282–1292. PMID: 36219577.
2. Janda AM, Spence J, Dubovoy T, et al. Multicentre analysis of practice patterns regarding benzodiazepine use in cardiac surgery. Br J Anaesth. 2022;128:772–784. PMID: 35101244.
3. Mathis MR, Janda AM, Kheterpal S, et al. Patient-, clinician-, and institution-level variation in inotrope use for cardiac surgery: a multicenter observational analysis. Anesthesiology. 2023;139:122–141. PMID: 37094103.
4. Burns ML, Saager L, Cassidy RB, et al. Association of anesthesiologist staffing ratio with surgical patient morbidity and mortality. JAMA Surg. 2022;157:807–815. PMID: 35857304.
5. Sutherland K, Levesque JF. Unwarranted clinical variation in health care: definitions and proposal of an analytic framework. J Eval Clin Pract. 2020;26:687–696. PMID: 31136047.
6. Sessler DI. Implications of practice variability. Anesthesiology. 2020;132:606–608. PMID: 32053562.
7. Brooke BS, Dominici F, Pronovost PJ, et al. Variations in surgical outcomes associated with hospital compliance with safety practices. Surgery. 2012;151:651–659. PMID: 22261296.
8. Portuondo JI, Farjah F, Massarweh NN. Association between hospital perioperative quality and long-term survival after noncardiac surgery. JAMA Surg. 2022;157:258–268. PMID: 35044437.
9. Casey JD, Courtright KR, Rice TW, Semler MW. What can a learning healthcare system teach us about improving outcomes? Curr Opin Crit Care. 2021;27:527–536. PMID: 34232148.
10. Foley T, Vale L. A framework for understanding, designing, developing and evaluating learning health systems. Learn Health Syst. 2023;7:e10315. PMID: 36654802.
11. Sheetz KH, Englesbe MJ. Expanding the quality collaborative model as a blueprint for higher-value care. JAMA Health Forum. 2020;1:e200413-e200413. PMID: 36218502.
12. Smith M, Saunders R, Stuckhardt L, McGinnis JM. Committee on the Learning Health Care System in America; Institute of Medicine. Best care at lower cost: the path to continuously learning health care in America. National Academies Press (US); 2013 May 10. PMID: 24901184.
13. Olsen L, Aisner D, Mcginnis JM, eds. The Learning Healthcare System: Workshop Summary. National Academies Press; 2007. PMID: 21452449.
14. Colquhoun DA, Shanks AM, Kapeles SR, et al. Considerations for integration of perioperative electronic health records across institutions for research and quality improvement: the approach taken by the Multicenter Perioperative Outcomes Group. Anesth Analg. 2020;130:1133–1146. PMID: 32287121.
15. MPOG Measure Specs—Measure List. https://spec.mpog.org/Measures/Public. Accessed February 16, 2024.
16. Toolkits. MPOG. Published July 24, 2019. https://mpog.org/toolkits/. Accessed February 16, 2024.
17. Howard R, Grant J, Leyden T, Englesbe M. Improving the quality of health care through 25 years of statewide collaboration in Michigan. NEJM Catalyst. 3:CAT.22.0153. doi: 10.1056/CAT.22.0153.
18. Collaborative quality initiatives—value partnerships.com — blue cross blue shield of Michigan. https://www.valuepartnerships.com/programs/collaborative-quality-initiatives/. Accessed February 16, 2024.
19. Janda AM, Vaughn MT, Colquhoun DA, et al. Does anesthesia quality improvement participation lead to incremental savings in a surgical quality collaborative population? A retrospective observational study. Anesth Analg. 2023;137:1093–1103. PMID: 37678254.
20. Sun E, Mello MM, Rishel CA, et al. Association of overlapping surgery with perioperative outcomes. JAMA. 2019;321:762–772. PMID: 30806696.
21. Sun EC, Mello MM, Vaughn MT, et al. Assessment of perioperative outcomes among surgeons who operated the night before. JAMA Intern Med. 2022;182:720–728. PMID: 35604661.
22. Howell MD, Corrado GS, DeSalvo KB. Three epochs of artificial intelligence in health care. JAMA. 2024;331:242–244. PMID: 38227029.
23. Reddy S, Allan S, Coghlan S, Cooper P. A governance model for the application of AI in health care. J Am Med Inform Assoc. 2020;27:491–497. PMID: 31682262.
24. Finlayson SG, Subbaswamy A, Singh K, et al. The clinician and dataset shift in artificial intelligence. N Engl J Med. 2021;385:283–286. PMID: 34260843.
25. Chin MH, Afsar-Manesh N, Bierman AS, et al. Guiding principles to address the impact of algorithm bias on racial and ethnic disparities in health and health care. JAMA Netw Open. 2023;6:e2345050. PMID: 38100101.