Lessons from Europe
The need for data to help manage the process of healthcare delivery has become painfully clear in recent years. Irrespective of whether healthcare is financed from the public or private sectors, virtually every country in the world is challenged to ration healthcare funds. The underlying question for all involved—managers, providers and patients—is the relationship between the outcome of the healthcare process and the resources expended to provide that care. This relationship can only be understood through the analysis of appropriate data that documents meaningful outcome measures and captures the costs incurred. How much data, and of what type, do we need to collect to understand the relationship between expense and outcome? If we attempt to collect too much data, the process will be costly and burdensome . If we settle for collecting a small amount of data that is easy to manage, we may end up with too little data to draw meaningful conclusions. How much data is “just right?”
Members of the European anesthesia community have made some progress towards defining how much data is “just right” through their efforts to define and collect a minimal anesthesia dataset for every anesthetic encounter. In 1993, The European Society for Computing in Anaesthesia and Intensive Care (ESCTAIC) and the Society for Computing and Technology in Anaesthesia (SCATA) developed a minimal anesthetic dataset that has been accepted by the Royal College of Anaesthetists (UK) and by the German Anaesthetic Society (Deutsche Gesellschaft fuer Anaesthesiologie und Intensivmedizin, DGAI, which has incorporated significant changes). This dataset was designed to capture a spectrum of information about the anesthetic encounter that could be pooled among many institutions to facilitate analysis of large patient populations. The data fields include demographic information about the patient, identifying information about the anesthetic encounter and a field for entering one or more critical incident codes.1 The committee that developed this dataset recognized that each data field would need to be clearly defined if data were to be pooled among departments.
Since this dataset was developed, the DGAI has made progress towards implementing processes for collecting and analyzing data for every anesthetic encounter. Several regions have formed quality councils in which member institutions collect data for every anesthetic and pool the data in an effort to develop national benchmarks for anesthetic outcome. Participation in this process is voluntary, although each participating institution must comply with documentation requirements, which include a routine documentation tool for all anesthetic encounters, a record that integrates patient care information with quality assurance data and collection of a standard dataset for each encounter that can be pooled with identical datasets from other institutions. The goal is to establish quality benchmarks and to compare each institution against these benchmarks so that when variations are identified, further quality analysis can be triggered. At present, there are two quality councils each encompassing as many as 50 institutions. More than 1,000,000 patient encounters have been studied to date by these councils.
An important component of the DGAI effort is documentation of the occurrence of anesthesia incidents, events and complications (IECs). To be considered worthy of recording, each IEC must satisfy three conditions: 1) it must occur during the perioperative period i.e., from the induction of anesthesia until discharge from post-anesthetic care, 2) it must lead to an intervention by the anesthetist, and 3) it must either have caused, or could have caused, morbidity or mortality if the anesthetist had not intervened (making it roughly equivalent to the traditional definition of the term “critical incident”). The DGAI has defined 63 different kinds of IECs, which include varied types of events such as, hypotension, unanticipated difficult intubation, and bronchospasm. One of five grades of severity is assigned to each IEC based on outcome. The severity grades vary from no subsequent impact on patient care to permanent damage or death resulting from the event.
Experience with this process has been reported from the University of Ulm by Bothner and associates.2 These efforts began in 1992 and have continued ever since accumulating data on almost 100,000 patient encounters. The heart of the data collection process is an optically read paper-based anesthesia record that contains preoperative information, intra-operative documentation and data on the post-anesthetic course up to the time of discharge from the PACU. The paper record is designed to facilitate entry of data items from the minimal dataset into a computer using scanner technology. Data are validated first by the consultant anesthetist responsible for the procedure, by a documentation clerk and finally by the database system when the information from the records is entered into the computer. Any questions about the completeness or validity of the data are resolved by the consultant anesthetist in charge of the procedure before the data are included permanently in the database. Analysis of the data accumulated up to 1997 demonstrated an overall incidence of all IECs of 22%. The highest proportion of IECs occurred in the lowest severity group while 0.1% of the patient encounters reported IECs that resulted in either death of permanent disability.
One of the most important lessons reported by Bothner and his associates is that “leading a project of this size is no task for those that are easily discouraged.” It is difficult to motivate individual providers to report incidents that ultimately reflect on the outcome of their own patients. Not only must a certain discipline about the recording process be taught, but the culture of the department must change to recognize quality improvement activities as constructive and instructive instead of punitive.
Efforts by the DGAI towards quality improvement are not the only such efforts that have been attempted on a national scale. In some countries, these efforts have been underway for some time. The Confidential Enquiry into Perioperative Death (CEPOD) in the UK, and the Australian Incident Monitoring Study are two previous examples of such activities. All of these activities have had some success at documenting the incidence of certain complications, although understanding causation is a more difficult problem.
Efforts like those in Europe and Australia to collect data on a national scale regarding anesthesia outcome are notably absent in the United States. Despite the emphasis on continuous quality improvement by managed care organizations and the Joint Commission on Healthcare Organizations, the U.S. seems to lack the organizational and legal infrastructure to collect and share data on a national level. APSF has turned attention to this issue and established task forces made up of clinicians and industry representatives to investigate the many problems that must be faced to achieve a national quality assessment data collection and analysis process. The potential to realize this goal likely lies in the availability of well designed anesthesia information systems. As these systems mature, they will offer the ability to record the type of information needed to perform meaningful analysis of anesthesia care outcome and the relationship to cost.
Although every anesthesia department can, and should, perform meaningful quality assurance, analysis of anesthesia outcome requires that data be pooled amongst more than one institution to support statistical analysis. It will take time and experimentation to determine how much data, and what type of data to collect. As we start the process, we may design datasets which are too large, and those that are too small, but with persistence we may find a dataset that is “just right.”
Dr. Feldman is Anesthesiologist, Virtua Memorial Hospital and Consulting Medical Director, Draeger Medical; Dr. Schwilk is Assistant Professor in Anesthesiology, Medical Center, University of Ulm, Germany and General Secretary of the European Society for Computing and Technology in Anaesthesiology and Intensive Care (ESCTAIC: www.esctaic.org); Dr. Houweling is from the Department of Anesthesiology, Diakonessenhuis Utrecht, the Netherlands.
References
1. Lack JA. Stuart-Taylor M. Tecklenburg A. An anaesthetic minimum dataset and report format. Br J Anaesth. 1994;73:256.
2. Bothner U. Georgieff M. Schwilk B. Building a large-scale perioperative anaesthesia outcome-tracking database: methodology, implementation and experiences from one provider within the German quality project. Br J Anaesth. 2000;85:271.