As in all scientific and technical human endeavors, the best strategies to deal with a problem are based on firm knowledge of the magnitude and characteristics of that problem. Hence, for the protection of patient safety in the setting of "office-based anesthesia" ("OBA"), planning to collect carefully useful data on what happens in such settings is a crucial part of any long-term effort. Note that in considering patient safety within the context of OBA, certain aspects of the office-based surgery itself, such as the use of local anesthetics, should be considered even when anesthesia personnel may not directly participate.

For what purpose are OBA data to be collected? Are data needed to support a declaration calling for one single high level of safety for surgery and anesthesia regardless of its physical location? Some clinicians argue that the only justification for extending to the OBA setting the level of safety practices, regulation, and oversight found in hospitals or accredited surgicenters (hereafter referred to as "HSA") would be the finding that patient care with OBA is clearly less safe than in the HSA setting. I emphatically argue otherwise. The declaration of the need for a single high level of safety for patients is a matter of principle and not a matter of data. Data should be collected to determine how best to enact this principle, but the burden of proof to suggest any lesser oversight, regulation, credentialing, or accreditation for the OBA setting than is currently in force in the HSA setting lies firmly on the shoulders of those suggesting that such reduced standards and scrutiny are safe. In other words, the null hypothesis must be that any setting that lacks equivalent equipment, facilities, competence of personnel, established safety procedures, and oversight is not equally safe. Data in the future could perhaps show that this null hypothesis can be rejected and that OBA safety procedures can be relaxed, but, until that time, it is appropriate that policies today call for using the best practices for safe surgery and anesthesia in all settings in which they are conducted.

The relevant goals of OBA data collection include:

• Determining the types and frequency of "surgical" procedures that are now being done in clinician offices and those that are likely to emerge in these settings in the foreseeable future. These may include procedures that are rarely if ever done in the HSA setting.
• Determining the spectrum of anesthetic management that is required now, or may be required in the future, for these procedures in office-based settings.
• Determining the spectrum of safety practices in the office-based setting and how these can or do differ from the spectrum seen in hospital and surgicenter settings. These practices will include (but are not limited to): i) Equipment, including anesthesia delivery devices, patient monitoring, and emergency care devices; ii) Physical facilities and supplies, including compressed gases, suction, scavenging, and electrical power; iii) Personnel including "surgeon" (or clinician performing a non-surgical procedure), assistants, anesthesia provider, and recovery personnel; iv) Emergency equipment and procedures; and, v) Arrangements for backup care and transfer of patients.

• Determining what kinds of problems occur in OBA settings. How, if at all, do such problems differ from those that occur during analogous procedures in HSA settings? What strategies would best prevent or mitigate these problems? Problems vary widely in severity, but useful data could be collected about the following occurrences during or after procedures in OBA and HSA procedures: i) Death; ii) Cardiac arrest or other serious critical incidents; iii) Abnormal clinical events not causing a significant negative outcome (critical incidents); iv) Litigation involving complications alleged to have occurred during or after office-based procedures.

A variety of techniques and systems could be utilized to collect these kinds of data and these can be divided into several categories:

• Surveys of practices in OBA settings. These can target: i) practitioners (clinicians each for the procedure and the anesthesia); ii) patients; iii) payors (although admittedly much of OBA is based on patient cash payments); iv) liability insurers.
• Inspections/audits of OBA settings. Inspections or audits information could come from accreditation agencies or from specific efforts limited to data collection only (no accreditation or enforcement mechanism). Inspections could be pre-announced (as for routine JCAHO surveys) or unannounced. Although most inspections would be overt, one can imagine the possibility of "covert" inspections conducted by individuals posing as prospective patients or posing as prospective colleagues or employees of a facility.

• Traditional epidemiologic studies of patient outcome. The gold-standard would be prospective studies of patients randomized to either the OBA or HSA setting. As will be discussed below it is very unlikely that such a study can be performed, especially one that would compare the actual spectrum of care in all OBA settings vs. all HSA settings. Another classic technique would be to do a retrospective case-control study of matched patients having matched procedures in the different settings.
• Collection of reports of events or incidents. The systematic study of such occurrences and reports has been very important in other high-hazard industries including aviation and nuclear power generation. Current medical practice, especially in HSA settings, provides some mechanisms for clinicians to report and discuss abnormal events or incidents. Some case reports are published in scholarly journals, although only a handful appear each year. Submission of such reports could be encouraged, and editors should be encouraged to consider favorably publication of these reports as they apply to new issues of OBA. However, in the current medicolegal climate there are strong disincentives to report problems to any forum from which a plaintiff might subpoena information.1 This problem plagues even mandatory reporting systems such as that required by the FDA for events involving medical devices that cause serious injury or harm,2 or for the sentinel-event reporting internally mandated by the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) for its accredited organization. Individual clinicians in private offices are probably even more influenced by these disincentives because they do not have the economy of scale of larger organizations and may thus perceive an even greater medicolegal exposure and marketplace risk from such reporting.

• Analysis of closed malpractice claims. The ASA Closed Claims Project has proven clearly the benefits of systematic study of closed malpractice claims.⁴ Although these cases constitute only a subset of all occurrences, they are by definition highly salient. Since litigation is already completed, there seems to be little risk in having these cases analyzed. Nonetheless, even insurance companies and self-insured institutions with a strong interest in safety improvement often are reluctant to make closed claims data available for analysis.

In considering any data collection and analysis techniques, it is important to take into account potential biases and difficulties, in particular as they affect one approach more than another. Such considerations include:

1. Selection Bias:

Any technique requiring initiation or cooperation by a site can suffer seriously from reporter self-selection bias. This can apply to surveys, voluntary reporting, or participation in a prospective or retrospective focused study. In particular:

• Sites with poor safety practices will be less likely than will sites with good safety practices to instigate, participate in, or even cooperate with reporting, data collection, inspections, or audits. Sites or practitioners with poor safety practices are likely to under-report incidents, complications, or even deaths. Mandatory reporting is often not much better, especially if there is a subjective element to the criteria defining when a report is required. Experience with reporting mandated by the Safe Medical Devices Act supports this contention.² Conversely, sites with strong safety practices are more likely to report complications honestly and to cooperate with other types of studies. Further, HSA sites with weaker than average safety practices may be more likely to participate than OBA sites with weaker than average safety practices. This would occur if the HSA sites feel more protected or comfortable with reporting due to their accredited status. It is thus possible that OBA sites with particularly strong safety practices will end up being compared to HSA sites having a wide spectrum of safety practices.

1. Biases of Analysis:

• Hindsight bias, sometimes also called the "knew it all along" phenomenon is a very potent bias, especially when it is based on knowing the actual outcome of an event.5 In 3rd party reviews, hindsight bias often results in analyses that are inappropriately critical of decisions or actions which seem simple in hindsight but which were difficult at the time.6,7 However this kind of hindsight bias can be controlled by:

1. Blinding reviewers to eventual outcome and/or adjusting apparent outcomes of cases under analysis to be similar
2. Matching surgical procedure and blinding reviewers to the practice setting of a case.

• On the other hand, there are other kinds of biases introduced into retrospective analyses which can reduce the level of criticism of decisions and actions:

1. Halo effect: The identity or reputation of the individual(s) or site being reviewed is known and/or those under review are higher in a hierarchy than those doing the review. Even for identical facts of a case, a bad outcome from a "reputable" site is likely to be judged better than a bad outcome from a less reputable site.
2. Reciprocity effect: The reviewers may find themselves judged by those under review at some later date and occasion and thus will soften their criticism lest they themselves be harshly criticized in the future.
3. Techniques of analysis: The techniques and policies of the review can preclude in-depth investigation or analysis of the events, and lead to accepting prematurely a conclusion that exonerates the practitioner.
4. Politics: Reviewers may have conscious or unconscious biases favoring a given political or policy outcome that will be affected by the results of their review.

Many of these biases can be handled by blinding reviewers to the identity and reputation of the site, excluding information on the ultimate outcome unless absolutely necessary, excluding reviewers with a known conflict of interest, and by establishing careful procedures for the review.

Collecting information is useless if it cannot be analyzed to good purpose. Although outcomes research is focused on collecting quantitative data on patient outcomes, another approach has proven equally useful in other high-hazard industries. This approach assumes that attempting to count outcomes (especially rare but serious ones) may be pointless, and that more is gained from the intensive analysis of even a small number of events, focusing on the underlying mechanisms of adverse events and ways to prevent or mitigate them. The adequacy of data in self-reports can vary greatly. If reports are anonymous there is no way to seek additional information and no useful analysis can be made from many reports. The NASA Aviation Safety Reporting System (ASRS) of voluntary reports of near-misses uses identified reports that are maintained as confidential until all necessary information and elaboration has been received, after which they are de-identified (rendering them essentially anonymous).^8,9 An equivalent system to the ASRS has been proposed by the APSF for anesthesiology but was never started because of the medicolegal complexities involved.¹⁰ Systems in analogy to the NASA ASRS are under construction within the Veterans Health Administration, within the Food and Drug Administration,² and further development is encouraged by the IOM report.³ In fact, the IOM report recommends legislation to provide protection for voluntary reports of near-misses and foresees the development protected incident reporting systems within the next five years.³ These systems would use specially trained analysts (probably retired or volunteer clinicians) to scrutinize the incident reports in order to come up with recommendations for improvement. Because reporting can be confidential and protected, individuals can disclose facts of near-misses with less fear of medicolegal ramifications. On the other hand, it is not always clear when an incident is merely a "near-miss." A patient may suffer an apparent near-miss, appear fine at the end of a procedure and be sent home, only to later develop an adverse outcome that might be linked (correctly or incorrectly) to the incident.

In aviation, serious events (accidents) are investigated intensively by a neutral expert panel (the National Transportation Safety Board). Can anesthesia safety in HSA and OBA settings benefit from similar analysis techniques? The NTSB is successful because aviation accidents are rare, unambiguous (in most cases it is clear that an accident rather than a near-miss has occurred and the outcome is never "normal"), and public (the event cannot be hidden). This allows the NTSB to wield its investigation machinery on a small number of events of which they will surely be informed and which they know must be looked into.

In health care, the situation is different. Catastrophic adverse events (death or cardiac arrest) are also rare, but lesser complications occur much more frequently. Many adverse outcomes are due to underlying disease or to known, typically unpredictable complications of care (e.g. post-operative wound infection). Finally, adverse events can easily be hidden in many cases since they are witnessed only by those involved (who have a significant incentive not to discuss them) or are remote from the procedure, allowing any causal connection to be dismissed. This would put an equivalent anesthesia safety board in the position of being bombarded with hundreds of cases per day with adverse events that might or might not need investigation.

The OBA setting may be unique. Patients having OBA procedures should be relatively healthy and will be unlikely to naturally suffer an immediate serious adverse event (e.g. death, cardiac arrest, neurologic damage, or medical complication requiring ICU admission). The number of such occurrences nationwide is probably only a few per day at most. It is thus conceivable that from the standpoint of logistics, a neutral investigation of each such event could be undertaken. However, due to the medicolegal issues detailed above, obtaining immediate knowledge of the event and access to the site and relevant information would be extremely difficult. In fact, there could be a substantial risk that, in order to avoid public disclosure of an adverse event, clinicians in an OBA setting might delay or avoid calling emergency medical response (i.e. calling 911), attempting to treat an evolving problem without sufficient expertise or equipment. The potential harm to patient safety should this occur would not be outweighed by the information gained for the future.

Another problem is that in anesthesia care, there is not yet a routine equivalent to the flight data recorders and cockpit voice recorders in use in commercial aviation. The installation of such equipment in HSA or OBA environments is very unlikely for a variety of privacy, medicolegal, and economic reasons. Thus, the amount of objective data available about a medical event may be far less than that available to the NTSB about an aviation event and this lack of data would be expected to be worse in the OBA setting than in the HSA setting because the economics of OBA in particular disfavor installation of such recording equipment. Non-objective data such as recollection and testimony are distorted by a variety of effects and by self-interest of the parties present. Therefore, NTSB-like investigations may not be feasible even if the legal and logistical obstacles could be cleared.

There is little question that unannounced inspections or even covert inspections can more accurately reflect actual work practices than can pre-announced inspections that allow safe practices to be demonstrated only for the benefit of the inspectors. In a covert inspection, an inspector might, for example, pose as a potential (but unusually inquisitive) patient to assess the characteristics of a site. Such levels of intrusion are rarely utilized at present in health care regulation even for hospital and surgicenter sites. Such inspections would probably not achieve wide support, but that means that sites with poor practices might easily escape detection.

Each of the techniques has advantages and disadvantages.

• Surveys may be a good technique for assessing the spectrum of procedures being done and of the anesthetic requirements. Due to response selection bias and biases encouraging "favorable" answers, surveys are less likely to accurately reveal the spectrum of safety practice levels of facilities and procedures or of problem situations.
• Inspections or audits may better reveal the spectrum of physical plant and written procedures, but unless they are covert they may not reveal the spectrum of safety practices actually conducted in real patient care. They cannot adequately document problems occurring during patient care.
• Epidemiologic outcome studies, especially prospective studies, would collect accurate data on problem situations. However, even in settings exhibiting poor safety practices the incidence of problems would be low, making such data collection expensive and time-consuming. Further, as detailed below, such studies require substantial cooperation from the sites leading to a large selection bias. If record-keeping were equal at all sites, retrospective chart reviews would be effective and less expensive. However, there is a strong likelihood of selection bias here both for the adequacy of record-keeping and for willingness to participate at all.
• Analysis of case reports may be valuable. There is wide agreement however that the analysis of case-reports of both actual adverse events and of near-misses can generate useful qualitative information leading to improved patient care. However, collection of case reports in general cannot yield data on the frequency of occurrence of problems because of uneven rates of reporting (although this might be possible for occurrences that cannot be hidden such as death or cardiac arrest).
• Analysis of closed malpractice claims. As for case reports these cannot yield frequency of occurrence data but can generate useful information leading to improved patient care. There is a very long lag time (several years) between any occurrence and the closure of the claim, followed by the time required for analysis.

Improving patient safety in office-based anesthesia will require the collection of data regarding current and future practices in this arena and to assess the types of problems encountered in this setting in order to define preventive and mitigating strategies. Given the various trade-offs between data accuracy, validity, impact, and difficulty of acquisition, the following data collection strategy can serve as a starting point for discussion:

1. Conduct surveys of OBA & office-based surgical practitioners, staff, and patients to determine the spectrum of procedures being conducted in OBA settings and the types of facilities and practices encountered. Surveys are inexpensive, simple, and easy to administer to large groups of individuals. Surveys can be conducted anonymously. The biases of reporter self-selection can be controlled in several ways.
2. Collect and analyze reports of problems in OBA and HSA that come to public attention whether via case reports in the medical literature or via the popular media. Although these reports may be incomplete (especially those obtained from the media) enough safety-related information may be gleaned from them that can allow useful ameliorative strategies to be disseminated.
3. Encourage the establishment and utilization of data from protected voluntary safety reporting systems. Again, although data will suffer from selection bias and incomplete reporting, the analysis of those cases that are reported may generate important ameliorative strategies.
4. Incorporate OBA cases as a specific focus of the ASA Closed Claims Study. This study has already provided important information about patient care in HSA settings and can extend this significantly to cases arising from OBA settings as they begin to come in. It may also be useful to compare the analysis of the same cases based on public media data with that from closed claims data. The ASA Closed Claims Study group and processes are already in place and thus should easily be used on OBA setting cases. On the downside, there will be a substantial time lag in acquiring a substantial body of such cases for closed claim analysis.
5. Consider long-term focused prospective and retrospective studies of matched patients having equivalent procedures in OBA vs. HSA settings. In designing these studies it will be important to carefully account for the relevant biases. Due to selection bias it may have to be acknowledged that such studies can provide at best an "optimistic" view of the safety of OBA.
6. A feasibility study should be commissioned of the possibility of forming a central neutral investigative body analogous to the NTSB to analyze serious adverse events occurring in OBA settings.
7. If sufficient information and ameliorative actions cannot be generated by the above data collection measures, it may be necessary to incorporate more intrusive or onerous measures such as covert inspections.

1. Liang BA: Error in medicine: legal impediments to U.S. reform. J Health Polit Policy Law 1999; 24: 27-58
2. Gardner S, Flack M: Designing a medical device surveillance network: report to Congress. Washington, D.C., Food and Drug Administration, 1999.
3. Corrigan J, Kohn L, Donaldson M: To Err Is Human: Building a Safer Health System. Washington, D.C., National Academy Press, 1999.
4. Cheney FW: The American Society of Anesthesiologists Closed Claims Project: what have we learned, how has it affected practice, and how will affect practice in the future? Anesthesiology 1999; 91: 552-556.
5. Fischoff B: Hindsight`"foresight: The effect of outcome knowledge on judgment under uncertainty. J Exp Psych: Hum Perception Perf 1975; 1: 288-299.
6. Caplan RA, Posner K, Cheney FW: Effect of outcome on physician judgments of appropriateness of care. JAMA 1991; 265: 1957-1960.
7. Woods DD, Johannesen LJ, Cook RI, Sarter NB: Behind Human Error: Cognitive System, Computers, and Hindsight. Wright Patterson Air Force Base, Crew Systems Ergonomics Information Analysis Center, 1994.
8. Hardy R: Callback: NASA’s Aviation Safety Reporting System. Washington, D.C., Smithsonian Institution Press, 1990.
9. Reynard WD, Billings CE, Cheaney ES, Hardy R: The development of the NASA Aviation Safety Reporting System, NASA Reference Publication 1114. NASA, 1986
10. Gaba DM: Improving anesthesia safety by analyzing case reports. American Society of Anesthesiologists Newsletter 1993; 57: 9-12.