Operative Risk Assessment In Cardiac Surgery

Fred H. Edwards, M.D., Professor and Chief of the Division of Cardiothoracic Surgery, University of Florida / Jacksonville

Operative "risk assessment" has traditionally been determined by a subjective estimate. This estimate generally comes from a combination of personal experience and reported patient series in the surgical literature.

Each patient presents a unique array of operative risk factors. It is unlikely, of course, that we will find a large series of patients having exactly the same characteristics as an individual patient at hand. We may analyze literature reports of patients undergoing similar operations, but even though some matching risk factors may be found, it will be almost impossible to find an exact match. Even if a close match is found, it may not be appropriate to extrapolate the experience of that reporting institution to our own hospital. So, after all is said and done, we really guess at the predicted risk. This is hopefully an educated guess, but a guess nevertheless.

This method of risk assessment has served us well in the past, but today we have more reliable and objective techniques at our disposal. In cardiac surgery, these methods center around statistical risk models used in The Society of Thoracic Surgeons National Cardiac Surgery Database, commonly called the STS Database. It is worthwhile to review the evolution of this database.

The Society Of Thoracic Surgeons National Database

In the early 1980's, coronary bypass surgery (CABG) was characterized by operative mortality rates in the range of 1 - 2%. Five years later, however, urgent and emergent operations and "redo" procedures became common. Percutaneous revascularization absorbed low risk patients from the surgery pool. Predictably, the operative mortality predictably rose to the 5 - 6% range. More patients were leaving the hospital alive, but a significant number of patients that previously would have died on the cardiology service now were dying postoperatively on the surgical service.

This rise in operative mortality was understood by surgeons and cardiologists, but others were unaware of the changes that had produced these higher mortality rates. Quite appropriately, hospital administrators began to ask for justification of the observed increase in CABG mortality. This often prompted a time-consuming and expensive chart review. Even with this information, it was difficult to objectively determine the impact of these new and compelling risk factors.

It soon became apparent that the collection of raw data was inadequate. Proper analysis required risk models that could generate a predicted operative mortality based on results of an accepted standard of care ^1,2. Unfortunately, this type of analysis was unavailable to most surgical groups. The issue was further complicated when, in 1986, the Health Care Financing Administration (HCFA) released raw CABG operative mortality data into the public domain. This information promptly appeared in the press and suddenly surgeons found the operative results of their hospital in the newspaper. This was clearly an injustice to all concerned, but the ill-advised precedent had been set.

Recognizing the need for an objective national standard, the Society of Thoracic Surgeons responded by appointing a formal committee to develop a national cardiac surgery database. This committee, under the leadership of Dr. Richard E. Clark, was charged with the responsibility to gather and to analyze patient data in a manner that would establish a national standard of care in cardiac surgery. Most surgeons recognized the need for this database and voluntarily participated. Enrollment has steadily increased each successive year, so that now the STS Database contains records of just under two million patients, thereby making it the largest cardiac surgery database in the world. The database has come to be regarded as a valuable resource to cardiac surgeons and its risk assessment algorithms are generally accepted as standard benchmarks in coronary bypass surgery.

Risk Models

The STS Database is far more than a "bean counter." Certainly a large amount of demographic data is collected and presented as observational information, but this database is distinguished primarily by its use of sophisticated risk models.

Clearly there is a compelling need for an objective, accurate, reproducible and scientifically valid estimate of risk for an individual patient. Statistical risk models are designed to perform exactly this task. Figure 1 shows the typical design of a risk model. Individual patient risk factors are entered into the model, which then uses a statistical algorithm to calculate the probability that a given operative outcome will occur. It is important to note that the model determines the net impact of all risk factors for an individual patient. Risk models, then, provide us with an objective estimate of risk for an individual patient.

 

Figure 1. Risk Model Schematic. Individual patient risk factors are entered into the software program for mathematical manipulation by the statistical algorithm. Typically 20-30 risk factors are used to calculate the predicted surgical outcome.

Applications of the STS Database

Individual risk assessment

Risk factors for an individual patient can be directly entered into the standard software to calculate a predicted operative mortality associated with CABG. The model does not predict "survival" or "death", but rather provides an estimate of the probability of operative death. This predicted probability essentially tells the surgeon how a patient with similar comorbidity would be expected to fare based on a national standard of care.

For example, let us say that a predicted risk of 9 % is calculated. The following statement is then warranted: "Based on the aggregate national experience of centers enrolled in the STS Database, of every 100 patients presenting with similar risk factors, 9 would be expected to die after CABG." This has obvious applications in patient counseling and medical decision-making. It should be emphasized, however, that the predicted mortality does not dictate patient management. These results should be regarded as a single piece of information to be interpreted in concert with other, more traditional aspects of patient care.

Quality Improvement

The STS risk models can be used as powerful quality improvement tools. Thoracic surgeons have long recognized that operative outcomes cannot be properly analyzed using only raw operative mortality rates. Before comparing the results of one group against another, one must ensure that patients of equal risk are being compared. Models provide an objective way to carry out this type of risk-adjusted comparison [2]. In general, the risk model is used to calculate the predicted risk for all patients in the groups to be compared. Patient records in each group are then arranged in order of predicted risk and divided into several subsets. These subsets, then, consist of patients all having similar operative risk. This risk stratification allows subsets of one group to be compared against the corresponding subset of the other group so that a true "apples to apples" comparison can be made. With these matched subsets, one then statistically compares the actual mortality in each group.

This is the kind of risk-matched group comparison carried out in the STS system. One group is made up of the subscriber's patient population and the other group is the aggregate national population (which represents the standard of care). This approach allows the individual practice to obtain a risk-matched comparison against the national standard. Subscribers are given periodic reports in which their risk-adjusted results are compared against the national norm. If results are comparable to those of the STS Database, then one concludes that the surgical care is in keeping with an accepted standard. On the other hand, if the results significantly deviate from those of the database, a closer inspection seems in order. It is important to emphasize that being identified as an "outlier" is not a form of condemnation. The STS Database Committee does not consider "outlier" status as an invariable indication of substandard care, but it does imply that a local examination of operative results is appropriate.

Managed Care Responses

In recent years, it has become common to receive requests from managed care groups seeking detailed information on operative results. For example, in the mid-1990's, several cardiac surgery groups were notified by U.S. Healthcare that continued membership in their group would be predicated on a concise and accurate reporting system, as well as a risk assessment protocol to evaluate operative results. The STS Database was able to provide an ideal solution to these requirements, thereby allowing participating institutions to comply with the reporting mandate without expensive and time-intensive chart reviews or statistical consultations. It appears that managed care organizations have recognized the importance of risk stratification and may require this type of analysis as a condition for enrollment.

State Regulatory Requirements

Several state regulatory agencies have arbitrarily mandated that cardiac surgery centers submit morbidity and mortality data. In many instances it is apparent that the agencies have little expertise in this complex field and the administrative burdens placed on the surgeon have been both oppressive and non-productive. Fortunately, some state agencies have recognized the value of the STS Database and have allowed centers to provide patient information in the standard STS format. Other regulatory agencies have insisted on their own unique reporting system along with primitive risk algorithms designed by individuals with minimal understanding of cardiac surgery. Sadly, this approach has cost taxpayers untold thousands of dollars for no good reason. The STS has developed a formal program to assist states facing regulatory mandates of this sort. Ideally the STS and the cooperating state surgery centers will present a unified front encouraging state regulatory agencies to follow a more rational approach in which cardiac surgery results will be monitored using STS Database information.

Surgical Research

Risk models may also be used as powerful research tools, particularly in the field of "single factor analysis". In this type of study, the goal is to determine the influence of a single factor on operative outcome. The study group is divided into one population in which the factor is present and another in which it is absent. Models are used to account for the impact of all significant risk factors to create risk-matched subgroups in each of the two populations. These matched subgroups will then differ only in the presence or absence of the factor being investigated, so any significant difference in outcome will necessarily be attributed to the influence of that factor.

The STS Database has been carefully studied to identify clinically useful information of importance in cardiac surgery. Single factor analysis has been used in conjunction with the STS risk models to isolate the influence of internal mammary artery grafts³ and to investigate the impact of gender⁴ on CABG outcome. In each of these studies, the protocol described above was used to definitively isolate the investigated factor to determine its influence on operative mortality.

Future Direction

Future efforts will focus on the development of risk models to predict serious operative morbidity as well as mortality. Models to predict outcomes associated with valve operations have recently been added to the standard software⁵. Congenital cardiac surgical procedures are being analyzed as a separate module of the database. In the future, the STS Database is likely to serve as an extraordinarily valuable national resource that will have considerable impact on the practice of cardiothoracic surgery.

References

1. Hannan EL, Kilburn H Jr, O'Donnell JF, et al: Adult open heart surgery in New York State: an analysis of risk factors and hospital mortality rates. JAMA 264:2768-72, 1990.
2. Kouchoukos NT, Ebert PA, Grover FL, and Lindesmith GG: Report of the ad hoc committee on risk factors for coronary artery bypass surgery. Ann Thorac Surg 45:348-9, 1988.
3. Edwards FH, Clark RE, Schwartz M: The impact of internal mammary artery conduits on operative mortality in coronary revascularization. Ann Thorac Surg 57:27-32, 1994.
4. Edwards FH, Carey JS, Grover FL, Bero JW, and Hartz RS: The impact of gender on coronary bypass operative mortality. Ann Thorac Surg 66:125-31, 1998.
5. Edwards FH, Peterson ED, Coombs LP, DeLong ER, Jamieson WRE , Shroyer AL, Grover FL: Prediction of operative mortality after valve replacement surgery. J Am Coll Cardiol 2001:37;885-92.

Jacksonville Medicine / October 2001

What's New · Northeast Florida Medicine Journal · Know Your Physician · Legal & Legislative
· DCMS Alliance · Academy of Medicine · Member Websites · Community Health
About the DCMS · Meetings Calendar · Member Benefits · Employment Connection · Home

Duval County Medical Society   ·   555 Bishopgate Lane  ·   Jacksonville, FL  32204
Phone: (904) 355-6561   ·     FAX:  (904) 353-5848   
General Email: dcms@dcmsonline.org    ·   Webmaster's Email: mdoran@dcmsonline.org
Privacy Policy and Disclaimers