240 F. HARRELL, K. LEE AND D. MARK
MULTIVARIABLE PROGNOSTIC MODELS 241
242 F. HARRELL, K. LEE AND D. MARK
MULTIVARIABLE PROGNOSTIC MODELS 243
244 F. HARRELL, K. LEE AND D. MARK
MULTIVARIABLE PROGNOSTIC MODELS 245
246 F. HARRELL, K. LEE AND D. MARK
MULTIVARIABLE PROGNOSTIC MODELS 247
248 F. HARRELL, K. LEE AND D. MARK
MULTIVARIABLE PROGNOSTIC MODELS 249
Tutorials in Biostatistics Volume 1: Statistical Methods in Clinical Studies Edited by R. B. D’Agostino
? 2004 John Wiley & Sons, Ltd. ISBN: 0-470-02365-1
251
252 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 253
254 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 255
256 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 257
258 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 259
260 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 261
262 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 263
264 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 265
266 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 267
268 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 269
270 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 271
272 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 273
274 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 275
276 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 277
278 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 279
280 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 281
282 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 283
284 F. HARRELL ET AL.
CLINICAL PREDICTION MODEL FOR AN ORDINAL OUTCOME 285
286 F. HARRELL ET AL.
TUTORIAL IN BIOSTATISTICS
Using observational data to estimate prognosis: an example
using a coronary artery disease registry
Elizabeth R. DeLong1;∗;†, Charlotte L. Nelson2, John B. Wong3, David B. Pryor4,
Eric D. Peterson2, Kerry L. Lee5, Daniel B. Mark2, Robert M. Cali 6
and Stephen G. Pauker3
1Outcomes Research & Assessment Group; Duke Clinical Research Institute; Duke University;
Department of Medicine; Biometry Division; Community and Family Medicine; 2400 Pratt Street;
Durham; NC 27705; U.S.A.
2Outcomes Research & Assessment Group; Duke Clinical Research Institute; Duke University;
Department of Medicine; Durham; NC 27710-7510; U.S.A.
3New England Medical Center; Department of Medicine; Box 302; Boston; MA 02111; U.S.A.
4Allina Health System; P.O. Box 9310; Minneapolis; MN 55440-9310; U.S.A.
5Duke Clinical Research Institute; Biometry Division; Community and Family Medicine;
Box 3363; Durham; NC 27710-7510; U.S.A.
6Duke Clinical Research Institute; Duke University Division of Cardiology and Department of Medicine;
Box 31123; Durham; NC 27710-7510; U.S.A.
SUMMARY
With the proliferation of clinical data registries and the rising expense of clinical trials, observational
data sources are increasingly providing evidence for clinical decision making. These data are viewed
as complementary to randomized clinical trials (RCT). While not as rigorous a methodological design,
observational studies yield important information about e ectiveness of treatment, as compared with the
e cacy results of RCTs. In addition, these studies often have the advantage of providing longer-term
follow-up, beyond that of clinical trials. Hence, they are useful for assessing and comparing patients’
long-term prognosis under di erent treatment strategies. For patients with coronary artery disease, many
observational comparisons have focused on medical therapy versus interventional procedures. In addition
to the well-studied problem of treatment selection bias (which is not the focus of the present study),
three signiÿcant methodological problems must be addressed in the analysis of these data: (i) desig-
nation of the therapeutic arms in the presence of early deaths, withdrawals, and treatment cross-overs;
(ii) identiÿcation of an equitable starting point for attributing survival time; (iii) site to site variability
in short-term mortality. This paper discusses these issues and suggests strategies to deal with them.
A proposed methodology is developed, applied and evaluated on a large observational database that has
long-term follow-up on nearly 10 000 patients. Copyright ? 2001 John Wiley & Sons, Ltd.
∗Correspondence to: Elizabeth DeLong, Outcomes Research & Assessment Group, Duke Clinical Research Institute,
Duke University, Department of Medicine, Biometry Division, Community and Family Medicine, 2400 Pratt
Street, Durham, NC 27705, U.S.A.
†E-mail: [email protected]
Contract=grant sponsor: Agency for Health Care Policy and Research; contract=grant numbers: HS08805, HS06503
Tutorials in Biostatistics Volume 1: Statistical Methods in Clinical Studies Edited by R. B. D’Agostino
? 2004 John Wiley & Sons, Ltd. ISBN: 0-470-02365-1
287
288 E. R. DELONG ET AL.
1. INTRODUCTION
One of the most important aspects of clinical decision making is selecting treatment strategies
for individual patients. A patient’s general health, demographic status and disease severity
will in uence both the choice of therapy and the prognosis. For patients with coronary artery
disease (CAD), the patient’s risk proÿle can determine whether a more invasive, and costly,
strategy is indicated. Because CAD accounts for a major portion of cardiovascular disease
(the U.S.A.’s number one cause of death, with incurred costs exceeding $150 billion annually
[1]), these decisions are relevant to society as well as to individual patients.
This study addresses some of the methodological issues in using observational data to
create prognostic models for CAD patients under di erent therapeutic options. Currently, the
three primary options after diagnosis by cardiac catheterization are medical therapy (MED),
percutaneous transluminal coronary angioplasty (PTCA), and coronary artery bypass graft
surgery (CABG), the latter two being revascularization procedures.
Ideally, data to assess the in uence of treatment strategy on prognosis would come from
randomized trials. However, these trials are expensive and often have limited sample size and
follow-up. Further, they cannot always be generalized to the broader spectrum of patients and
practice settings. Thus, observational data must sometimes supply information for medical
decision-making. In this case, the choice of therapy, the prognosis, and the relative survival
beneÿt depend to a great extent on the patient’s risk proÿle. In particular, because treatment
groups are not necessarily comparable prior to treatment, quantitative statistical models that
attempt to account for treatment selection bias while estimating survival under alternative
treatment strategies are needed. In addition to the selection bias that is inherent in treatment
comparisons with such data, a number of other issues are relevant, and these additional issues
are the focus of this manuscript.
1.1. Issues in assessing prognosis for CAD patients using observational data sources
A fundamental di culty when using observational data to estimate prognosis for CAD pa-
tients involves deÿning treatment-speciÿc survival. Physicians and patients who initially select
a less invasive treatment option understand that later cross-over to a more invasive alterna-
tive is possible. For example, a patient may begin treatment with medical therapy, but later
undergo CABG. The initial post-catheterization treatment ‘strategy’ incorporates this potential
change in treatment. The prognosis that includes survival from treatment initiation through
any subsequent cross-overs will be designated as arising from a ‘treatment strategy’ perspec-
tive. This perspective is in distinction to the ‘single treatment’ perspective, which evaluates
prognosis while receiving only the initial treatment and censors at any cross-over, such as
expected survival while on MED (described further in Section 1.2).
In randomized trials, treatment assignment is unbiased and survival time is initiated at ran-
domization. Observational studies of CAD patients, however, often lack an explicitly recorded
treatment assignment and thus have no uniformly logical treatment initiation time. Although
the treatment decision occurs soon after the catheterization, it is generally not recorded in
the observational data set. Furthermore, personal reasons or scheduling di culties can delay
the actual performance of a procedure for several days; consequently, some patients could
be lost to follow-up if they go elsewhere for procedures or some may die while awaiting
revascularization. For such patients, if no revascularization procedure has occurred following
ESTIMATING PROGNOSIS USING OBSERVATIONAL DATA 289
catheterization, a default assignment to the medical therapy arm would attribute both early
deaths and early losses to follow-up to MED. This policy assumes medical survival time
begins at catheterization, whereas procedural survival begins at the procedure date. Using
this convention and assuming the patient has survived the catheterization procedure, medi-
cal survival is unconditional, but procedural survival is implicitly conditioned on surviving
to receive the procedure. This problem is similar to one described in transplantation liter-
ature. When patients who do not survive long enough to undergo transplantation are, in
analyses, assigned to the non-transplant group, this creates a selection ‘waiting time’ bias in
favour of the transplantation group. The methodological problem of when to begin ‘survival
time’ has led to a serious and sustained debate questioning if heart transplantation survival
beneÿts may have been caused by selection bias [2–5]. With the pace of care increasing,
the delay from catheterization to PTCA is often far shorter than the delay from catheteri-
zation to CABG. Hence the potential ‘waiting time bias’ is particularly favourable toward
CABG. This dilemma underscores the need for establishing an equitable ‘treatment initiation’
time.
A further issue in assessing prognosis is that peri-procedural care (usually deÿned as the
ÿrst 30 days following a procedure) and long-term care are generally handled di erently and
by di erent types of care providers. The early survival after catheterization and treatment
assignment is a variable component of overall survival that depends to a great extent on insti-
tutional constraints and individual care providers. This ‘problem’ also exists when physicians
wish to make decisions based on RCT data. It is especially true for CABG and PTCA, which
depend on the skill of the operator. Hence, local e ects on early mortality need to be con-
sidered in long-term prognosis. Also, the factors that are important determinants of this early
risk may di er from those a ecting long-term survival.
An additional statistical concern is the di erential risk associated with alternative treatments
in the early period. CABG is known to incur a much higher early risk than either PTCA or
MED, but this risk declines sharply in the ÿrst few days after the surgery. Because the early
hazards for the three treatments are not proportional, a simple proportional hazards survival
model cannot be used directly to obtain estimates of relative treatment hazards. Treatment
e ects in the later interval are more likely to conform to proportional hazards.
A survival model that allows the variable 30-day mortality component to be estimated in-
dependently (possibly locally) and then coupled with the long-term component may be used
to compare prognosis under di erent short-term scenarios. In addition, estimates of the condi-
tional survival (dependent on surviving this initial 30-day period) can be used by patients and
providers following a successful procedure, or by those who want data that is not in uenced
by the early mortality rate.
1.2. Previous approaches
Some of the earliest observational prognostic assessments for CAD patients came from the
CASS multi-site data registry [6, 7]. Acknowledging the di culties of determining treatment
assignment and exposure time, these studies performed analyses using several methods. One
comparative analysis between MED and CABG assigned patients to medical therapy unless
CABG was performed within an established time window. Patients who did not undergo
CABG and either died or were lost to follow-up before the average time to CABG were
excluded from analysis to avoid biasing the estimates against medical therapy survival. This