Geography, Statistics, and Ecological Inference

Book Review Forum 601

Gould, P. 1970. Is Statistix Inferens the Geographical lations: Spatial Dependence and Regional
Name for a Wild Goose? Economic Geography 46 Context in Africa, 1966–1978. International In-
(Supp.):439 – 48. teractions 17:29–61.
———; Flint, C.; and Anselin, L. 1994. The Geogra-
Hänisch, D. 1988. Wahl-und Sozialdaten der Kreise und phy of the Nazi Vote: Context, Confession,
Gemeinden des Deutschen Reiches, 1920–1933: and Class in the Reichstag Election of 1930.
Benutzer-Handbuch. Berlin: Zentralinstitut für Annals of the Association of American Geographers
sozialissenschaftliche Forschung, Freie Univer- 84:351 – 80.
sität Berlin. ———; Flint, C.; and Shin, M. 1995. Regions and
Milieux in Weimar Germany: The Nazi Party
Jones, J.P., and Casetti, E., eds. 1992. Applications of Vote of 1930 in Geographic Perspective. Erd-
the Expansion Method. New York : Routledge. kunde 49:305–14 (2 map supps.)
———; Kolossov, V.; and Vendina, O. 1997. The
King, G. 1996. Why Context Should Not Count. Po- Electoral Geographies of a Polarizing City: Mos-
litical Geography 15:159–64. cow, 1993–1996. Post-Soviet Geography and Eco-
nomics 38:567–600.
———. 1997. A Solution to the Ecological Inference Owen, G., and Grofman, B. 1997. Estimating the
Problem: Reconstructing Individual Behavior from Likelihood of Fallacious Ecological Inference:
Aggregate Data. Princeton, NJ: Princeton Uni- Linear Ecological Regression in the Presence of
versity Press. Context Effects. Political Geography 16:675–90.
Pattie, C., and Johnston, R. 2000. “People Who Talk
———. 1999. The Future of Ecological Inference Re- Together Vote Together”: An Exploration of
search: A Reply to Freedman et al. Journal of the Contextual Effects in Great Britain. Annals of
American Statistical Association 94:352–355. the American Geographers 90:41–66.
Robinson, W.S. 1950. Ecological Correlation and the
Lohmöller, J.-B.; Falter, J.; Link, A.; and de Rijke, J. Behavior of Individuals. American Sociological
1985. Unemployment and the Rise of National Review 15:351–57.
Socialism: Contradicting Results from Different
Regional Aggregations. In Measuring the Unmea-
surable, ed. P. Nijkamp, H. Leitner, and N. Wrig-
ley, pp. 357–70. The Hague: Martinus Nijhoff.

O’Loughlin, J., and Anselin, L. 1991. Bringing Geog-
raphy Back to the Study of International Re-

Geography, Statistics, and Ecological Inference

Gary King
Department of Government, Harvard University

I am grateful for such thoughtful reviews offered “the” solution (as some reviewers and
from these three distinguished geographers. commentators have portrayed it), “a” solution
Fotheringham provides an excellent summary of (as I tried to describe it), or, perhaps better and
the approach offered, including how it com- more simply, as an improved method of ecologi-
bines the two methods that have dominated cal inference, is not important. The point is that
applications (and methodological analysis) for more data are better, and this method incorpo-
nearly half a century—the method of bounds rates more. I am gratified that all three reviewers
(Duncan and Davis 1953) and Goodman’s (1953) seem to support these basic points. In this re-
least squares regression. Since Goodman’s re- sponse, I clarify a few points, correct some mis-
gression is the only method of ecological inference understandings, and present additional evi-
widely used in Geography (O’Loughlin), adding dence. I conclude with some possible directions
information that is known to be true from the for future research.
method of bounds (for each observation) would
seem to have the chance to improve a lot of re- Ecological Inference as
search in this field. The other addition that EI Statistical Inference
provides is estimates at the lowest level of geog-
raphy available, making it possible to map re- John O’Loughlin argues that “the specific
sults, instead of giving only single summary problems of geographic data analysis require a
numbers for the entire geographic region. different mode of thinking than is usually found
Whether one considers the combined method

602 Book Review Forum

in inferential statistics” (p. 595). Although I problem,” or even as estimating quantities that
recognize the place-space divide he is referenc- may not exist. Language can be fun, but none of
ing here, inferential statistics should not be seen this should take away from the very real contri-
as a threat to anyone interested in learning bution these models make in enabling scholars
about the world. It is fully general and capable of to draw inferences about unknown spatial auto-
evaluating any logically consistent statement correlation parameters from known observa-
with observable implications. No other ap- tions of geographically coded variables. In sum,
proach dominates it, even when only qualita- models of ecological inference are not unique in
tive information is available (King et al. 1994). helping us learn about facts we do not know.
In this section, I address this point in the con- Whether this is “creating data where none exist”
text of ways the reviewers have characterized is a small semantic point. Whatever language
the ecological inference problem. one chooses, it applies to all types of inference.

What Is Inference? Evaluating Model-Based Inferences

Anselin correctly points out that the quanti- Ecological inference and most other statisti-
ties of interest in ecological inference are not cal analyses in the social sciences are model-
observable. Although they were originally ob- based, meaning that inferences depend on data
servable, these quantities have been lost in the as well as statistical assumptions. This means
process of aggregation. Unfortunately, he greatly that, Anselin’s claims notwithstanding, there
confuses matters when he describes ecological are indeed several “yardstick[s] to compare com-
inference as “creat[ing] data where no data peting methods” (p. 587) of ecological infer-
exist” (p. 587). In fact, all problems of statistical ence. They are the same yardsticks used for vir-
inference share exactly this feature. tually all statistical methods. For starters, we can
determine whether estimates have attractive
To be specific, inference is the process of statistical properties when the assumptions are
using facts we know to learn about facts we do correct. Anselin seems to apply this approach
not know.1 In ecological inference, the facts we when he concludes that “there is no overarching
do not know are the contents of the cells in a set statistical framework to encompass these meth-
of contingency tables. In forecasting, the facts ods (such as asymptotics, or normal distribution
we do not know are future values of our outcome finite samples)” (p. 588). He also recognizes, in a
variable. In descriptive inference, the facts we slightly different context, that the method “is
do not know are population parameters (which firmly grounded in modern statistical concepts
we would seek to learn using observations taken familiar in the literatures dealing with Bayesian
from some type of sample). In causal inference, hierarchical modeling, . . .” (p. 590). In fact,
the fact we do not know is the average differ- the latter statement, which is correct, can be
ence between the dependent variable for a unit used to correct the first, which is not. That is,
(respondent, country, etc.) when a “treatment” Bayesian models like EI have the full range of
is applied and the same unit when a “control” is desirable statistical properties. For example, ag-
applied, the problem being that only one value gregate estimates conditional on the assumptions
of the dependent variable is observed for any are admissible, statistically consistent, asymptot-
unit (this is known as the Fundamental Problem ically normal, and asymptotically efficient (the
of Causal Inference; see Holland 1986; King et same as, say, logistic regression or probit). The
al. 1994). extensive Monte Carlo evidence presented in
the book demonstrates that estimates are ap-
In another context, Anselin (1988) artfully proximately unbiased and efficient in samples as
reviews a variety of methods of making infer- small as twenty-five (something not true of lo-
ences about spatial autocorrelation parameters. gistic regression, for example). The model can
Since no one has ever observed a spatial auto- also be justified by appeal to likelihood theory,
correlation parameter in the real world, we since priors are not necessary.
could equally describe his methods as “creating
data where none exist,” being “essentially un- A second way to evaluate model-based infer-
verifiable” (p. 587), “observationally equiva- ence is to evaluate the properties of the esti-
lent” (p. 588) to other contradictory models, mators when the assumptions are wrong. As
“not a solution to the [spatial autocorrelation]

Book Review Forum 603

O’Loughlin and Fotheringham point out, my Ecological inferences involve nothing myste-
book contains a forty-page section about what rious or even unusual from a theoretical statisti-
can go wrong with these assumptions. The con- cal perspective. They share the same character-
clusion from this exploration by me and others istics as all other model-based inferences. What
is that the method is more robust to incorrect makes ecological inference an especially hazard-
assumptions than Goodman’s regression, but ous process is that we do not always have suffi-
the degree of robustness is specific to the data cient external information to be comfortable with
chosen. Sometimes including the information the assumptions of the model. The fix, for eco-
from the bounds provides enormous amounts logical inference and all other such inferences,
of additional information; other times the is to gather this information. Fortunately, geog-
bounds do not add much. Fortunately, we can raphers are in an especially good position to do so.
often tell which situation we are in, and there is
little chance of doing worse by using additional Consequences of Ignoring
information. Spatial Autocorrelation

Third, observable implications of the model In my book (x 9.3.1), I found that spatial au-
can be checked to assess the fit. Although infor- tocorrelation had minimal effects on estimates
mation is lost by the process of aggregation, and standard errors from the model. This re-
some observable implications do indeed exist sult was found to be “particularly interesting”
and can be used to rule out some models. It by Fotheringham (p. 585) but the method by
would be nice if there were more (just as it which I arrived at this conclusion was ques-
would be nice if there were no information lost tioned by the other two reviewers because I
to aggregation), but at least there are some, and evaluated the model under a relatively simple
they should be used. Indeed, the book describes, version of spatial autocorrelation. As a result,
and the software implements, many diagnostics O’Loughlin writes that “King’s spatial depen-
that take advantage of these observable implica- dency simulations are unconvincing.” Anselin
tions. I think all the reviewers recognize the agrees and explains that the right Monte Carlo
value of these kinds of diagnostics, though experiments should include “the simultaneity
Anselin worries that they require “nonrigorous induced by multidirectionality and two-dimen-
visual interpretation” (p. 591). The book describes sionality (Anselin 1988). It remains to be seen
and the software includes formal hypothesis how real spatial autocorrelation would affect the
tests for aggregation bias (based on the co- results of EI” (p. 590).
efficient on X in the mean function), but I do
prefer graphical diagnostics. This is in part a These are reasonable criticisms, and I am
matter of taste: Anselin’s view is that formal grateful for the opportunity they present to ex-
statistical tests are preferable since the decisions pand on the properties of EI. For this response, I
drawn are more clear-cut, and because of the ran another Monte Carlo experiment according
problems uncovered in the literature on cartog- to the specifications in Anselin (1988). In order
raphy and perception. Others prefer graphical to ensure a realistic form of multidirectionality
diagnostics, however, since they convey more and two-dimensionality, I used a spatial contigu-
information, consider a larger range of observ- ity matrix coded from all nations in the world
able implications, and have the ability to reveal (with at least one neighbor). So as to avoid con-
features of the substantive problem and data founding, the data were generated so that the
we did not realize to look for in the first place. other (distributional and correlational) assump-
Of course, graphical tests are less precise and tions of the model were satisfied. I used the same
leave more open to interpretation, and so graph- number of simulated datasets (250) as in my
ics and statistical tests can both be useful tools. book.
As a general matter, graphical tests are better
when we know less; formal statistical tests are The results, reported in Table 1, confirm the
better when we are confident of all aspects of conclusions from the simpler analysis presented
the model other than the one being tested (and in my book: Spatial autocorrelation has only a
are invalid otherwise). For ecological inference, minimal effect on model estimates and standard
I would prefer to assume we know less, but I errors. For example, the average absolute error
would welcome any development of other for- of the aggregate (global) quantity of interest is
mal theories. approximately zero for both the independently

604 Book Review Forum

Table 1. Consequences of Spatial Autocorrelation: Monte Carlo Evidence

Aggregate-Level Precinct-level
(S.D.)
Model Error Avg. S.E. Error (S.D.)

Independent .007 (.011) .014 .01 (.05)
Spatial .008 (.011) .014 .01 (.04)

Each row summarizes 250 simulations drawn from a model with areal units that are either independent or spatially autocorre-
lated, as described in the text. The aggregate-level columns report the average absolute difference between the truth and esti-
mates from the model. The precinct-level columns give the deviation from 80 percent in coverage of the 80 percent confidence
intervals (in both cases with standard deviations across simulations in parentheses).

generated and spatially autocorrelated data. The this contextual knowledge than geography. I en-
true standard deviation across the 250 simula- courage geographers to bring their skills to bear
tions (given in parentheses) is fairly close to the on this important question for their own re-
average of the estimated standard errors from search and for those in numerous other disci-
each simulation, indicating that the aggregate plines and nondisciplinary areas.
uncertainty estimates are reasonably accurate in
this case. In addition, the average error in cover- For example, John O’Loughlin approaches
ing the true values for the 80 percent-confidence the problem from one angle by writing that
intervals is very small for the simulations both “Only careful survey research that incorporates
with and without spatial autocorrelation. specific contextual questions will resolve the po-
litical science-political geography difference of
Since information is lost in the process of ag- opinion on the nature and significance of con-
gregation, using all information in the data and text” (p. 600). I agree about the benefits of in-
adding additional qualitative, geographical, and cluding aggregate and contextual information
other external information is the only way to be in public-opinion surveys (and have indeed ar-
reasonably confident when making ecological gued for this position in other contexts; see King
inferences. I welcome the suggestions of all three 1996), but this is only half the story. That is, sur-
authors about additional geographical informa- vey research is a very powerful tool, but even the
tion we might include in the analysis to further best surveys will not resolve the problem alone.
improve our inferences. I would welcome at- For example, I doubt anyone would want to
tempts to extend the model offered to incorpo- mount a survey in present-day Germany to ask
rate this information, along with analyses like survivors whether they voted for Hitler in the
the one in Table 1 that verify whether or not 1930s and 1940s! Indeed, even in the circum-
they make a difference. The additional evidence stances where survey responses are most likely to
offered here supports my conclusion in the book be sincere, no survey includes enough respon-
that spatial autocorrelation has minimal effects dents to provide good estimates for local areas.
on other aspects of this model.
The other half of the story is to use survey
Future Directions data to improve methods of ecological infer-
ence. There has been some other work on this
The time has never been better for develop- (e.g., Little and Wu 1991), but important op-
ing improved methods of ecological inference. portunities remain. In particular, what surveys
More scholars from more disciplines are working are good at, in practice, is providing a good
on improving and applying methods of ecologi- snapshot of averages over an entire country. For-
cal inference than ever before. As a result, more tunately, this type of information is precisely
methodological progress has been made in the what we need in order to relax some key meth-
last few years than at any time since Goodman odological assumptions in making ecological in-
and Duncan-Davis were writing. The most pro- ferences. For example, even a relatively small
ductive future paths will almost surely be those survey can greatly improve the estimation of the
that bring in the most quantitative and qualita- five key parameters in the basic version of my
tive knowledge external to the data at hand, and model, or the degree and direction of aggrega-
it is hard to think of a discipline with more of tion bias, and once we include this information,
estimating all the precinct-level parameters is
far less dependent on assumptions.

Book Review Forum 605

A second fertile area for future work is mod- continuous variables, and a dozen others have
eling the special types of measurement error in written articles pushing forward other aspects of
ecological data. Anselin raises one possibility the problem. As Fotheringham reemphasizes,
when he expresses concern about “sampling error further work developing aggregation-invariant
associated with Ti” and suggests that we model statistics will provide additional help in avoid-
this with binomial random variables. As it turns ing the Modifiable Areal Unit Problem in new
out, this has been tried in the literature, in areas. To further encourage this research, I make
Brown and Payne (1986) and King et al. (1999), the following offer: Develop a better method of
from which it is clear that including binomial ecological inference—a test, diagnostic, or other
variation is substantively meaningless in most related tool—demonstrate its value, and provide
realistic political geography applications. For ex- some code, and I’ll include it in the EI and EzI,
ample, consider the variance of Ti for an aver- the widely used software packages that imple-
age-sized electoral precinct (the smallest unit of ment these methods (available at http://GKing.
political geography in the U.S.). Such a pre- Harvard.edu).
cinct (with, say, 1000 people), evaluated at 0.5
probability (where the variance is largest), has a Acknowledgment
variance of only 0.00025. Of course, this should
not be surprising: sampling error is mostly a My thanks go to Luc Anselin, Stewart Fothering-
problem in sample surveys. Although aggregate ham, and John O’Loughlin for helpful comments and
data has far less sampling error, many other types conversations.
of measurement error need to be modeled. For
example, it would be very useful to develop Notes
models for errors induced when matching two
sets of data on overlapping geographies, or due 1. In Bayesian analysis, for example, the only two
to mismatched geocoding. As is, these are dealt types of quantities that exist are those that are
with only by cartographers and data providers known or unknown. Known quantities are fixed
and then ignored during statistical analyses. In numbers; unknown quantities are modeled via
fact, these are critical parts of the data genera- probable densities.
tion process but have not been modeled statisti-
cally in the context of ecological inference. 2. Precinct-level parameter estimates in EI have the
same properties as the aggregate estimates, except
The reviewers’ suggestions of using geograph- of course that their posteriors do not collapse
ically weighted regression, spatial econometrics, asymptotically (for the same reason that the non-
and spatial expansion are also promising general zero variance of Ϫ, ÿ 2, prevents the distribution
approaches, and useful for many specific prob- of y ϭ x Ϫ ϩ Ϫ from collapsing asymptotically in
lems, but of course they cannot be used without linear regression). A related confusion is Anselin’s
modification to make inferences about individu- claim that since EI has “2N parameters but only N
als from aggregate data. In the spirit of these sug- observations, estimation is clearly impossible”
gestions, however, I experimented with several (p. 589). This is an intuitive-sounding claim, but
extensions of the nonparametric method dis- it does not follow in ecological inference or in
cussed in the present work (x 9.3.2) with kernels other statistical models. For example, in least
defined on the basis of geographic rather than squares regression, with k explanatory variables
tomographic proximity. I have no doubt that (including the constant), there are k ϩ 1 parame-
this will help in some cases, but I could not find ters (i.e., the Ϫs and ÿ 2). But from these we can
a real example with political data where it no- estimate any number of conditional expectations
ticeably improved the inferences. This would (using fitted values from the regression). Similarly,
seem to be another productive avenue to follow. the basic version of EI has only five parameters,
from which we can easily estimate two (and in-
Other issues worth further exploration include deed many more, if desired) observation-level pa-
methods of model selection, exible specifica- rameters. Anselin’s misstatement is explained by
tions, methods for continuous individual-level the fact that one can estimate at most, N indepen-
variables, and explicit models of spatial autocor- dent parameters from N observations, whereas
relation. Ori Rosen, Martin Tanner, and I have precinct-level quantities of interest in ecological
been working on extensions to multiple cate- inference (and conditional expectations in least
gory variables, Philip Cross and Charles Manski squares regression) are not independent: in fact,
appear to be making progress on extensions to conditional on the data and one precinct-level

606 Book Review Forum

parameter, the other parameter may be computed Journal of the American Statistical Association
deterministically (see King 1997: 80, Equation 5.2). 81:945 – 60.
King, Gary. 1996. Elections and the National Elec-
References tion Studies. Paper prepared for the National
Election Studies, Congressional Elections Con-
Anselin, Luc. 1988. Spatial Econometrics: Methods and ference, copy available at http://gking.harvard.
Models. Kluwer Academic Publishers. edu/preprints.shtml#nes.
———; Keohane, Robert O.; and Verba, Sidney.
Brown, Philip J., and Payne, Clive D. 1986. Aggre- 1994. Designing Social Inquiry: Scientific Infer-
gate Data, Ecological Regression, and Voting ence in Qualitative Research. Princeton, NJ:
Transitions. Journal of the American Statistical As- Princeton University Press.
sociation 81:452–60. ———; Rosen, Ori; and Tanner, Martin A. 1999.
Binomial-Beta Hierarchical Models for Ecologi-
Duncan, O.D., and Davis, B. 1953. An Alternative to cal Inference. Sociological Methods and Research,
Ecological Correlation. American Sociological in press.
Review 18:665–66. Little, Roderick J.A., and Wu, Mei-Miau. 1991.
Models for Contingency Tables with Known
Goodman, Leo. 1953. Ecological Regressions and the Margins when Target and Sampled Populations
Behavior of Individuals. American Sociological Differ. Journal of the American Statistical Associa-
Review 18:663–66. tion 86:87–95.

Holland, Paul. 1986. Statistics and Causal Inference.