This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Dow, M. M. Search for Related Content Social Bookmarking                         What's this?

Saving the Theory: On Chi-Square Tests With Cross-Cultural Survey Data

Northwestern University

Using data from three well-known cross-cultural survey data sets that differ in size and geographical scope, this article examines the effects of spatial and/or cultural clustering of societies on the usual chi-square test of independence. Sample design effects for individ ual variables and for two-way contingency tables from all three data sets were found to be generally larger than unity, often consid erably so, suggesting that the associated chi-square test statistic var iances are considerably underestimated and that the risk of Type I errors is considerably greater than the 5% level usually assumed by researchers. This is especially true if variable categories have been collapsed to overcome small cell counts, a common occurrence in comparative research. These empirical findings imply that using chi-square tests with categorical variables from cross-cultural sur vey data sets is a relatively easy way to generate an excess of sta tistically significant results, thus increasing the likelihood of "sav ing the theory." The results also strongly suggest that the extensive reporting of naive chi-square independence tests using cross- cultural data sets over the past several decades has led to incorrect rejection of null hypotheses at levels much higher than the expected 5% rate. A reanalysis of chi-square-based inferences from a pre viously published study also supports this conclusion. A deflation factor is proposed for observed chi-squares that reduces the Type I error rates closer to the assumed levels.

Cross-Cultural Research, Vol. 27, No. 3-4, 247-276 (1993)
DOI: 10.1177/106939719302700305


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?