I applaud David Smilde and Matthew May for gathering the requisite data to evaluate some of the impressions sociologists of religion have about their field. Their analysis also provides some background for calls for a change of direction in the field, such as that found in “Toward a new sociology of religion,” coauthored by Peggy Levitt, Courtney Bender, Wendy Cadge, and David Smilde. We need to be clear about what is happening in the field before advocating any specific changes. To that end, I would like to look at Smilde and May’s findings through a thicker interpretive lens. The sociology of religion has actually changed very little in the past thirty years. For example, the number of religion articles in top journals, and the percentage of those articles focused specifically on Christianity, Protestantism, and the US have remained constant. While some of the changes reported in the paper are shown to be statistically significant, they are often so small as to be substantively insignificant. So, rather than reacting to a supposed change in the sub-discipline, we should instead be debating whether we want to change what is, by all appearances, a relatively stable field.
The Strong Program
One particular change that I would advocate is the continued cultivation of the “strong program.” The “strong program” refers primarily to the supposition that religion is not something to be explained in terms of external factors, but rather something that itself explains other phenomena. If this is a trend, then sociologists of religion have succeeded in convincing scholars in a wide range of sub-fields that religion matters. However, this finding should also be interpreted cautiously; as we see in Figure 2, the trend is flat for the first 20 years and tilts strongly upward only during the last four. These last four years, then, could be a blip in the data.
If such a change has occurred, it might be due less to having convinced other sociologists to examine religion than to the perception that the religious diversity of scholars has increased in recent years. This perception could lead to a declining tendency to treat religion as a dependent variable, and thus increasing interest in religion as an independent variable.
I remember one of the first bodies of literature that I encountered as a budding sociologist explained the rise of the religious Right among evangelicals as the result of “status discontent.” I wondered at the time what my graduate student colleagues who were evangelicals thought about this reductive handling of their beliefs. I suspect that many of the older articles in which religion was to be explained by other phenomena were motivated by the secularization theory-infused question, “why do people believe this stuff that we know is really something else?”
Of course, sociologists have always said that what the locals think is going on is not, in fact, what is going on. But I think that we’ve lost our taste for that intellectual move when it comes to a central component of identity like religion. Smilde and May describe something similar when they explain the positive portrayal of religion in less developed countries as due to the “romantic” sentiments of anthropologists who “admire the uniqueness and creativity of ‘cultural others.’” I think that sociologists now share this romanticism, and that this development dovetails with changing notions of civility in American culture, which dictate that explaining away someone’s sincere religious beliefs be viewed as disrespectful (especially if that person is on a panel with you at an SSSR meeting). Evidence for this interpretation can be ferreted out of Table 5, which shows that the one religious phenomenon that is still consistently explained is that of “new religious movements” (a.k.a. “cults”). People want to explain why anyone would ever join the Unification Church, but have lost their taste for explaining why anyone could be, for example, an evangelical. One still does not need to worry about encountering a “cult” member at SSSR meetings. If this is indeed what is going on, I would hope that we can get to a point where, within the bounds of civility, we can still investigate each others’ religious beliefs and associations, as such interrogation remains vital to the sociology of religion.
Pro-religious Research
There has been much buzz among sociologists of religion lately regarding claims of an increase in “pro-religious” research by people who are personally supportive of religion; and while Smilde and May do not derive any strong conclusions from the relevant data, much of the commentary has focused on this issue. Smilde and May articulate the common concern that such a trend would crowd out legitimate examinations of the negative effects of religion—or, more ominously, that gatekeepers could push negative research aside because it does not fit the interests of a pro-religious faction. We do not know how good or bad religion ultimately is, but if only sub-topics that tend to produce positive depictions of religion are supported, that would no doubt be bad for the sub-field, as it would prejudice our knowledge of religion.
I am sure that there have always been scholars suspected of focusing on issues that tend to produce positive representations of religion, and others thought to consciously or unconsciously neglect negative interpretations of their data. But, for all the talk of either a general increase or an up-then-down trend in “pro-religious” research, I read Smilde and May’s data, shown in Figure 3, as indicating no substantial change over the past thirty years related to the aforementioned concerns about crowding out negative findings. Because there is a concern that there exists a prevalent bias toward positive evaluations of religion (and thus a disincentive to produce negative findings), what matters is the ratio of positive to negative socio-evaluative findings. First, if we look at the gap between positive and negative evaluations over the thirty years surveyed, we see that it started at 5%, peaked at 21%, and then fell back to 13%. If that change is statistically significant, it is not substantively significant. If the argument is that there is a large gap, indicating discrimination against scholars who want to demonstrate the negative effects of religion, 13% is just not that much of a gap. Second, the authors claim the growth of a new category of article, in which both positive and negative aspects are portrayed. In the final five-year period of the survey, these articles were twice as common as during any previous time period. If there was an emerging bias against negative portrayals of religion, it appears to have subsided. In sum, I read this analysis as substantively showing that there has not been a surge in “pro-religious research” of the type that would crowd out negative research, but rather a growth in evaluative research in general, as would be expected with the emergence of the strong paradigm.
A related claim is that an increase in the ratio of positive to negative findings on religion has been driven by funding entities with a pro-religious orientation. While the authors do not give us the percentage of unfunded articles with a positive orientation, by my calculations (based on data derived from other tables) it is 20%. If so, then articles receiving government funding are 21% more likely than articles with no funding to positively evaluate religion. The difference between private funding and no funding is 12%, and funding from religious sources is no more likely to produce positive claims than nonreligious funding. So, religious funders are actually the least pro-religious, in terms of socio-evaluative findings, followed by private foundations, with the government the most pro-religious of all. It is hard to see this as religiously inspired funders supporting only research that will portray religion in a positive light, unless one sees government agencies as wanting to promote religion, and the religious foundations as not knowing their own interests. I have no doubt that some of the private funding sources have pro-religious agendas, but if they can only produce a 12% difference in positive articles, and have not upset the ratio of positive to negative research over time, concerns about them seem overblown. I suspect that many funding entities have nonreligious agendas that nonetheless happen to result in a very slight preference for topics in which religion happens to figure positively (e.g., public health).
Both the increasing religious diversity among sociologists of religion and the diversity of socio-evaluative findings on religion have overall been good for the field. The former, combined with changing notions of civility, may have contributed to a decline in attempts to explain religion itself. This needs to be worked on. However, it does not seem to have led to an orientation in the field whereby religion is only promoted, not analyzed. As with the other findings in the report, the central story here is that there has been no substantive change. On the other hand, we should debate case by case whether we want to prod the field into making some perhaps long overdue changes.
Professor Evans’ contention that not much has changed is overstated. Two things have shifted quite substantially, as Smilde and colleagues show. First, the “positive” effects of religion are much more common in recent publications—and the difference is both statistically and substantively significant, contrary to Professor Evans’ suggestion. Second, there is a shift from viewing religion as a dependent to an independent variable. This second development is not positive for those who want to see a scientific study of religion. We don’t know much about the contours of religious affiliations, identifications, participation, or beliefs. There is relatively little high-quality contemporary research examining the development of religious beliefs, identities, or trajectories of participation. And, these issues are contested by pundits and activist scholars who seek to reinterpret religious alignments from the lens of their distinctive faiths. Indeed, contemporary sociology is filled with myths about the contours of religion, particularly the myth that religious participation is constant (it is declining), the myth that sectarian denominations are growing (in reality, some grow, others shrink and the difference is a wash), and the myth that people choose their commitments voluntarily—when in fact childhood socialization and coercion play a substantial role in determining religious alignments.
What has not changed is that religious activists and foundations have continued to control the discourse in the sociology of religion. The same parochial orientations toward data, theory, and methodology that made the sociology of religion a backwater for most of the 20th century continue to hold sway, despite a surge of more general interest in the last 15 years.
Hopefully, a strong program in the sociology of religion will include a critical examination of the factors influencing religious organizations, individual religiosity, and the dynamics of religion over time and across nations. Ignoring religion as a dependent variable is not a strong move.
We think John Evans comes to our working paper with the right attitude of wondering whether the null hypothesis is actually overcome. That is the starting point that distinguishes social scientific research from other forms of knowledge and is the reason that we ourselves undertook a quantitative content analysis rather than simply writing a review article based on our impressions of the field.
That said, while we agree that the primary story regarding the insertion of the sociology of religion in the larger discipline, and the empirical topics addressed, is primarily the lack of change (and we will address this in a future post), we think there are some clear and important changes with respect to the emergence of a strong program, the increase in socio-evaluative findings, as well as the relationship of these latter with funding.
Evans is right to point out the difference between statistical significance and substantive importance in quantitative research. With a large sample size, a minuscule difference can achieve high levels of statistical significance. However, this is less of a problem in our study because our sample is relatively small (around 600 cases). Furthermore, we use the relatively high threshold of two-tailed tests. This means that when our findings reach statistical significance the reader can be relatively sure that there is something important going on.
But while statistical significance has clear measures, substantive significance has no measure other than whether you think there is something important going on. To get a feel for it, one needs to put differences into everyday narratives. The following hypothetical situations would be consistent with the findings in our working paper.
• If in the late 1970s a graduate student were to examine the causal models used in sociology of religion articles she would find almost 40% having religion as a dependent variable and only 25% having religion as an independent variable. If she were to do the same review thirty years later she would find only 20% of articles having religion as a dependent variable and over half with religion as an independent variable. Put differently, thirty years ago it was approximately fifty percent more common for religion to be explained than explanatory while in the most recent measurement it was more than twice as likely to be explanatory than explained.
• If our graduate student were charged with reading 100 journal articles on religion in 1978, 12 would reflect negatively on religion while 15 would reflect positively. If she did the same thing in 1993, 6 articles would reflect negatively on religion while 28 would be positive. If she carried out the same task in 2003 21 would negative and 34 positive.
We think our hypothetical graduate student would have quite different experiences in each of these cases and these experiences would likely affect her own developing perspective.
Indeed, other than in certain key moments such as social revolutions or natural disasters, social structure tends to work precisely through small inputs iterated over time. Race, gender, class, residential location, pollution, networks, quality of education, public services, and social programs each have very small effects on individuals when they are compared to each other. But together and iterated over time these and other variables combine to create the radical, structured inequality that characterizes our contemporary world.
Consider funding. The data show that both private and public funding are positively and significantly correlated with positive socio-evaluative findings. If articles with funding are between 12% and 20% more likely to report positive socio-evaluative findings than articles without funding, over time this can have a huge impact (as anyone with credit card debt could readily testify). Thinking about it in substantive terms, for every research project that gets funded there are several more that don’t. And these unfunded researchers tend to observe those who did get funding, think about what went wrong with their own proposal and how they can be more successful the next time. When this experience is iterated over and again, the results can be considerable.
Think about it in another way. Imagine that, for better or for worse, many public and private institutions with power and resources want US citizens to maintain a positive image of religion, and they promote this goal not only through funding for scholarship, but through funding for faith-based social programs, tax breaks for religious institutions, or providing political access to religious leaders. Each of these initiatives likely has a small impact; but taken together and over time, they undoubtedly would affect people’s perceptions of religion.