In “Measuring Evangelicalism: Consequences of Different Operationalization Strategies,” which is featured in the latest issue of the Journal for the Scientific Study of Religion, co-authors D. Michael Lindsay and Conrad Hackett report that depending on how one defines an “evangelical”—based on denominational affiliation, self-classification, or specific religious beliefs—this group’s political impact can vary widely. According to a press release announcing the study:
“In this election year, there is much debate over whether Sen. Obama can shave off enough evangelical votes to carry certain swing states, said Rice’s D. Michael Lindsay, one of the researchers. “That depends a great deal on which poll you are looking at and, more importantly, how the survey defines the evangelical population.”
[…]“Evangelicals continue to be the most organized constituency of the Republican Party,” Lindsay said. “However, Democrats have made unprecedented efforts to woo religious voters. They won’t win the votes of all evangelicals, but in a tight election year, they don’t have to — only a few are needed. The perception over whether evangelicals are remaining loyal Republicans or are leaning Democratic depends, in great measure, on which survey is being cited. Evangelicals are the most discussed but least defined population among the American electorate.”
Read the executive summary of the report here.
That last line is an important and informative statement. As a former religion reporter in Dallas and author of “The Fall of the Evangelical Nation,” I’m the only reporter who seems to have followed the numbers instead of merely reporting whatever the polls or the evangelical/political leaders say. If you look at statistics evangelicals themselves put out, you’ll see that their numbers can’t possibly be as large as the media and the polls indicate. They know that. But the public doesn’t.
The problem? All these surveys rely on self identification. And evangelical or Baptist is the default position in the South. Saying you’re a born again or an evangelical or a Baptist is more likely to indicate that you’re a socially conservative Southerner than that you’re among the Religious Right.
Having said that, I’m happy someone is looking more closely at the numbers.
Christine,
Thanks for your comment and interest in this topic. I agree that journalists and leaders of evangelical organizations have incentives to promote generous estimates of the size of the evangelical population. I am not sure that the problem is self identification, whether understood as self-identified evangelical identity or self-disclosed denominational affiliation.
Self-identification methods do not necessarily lead to wildly exaggerated estimates (though see Gallup’s 1998 estimate that 47% of the country is evangelical). Christian Smith’s 1998 book American Evangelicalism: Embattled and Thriving asked churchgoing, self-identified Protestants to choose the most apt label among several different possibilities. This self-identification approach suggested that 7% of the population is evangelical. So the problem isn’t necessarily the approach of asking people about their religious identity but we do have to be very thoughtful about how we ask the questions and how our questions and methodological assumptions influence the answers we receive.
To be clear: I think that self-identified evangelical status can be very informative, especially if the question is asked well (see a recent post on this blog for my concerns about the Gallup/election exit poll question about born again/evangelical identity). It would also be nice to have additional information in surveys about how people are connected to evangelical organizations and whether they consume evangelical media. I think that knowing someone’s denominational affiliation, especially combined with the frequency of their church attendance, is also a useful predictor of various behaviors. Affiliation with a denomination in the evangelical Protestant tradition is associated with different behaviors and levels of religious commitment than affiliation with a denomination in the mainline Protestant tradition.
Since you express concern about official statistics, self-reported evangelical status, and self-reported denominational affiliation, I wonder what method you prefer for making any claims regarding the evangelical population.
Conrad,
Thank you for your comments.
Self identification is a useful tool, but it’s been used as the only measure. The point you make about church attendance is a good one. But as you know, church attendance is also exaggerated because that, too, is by self report.
These polls unlike other many other social surveys have no controls built in so that we can guess about whether people are telling the truth. They are incredibly blunt instruments being used in the public square as though they tell us much more than they do. If they were being used only by religion scholars and not to legitimize a huge political movement, this question wouldn’t be so critical.
When Gallup added some qualifications to their question, their evangelical numbers plummeted. Barna’s stats are often criticized but they’re the most accurate because he understands what’s happening in the evangelical world. As you know, he looks at beliefs. What’s interesting about the beliefs he uses is that even within the evangelical world they are now considered too stringent.
If you come from an evangelical background in the Deep South of the ’60s and ’70s as I do, you know that Barna’s filtering beliefs are basic to what being an evangelical has always meant. That so few evangelicals agree with them is the big story but it’s been pushed aside. (Although when the SBC wanted to challenge Pew’s stats about 50 percent of evangelicals believing there are multiple ways to heaven, they selected only Barna-definition evangelicals for their poll.) We ought to be asking, “Who are these self-identified evangelicals who make up 18 percent of the population and what do they really believe?” The answers are astonishing.
I’d be willing to bet that Christian Smith, who came up with about the same number Barna and I did, put in some filters to weed out people who were calling themselves evangelicals but don’t act as traditional evangelicals or believe as traditional evangelicals (who should now actually be called fundamentalists) do.
I’d like to send you a copy of my book and hear what you think of how I came up with my numbers. I used Barna’s work as one of my measures.
If you use the numbers that the NAE and the SBC put out, you will believe the polls are correct when they say that one out of four Americans is an evangelical. You would think as the press and politicians do that at least 25 percent of white Americans are evangelicals as traditionally defined and as they are characterized in the public square (which is as members of the Relgious Right.)
But I didn’t use the numbers those groups give to the public. Instead, I looked at numbers from evangelicals’ own studies of themselves. These are typically ignored by scholars and the press.
Instead of saying the NAE has 30 million members and the SBC has more than 16 million as they claim, I took the list of churches that are actually in the NAE and added their membership figures together. I looked at SBC statistics and used the method they use in their internal calculations as being most significant — attendance, resident members and Sunday school membership. I added millions more to account for the evangelical nondenominational churches and other evangelical groups.
The numbers were so low that they alarmed me. So I padded them. Only then could I come up with 7 percent. That’s almost certainly an over estimate of true evangelical/fundamentalists who could be counted among Religious Right followers. Or even nominally devoted church members.
Forgive me for going on so long. The picture is much more complicated but I’ll stop now.
Hi Christine,
I think that we both agree on the importance of thinking carefully about how evangelicals are identified. I appreciate your critical perspective on the numbers that get tossed around. Before I say any more, I would like to take your bet about Christian Smith’s study of evangelicals. He does not “weed out people” who call themselves evangelical but have unorthodox beliefs.
My general concern regarding your comments is that I think your approach to evangelicalism tends to sample on the dependent variable. You are interested in what evangelicals believe but your understanding of evangelicalism seems to stem from your “evangelical background in the Deep South of the ‘60s and ‘70s,” which leads you to know that, “Barna’s filtering beliefs are basic to what being an evangelical has always meant.” Of course, if you define evangelicals by certain beliefs, it is not possible to be a genuine evangelical and believe otherwise. Hence, it is problematic that you allude to an evangelical world in which Barna’s belief measures are “considered too stringent.” As your comments suggest, there are people who consider themselves evangelical who do not necessarily share all of your (and Barna’s) assumptions about what it means to be evangelical. Along these lines, you might be interested in Smith’s follow-up book Christian American? What Evangelicals Really Want.
As a social scientist, I assume that people tend to give what they regard as true answers regarding their religion when asked in surveys. They do exaggerate their church attendance but this is not necessarily intentional. While 40 percent of the U.S. population reports attending church in the previous week, studies suggest that the more accurate figure is about 25 percent of the population. But I think that those 40 percent of Americans are people who often go to church, even if they were not technically in the pews within the seven days prior to being surveyed. Likewise, in a well designed study such as Smith’s, those people who self-identify as evangelical seem to genuinely believe they are evangelical, regardless of what Barna or anyone else might say. Incidentally, Michael and I analyzed how various evangelical measures overlap in the article referenced at the top of this post, using a survey that had enough religion measures that we could approximate various measurement strategies. For every 100 respondents in our sample of the general population, 9 were classified as evangelical using the Barna-like belief measures and 5 were classified as evangelical using Smith’s approach. Only 3 out of every 100 respondents were evangelical by both measures (see Venn diagram on page 508 of the article).
It is easy to assume that certain relationships should exist between self-identity and behavior. For example, Barna has very strong convictions about what evangelicals should believe. Theologically, there is no problem with such normative convictions—they are appropriate. Sociologically, however, this is problematic. If we are to describe the world as it is rather than as we think it should be, we have to accept disconnects between religious identity, belief, and behavior. From a theological perspective, it is easy to make the case that a religious adherent should be active in a local congregation. However, many people claim particular religious affiliation but go for years without attending a worship service.
Assumptions we make about definitions matter. All of Smith’s evangelicals are, by his measurement strategy, churchgoing Protestants (or those who say their faith is really important, a loophole for those who may not be able to make it to church due to mobility issues). So, according to Smith’s approach, all evangelicals are Protestants and basically all are active in a local congregation. Neither of these things is necessarily true in either Gallup or Barna’s method of defining evangelicals.
My point here isn’t to defend one approach but to call for further reflection about the tradeoffs inherent in various operationalization strategies. If pressed, I am inclined to use a combination of self-reported attendance and affiliation information in order to understand religious commitment and its consequences. You and many of the leaders in the Southern Baptist Convention are distressed that there is a disconnect between the 16 million on the rolls and the much smaller number in SBC pews each week. However, there are millions of people who think of themselves as Southern Baptist despite being inactive. For many, a Southern Baptist upbringing is consequential later on in life even if one stops going to church. Considering both affiliation and attendance makes it possible to take seriously both religious identity and commitment.
Oh, and I would be happy to read your book—it has been on my radar for some time due to the positive reviews but I haven’t seen a copy yet.
Conrad,
We do agree on a lot. This is a hard population to pin down.
I would never say that the polls are of little value. Certainly over time, they tell us a lot. And just to know that so many people say they are evangelical is interesting. But we have no good definitions for what that actually means, while the public has an increasingly narrow view of what it means.
(I suspect that politicians have merely substituted the evangelical vote for what used to be called the Southern vote. A nice little bit of wordplay.)
Part of my problem is with how the polls are used and how powerful they have become in shaping public discourse – actually in limiting public discourse. Media often fail to mention that these polls are what people say they are or do. When you and other academics quote them, you do so with all sorts of knowledge about what the numbers actually do and don’t do. But that is not passed on to the public.
I’m not saying that you are responsible for the media, of course. But they’ve allowed the Religious Right to claim much more power than they have — and so have helped create that power – using numbers that don’t tell us nearly as much as we think they do.
If the only qualifier Christian Smith used was that “your faith means a lot to you” and got 7 percent, that’s truly amazing and really does speak to how weak the connection is between that 25 percent and the evangelicals we see in the news so often.
That’s a huge story. But as I’ve found out, the mainstream press isn’t interested in it.
I’m going to read both his books. I wish I’d had those numbers for my own book.
Part of my point is that evangelicals are seen as a monolithic group when actually the term has come to mean so many things that it’s almost useless. I wish the polls were more enlightening about what’s going on. Instead they tend to mask the great changes happening among those who call themselves evangelicals. And reinforce stereotypes about American faith.
I’ve started to suspect that the so-called resurgence of evangelical faith rests on much weaker evidence than most of us knew.
It may be that what’s happening with evangelicals is what happened with Catholics. At one time, we had a pretty good idea what a Catholic believed (or thought we did) but now we know that Catholics split off in many directions and that listening only to the bishops won’t give you a good measure of what’s happening in the pews.
Now when we see stats that say so many people say they are Catholic and go to church, we have enough perspective to know that doesn’t mean those numbers represent a huge mass of undifferentiated thinkers bent on taking over the world.
Christine,
Like you, I am a skeptic but you are more skeptical than me, it seems. I am more optimistic about survey research and I am hopeful that evangelicalism can be discussed meaningfully. Corwin Smidt did a nice job of outlining various ways of understanding and measuring evangelicalism in his post on this blog.
Corwin’s post would be a nice primer for journalists and others interested in understanding different approaches to American evangelicalism.
Dr. Conrad and Dr. Lindsay make an excellent contribution with this piece. I especially like their clear distinctions between institutional affiliations, collective identities, and religious beliefs. All three matter for explaining a variety of religious phenomena. What should also have been amplified (though the authors were not allowed much space…), was how identities tend to be contested and transient. We even see this with such seemingly unchangeable social characteristics like race–racial identities change in fairly short expanses of historical time. Josh Gamson and Nancy Whittier have done excellent work showing how contested identities shift across coalition groups and very short generation units. “Born Again” seems now an identity comparable in salience to “Negro” across the respective groups. Identities are more volatile than the groups which produce them, and they are also more chageable than specific beliefs. “Born Again” may be old news, but “Evangelical” is more “in”. With political and cultural tides turning against conservative Christianity, it seems likely that the “Evangelical” identity will also fade in salience among it’s current identifiers. Yet, the proportions adhering to sectarian Christian denominations, or professing beliefs in the inerrancy of Christian scriptures change in predictable ways based on demographics.