Skip to main content

Notebook Religion

Questioning the survey

(Illustration by Krieg Barrie)

Religion

Questioning the survey

Barna Group’s latest research on faith and culture lacks precision

The research organization Barna Group reported in February that two-thirds of American adults who attend church “do so largely because of personal enjoyment.” That claim may not be sensational, but it is misleading.

Barna’s State of the Church 2020 study highlights five trends “essential in understanding the Church’s place in the U.S. today.” It offers little new or noteworthy: Barna concluded that many Christians “church hop,” that church membership is common but declining among younger churchgoers, and that many non-Christians believe church to be irrelevant.

But Barna also reported that 65 percent of American churchgoers attend because they personally enjoy doing so. On the surface, this number seemingly reinforces a common notion that American Christians possess a consumer mentality—that they attend church not to worship the Lord but to satisfy self.

Barna apparently did not intend to draw that conclusion. Digging deeper reveals more about Barna Group than about American Christians.

Barna focused on what worshippers felt they had received from worship, while giving them no opportunity to describe what they had given in worship.

Barna asked survey respondents: “Do you usually attend church because you enjoy doing it, because you feel you have to do it, or you do it out of habit?” The question gave respondents no option to reply, “I attend church to worship God.” The best possible answer respondents could choose (as 65 percent did) paints them in a potentially narcissistic light. A bad question produced a misleading answer: It is inaccurate to conclude, as Barna did, “Those who frequent worship services do so largely because of personal enjoyment.”

The very structure of another survey question reinforced a self-oriented approach to worship. Barna asked: “Thinking about the worship services you attend at your church, how often do you leave the worship services feeling …” Respondents chose from nine possible answers, and while “inspired,” “encouraged,” and “forgiven” scored higher than “disappointed” or “guilty,” Barna focused on what worshippers felt they had received from worship, while giving them no opportunity to describe what they had given in worship. 

Stranger still was Barna Group’s conclusion: “Today’s church leaders are tasked with meeting a diverse set of emotional expectations.” But data describing the emotional experiences of people during worship does not speak to the emotional expectations of those who enter worship. Nor does it imply church leaders must craft worship services primarily to satisfy emotional expectations.

Worship is for the Lord, and maybe American Christians know it. But Barna couldn’t tell you because Barna didn’t ask.

Comments

You must be a WORLD Member and logged in to the website to comment.
  • CR
    Posted: Sun, 03/15/2020 06:46 am

    I believe what you are saying.  While I have no way to check it, your criticism seems creditable.  I remember my college class on how to write a survey question.  It is very easy to write unintentionally biased questions.  I have seen this or intentionally biased questions in Guttmacher institute which claims "The Guttmacher Institute provides reliable, balanced, nonpartisan information on sexual activity" https://www.guttmacher.org/guttmacher-institute-faq.  Yet it a paid political tool of Planned Parenthood.  Read this page and you see it is not a nonpartisan institution,

  •  Georgia Eagle's picture
    Georgia Eagle
    Posted: Sun, 03/15/2020 02:19 pm

    Indeed!  The same dynamic regarding how we interpret surveys also exists in how Christian organizations interpret the Guttmacher Institute's findings.  The Personhood Alliance's "Game-Changer Questions Every Pro-Lifer Should Ask About the Guttmacher Report" could be the starting point of a number of Caleb Team reports.

    From the article:

    Pro-life organizations pointed to abortion restrictions and the public’s increasing rejection of abortion as the key drivers for these declines, while pro-abortion organizations lauded increased use of birth control as a major factor. But do the data back up these claims on either side? Let’s look beyond the headlines and consider the questions every pro-lifer should ask about this report:

    * Are these declines the result of abortion restrictions?
    * Are the declines the result of clinic closures?
    * Is this happening because more women are choosing life?
    * Could birth control really have this much of an impact?
    * Is America becoming more pro-life?
    * What trends are the data showing us, and how are we addressing them?

     

    Personhood Alliance is doing the kind of investigative research that tests both claims and counterclaims from a biblically-objective vantage that World readers would welcome.

  • Steve Shive
    Posted: Sun, 03/15/2020 07:58 am

    I would question the very use of the word "worship" in regards to services offered or for church attendance. I wonder if Barna might have given us some useful and accurate numbers after all.

    The dual misunderstanding that we go to church to worship and that somehow our services are structured for worship is found wanting.

    Is standing and singing a few hymns/choruses worship?

    How did we end up defining that time as worship?

    Do we only worship in some structured environment?

    What is the very nature of worship?

    It seems to me that 65% of people interviewed enjoy going to church would indicate that something good is happening. How many of us can tease out enjoy vs giving to God vs desire vs duty? 

    Does it really matter?

     

  •  Chrisi56's picture
    Chrisi56
    Posted: Sun, 03/29/2020 12:45 pm

    Thank you for questioning the survey!  Churches today are organizing their whole ministry based on surveys and recommendations based on them.  It's a problem of yielding to cultural pressure rather than listening to God. This example proves that they are flawed.  I've taken surveys from Barna and I realize that even my answers often depend on my mood and what happened in the most recent services.  They are not something on which we should rely.