{"id":10382,"date":"2019-12-16T10:00:12","date_gmt":"2019-12-16T15:00:12","guid":{"rendered":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/short-reads\/%year%\/%monthnum%\/%day%\/why-we-studied-american-sermons-and-how-we-did-it\/"},"modified":"2024-04-14T01:18:39","modified_gmt":"2024-04-14T06:18:39","slug":"why-we-studied-american-sermons-and-how-we-did-it","status":"publish","type":"short-read","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/short-reads\/2019\/12\/16\/why-we-studied-american-sermons-and-how-we-did-it\/","title":{"rendered":"Q&A: Why we studied American sermons and how we did it"},"content":{"rendered":"

Every week tens of millions of Americans listen as their religious leaders provide teaching, comfort and guidance from the pulpit. But what are they hearing?<\/p>\n\n

Today, Pew Research Center published \u201cThe Digital Pulpit<\/a>,\u201d its analysis of a broad swath of sermons delivered in U.S. churches during an eight-week period in 2019. Years in the making, the project employs advanced \u2013 and often specially built \u2013 computational tools to identify, transcribe and analyze nearly 50,000 sermons that U.S. churches livestreamed or shared on their websites.<\/p>\n\n

We spoke with Dennis Quinn, the computational social scientist on the Center\u2019s Data Labs team who spearheaded the project, on how it came together and the special challenges that arise when religion meets big data. The interview has been edited and condensed for clarity and concision.<\/p>\n\n

This project has been a long time in the making. How did the idea for it come about?<\/h4>\n\n
\"Republicans<\/a><\/p>\n

\"Dennis
Dennis Quinn, Computational Social Scientist, Pew Research Center<\/figcaption><\/figure><\/figure>\n\n

I was interested in big data when I came to work at the Center\u2019s Religious Restrictions project. So I asked Alan Cooperman, our director of religion research, if he had any ideas that might benefit from a big data approach, and he immediately brought up the idea of analyzing sermons.<\/p>\n\n

The fundamental question was whether this was even feasible. For instance, is there a way to get any comprehensive list of churches with their websites? That led us to Google Maps, which we used to develop a database of churches. Was there a way for computers to identify sermons on churches\u2019 websites? That led us to develop the machine learning technology that we used to identify the pages where congregations share their sermons.\n<\/p>\n\n

Over that almost two and a half years, what were some of the biggest challenges you faced?<\/h4>\n\n

A project like this spans everything from the minutiae of database design and computer code to big issues of policy and direction. I had to really be careful that a technical decision I was asking an engineer to make in August 2017 wouldn\u2019t somehow come around and cause some unintended consequence for others down the line in 2019.<\/p>\n\n

For instance, at the time we searched for congregations on Google Maps we could not choose a single inclusive term for all types of congregations, so we used the term \u201cchurch.\u201d There was an alternative term, \u201cplace of worship,\u201d but it wasn\u2019t supported anymore in the program. That\u2019s an example of a single line of code written in fall of 2017 which had huge implications for how we described our data in fall 2019 when we were writing the report. The \u201cbutterfly effect\u201d of a big data project is staggering \u2013 the small, technical decisions that you make up front have colossal implications for the later direction of the project.<\/p>\n\n

What were some of the privacy concerns that arose in the course of the project, and how did you address them?<\/h4>\n\n

Sermons often include people\u2019s private religious moments, which they experienced in a real and often profound way. The churches did, of course, choose to share them online, so we felt it was appropriate to collect and analyze them, but we had to make sure that we were stewarding those data in a respectful way.<\/p>\n\n

In a more technical sense, there were plenty of times that a website would have a password to get to the sermons, or they would be visible but stored in a way that they were hard to get to, and we decided that we weren\u2019t going to touch those. If there was any effort at all on the part of the congregation to prevent any sort of automated collection, we made no effort to get past that. We also set limits on how fast the scraping program could move between pages, so as to ensure we didn\u2019t overburden the congregational websites. We also decided as an added privacy precaution not to list their name or locations of specific congregations or make any of the sermon texts available.<\/p>\n\n

Some readers might not know how common or uncommon it is for congregations to share their sermons online. Can you talk a bit about this?<\/h4>\n\n

The scraper found sermons on about 6,000 out of the roughly 38,000 congregational websites we examined. Bear in mind that since those congregations have websites on Google Maps, they\u2019re already more online than some. But if you think about it from a pastor\u2019s perspective, a sermon is the fruit of your labor, so it\u2019s not necessarily outlandish that you\u2019d want it to be heard by the broader world \u2013 or, for that matter, available to congregants who can\u2019t make it to church.<\/p>\n\n

Why did you decide to build your own dataset of sermons, rather than using a ready-made database like a sermon aggregator?<\/h4>\n\n

The \u201cbutterfly effect\u201d of a big data project is staggering \u2013 the small, technical decisions that you make up front have colossal implications for the later direction of the project.\nDennis Quinn<\/cite><\/p><\/blockquote>\n\n

When we at the Center are deciding whether to launch a new research project, we ask ourselves if this is something that we can do it in a meaningful and rigorous way that harnesses our technical abilities and resources. Of course, the data we did collect is still not representative of all U.S. sermons \u2013 these are still sermons that congregations with websites chose to share online \u2013 but by collecting them from actual congregations, we at least know that they can tell us about what a real<\/em> swath of real<\/em> churchgoers really<\/em> heard during an eight-week period in 2019. We felt that if we were going to try to build a limited but insightful window into American religious services, we were going to do so in the best way possible. It was a \u201cgo big or go home\u201d kind of moment.<\/p>\n\n

Was there anything about the actual results that you found surprising?<\/h4>\n\n

The thing I was consistently in awe of was the sheer volume of information that we were working with, and not just in megabytes. The median sermon in the dataset is about 5,500 words, which is the length of a good-sized magazine article. I calculated that that\u2019s about 80% longer than Federalist Paper Number 10<\/a>. That\u2019s a lot of information \u2013 we have 50,000 of these \u2013 and there are people out there internalizing this much information about the world around them on a weekly basis. And, in a technical sense, the fact that we were working with the equivalent of 50,000 magazine features was frankly intimidating.<\/p>\n\n

Given the inherent limitations in your approach, what should readers of the report bear in mind as they read it?<\/h4>\n\n

Readers should approach the findings with those limitations in mind. The congregations that shared these sermons are by definition technology-enabled. They\u2019re also larger and more urban \u2013 and of course, these are the sermons they chose to share.<\/p>\n\n

Still, you can see parallels to what we know about American sermons from other sources. For instance, there are plenty of conceptual reasons you might expect discussion of the Old Testament to drop around Easter Sunday. Well, that\u2019s exactly what happened in the data. The National Congregations Study<\/a>, which is a representative survey of U.S. religious congregations, asks each congregation how long their most recent sermon lasted. They find that the median congregation reports 30 minutes. Well, we find that our median sermon runs 37 minutes. Considering that these are two entirely different ways of answering the same question, that\u2019s not that far off.<\/p>\n\n

If you could do a follow-up to this study, how would you build on it?<\/h4>\n\n

It would be great to get a better sense of the real humans on both sides of the altar \u2013 of the pastor and also the congregants. So if we were going to do this bigger and better, it would be a value-add to know more about the opinions of the pastors, the contents of the sermons, and how they affect the opinions of the congregants.<\/p>\n\n

Note: See full report<\/a> and methodology<\/a>.<\/em><\/p>","protected":false},"excerpt":{"rendered":"

Dennis Quinn, computational social scientist, explains how our analysis of sermons came together and the challenges that arise when religion meets big data.<\/p>\n","protected":false},"author":340,"featured_media":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"sub_headline":"","sub_title":"","_prc_public_revisions":[],"_ppp_expiration_hours":0,"_ppp_enabled":false,"ai_generated_summary":"","apple_news_api_created_at":"","apple_news_api_id":"","apple_news_api_modified_at":"","apple_news_api_revision":"","apple_news_api_share_url":"","apple_news_cover_media_provider":"image","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_cover_video_id":0,"apple_news_cover_video_url":"","apple_news_cover_embedwebvideo_url":"","apple_news_is_hidden":false,"apple_news_is_paid":"","apple_news_is_preview":false,"apple_news_is_sponsored":"","apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_suppress_video_url":false,"apple_news_use_image_component":false,"apple_news_api_pending":"1713063541","relatedPosts":[],"datacite_doi":"","datacite_doi_citation":"","_prc_seo_qr_attachment_id":0,"spoken_article_player_enabled":true,"displayBylines":true,"footnotes":"","prc_watchers":[],"_prc_fork_parent":0,"_prc_fork_status":"","_prc_active_fork":0},"categories":[161,179,353,36],"bylines":[842],"collection":[],"datasets":[],"_post_visibility":[],"formats":[467],"_fund_pool":[],"languages":[],"regions-countries":[515],"research-teams":[521,517],"workflow-status":[],"class_list":["post-10382","short-read","type-short-read","status-publish","hentry","category-beliefs-practices","category-christianity","category-data-science","category-methodological-research","bylines-drew-desilver","formats-short-read","regions-countries-united-states","research-teams-data-labs","research-teams-religion"],"label":"Short Read","post_parent":0,"word_count":1330,"canonical_url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/short-reads\/2019\/12\/16\/why-we-studied-american-sermons-and-how-we-did-it\/","art_direction":{"A1":{"id":20396,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/12\/FT_19.12.13_SermonQA_feature.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/12\/FT_19.12.13_SermonQA_feature.png?w=564&h=317&crop=1","width":564,"height":317,"chartArt":false},"A2":{"id":20396,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/12\/FT_19.12.13_SermonQA_feature.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/12\/FT_19.12.13_SermonQA_feature.png?w=268&h=151&crop=1","width":268,"height":151,"chartArt":false},"A3":{"id":20396,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/12\/FT_19.12.13_SermonQA_feature.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/12\/FT_19.12.13_SermonQA_feature.png?w=194&h=110&crop=1","width":194,"height":110,"chartArt":false},"A4":{"id":20396,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/12\/FT_19.12.13_SermonQA_feature.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/12\/FT_19.12.13_SermonQA_feature.png?w=268&h=151&crop=1","width":268,"height":151,"chartArt":false},"XL":{"id":20396,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/12\/FT_19.12.13_SermonQA_feature.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/12\/FT_19.12.13_SermonQA_feature.png?w=720&h=405&crop=1","width":720,"height":405,"chartArt":false},"social":{"id":20396,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/12\/FT_19.12.13_SermonQA_feature.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/12\/FT_19.12.13_SermonQA_feature.png?w=1200&h=628&crop=1","width":1200,"height":628,"chartArt":false}},"_embeds":[],"watchers":[],"table_of_contents":[],"datacite_doi":"","prc_seo_data":{"title":"Why and how we studied sermons of U.S. churches","description":"Dennis Quinn, computational social scientist, explains how our analysis of sermons came together and the challenges that arise when religion meets big data.","og_title":"Q&A: Why we studied American sermons and how we did it","og_description":"Dennis Quinn, computational social scientist, explains how our analysis of sermons came together and the challenges that arise when religion meets big data.","schema_type":"Article","noindex":false,"canonical_url":"","primary_terms":[],"custom_schema":[],"twitter_title":"Q&A: Why we studied American sermons and how we did it","twitter_description":"Dennis Quinn, computational social scientist, explains how our analysis of sermons came together and the challenges that arise when religion meets big data.","og_image":20396,"twitter_image":324872,"indexnow_submitted_at":null,"gsc_index_status":null},"prepublish_checks":{},"apple_news_notices":[],"jetpack_sharing_enabled":true,"relatedPostsOrdered":[],"bylinesOrdered":[{"key":"b4c479ea6e9e1e003b72aecd3177ad30","termId":842}],"acknowledgementsOrdered":[],"_links":{"self":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/short-read\/10382","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/short-read"}],"about":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/types\/short-read"}],"author":[{"embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/users\/340"}],"replies":[{"embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/comments?post=10382"}],"version-history":[{"count":2,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/short-read\/10382\/revisions"}],"predecessor-version":[{"id":101092,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/short-read\/10382\/revisions\/101092"}],"wp:attachment":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/media?parent=10382"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/categories?post=10382"},{"taxonomy":"bylines","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/bylines?post=10382"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/collection?post=10382"},{"taxonomy":"datasets","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/datasets?post=10382"},{"taxonomy":"_post_visibility","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_post_visibility?post=10382"},{"taxonomy":"formats","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/formats?post=10382"},{"taxonomy":"_fund_pool","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_fund_pool?post=10382"},{"taxonomy":"languages","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/languages?post=10382"},{"taxonomy":"regions-countries","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/regions-countries?post=10382"},{"taxonomy":"research-teams","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/research-teams?post=10382"},{"taxonomy":"workflow-status","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/workflow-status?post=10382"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}