A striking feature of the 2012 race for the White House – a contest that pitted the first Mormon nominee from a major party against an incumbent president whose faith had been a source of controversy four years earlier – is how little the subject of religion came up in the media. According to a new study by the Pew Research Center’s Project for Excellence in Journalism and the Pew Forum on Religion & Public Life, just 1% of the campaign coverage by major news outlets (including broadcast and cable television, radio, newspaper front pages and the most popular news websites) focused on the religion of the candidates or the role of religion in the presidential election. Only 6% of the election-related stories in major news outlets contained any reference to religion.

Media attention to religion’s importance in the campaign peaked during the primaries, when several Republican candidates spoke about their Christian beliefs. The prominence of religious rhetoric in speeches by Rep. Michele Bachmann, Texas Gov. Rick Perry, former U.S. Sen. Rick Santorum and others fueled speculation about whether white evangelical Protestants – who made up about one-third of all Republican and Republican-leaning registered voters in 2012 – would withhold support from Mitt Romney because of his Mormon faith. Indeed, the biggest single religion-related campaign story came more than a full year before the election, when a Texas minister publicly called Mormonism a “cult.” That incident, in October 2011, generated fully 5% of all coverage of religion in the presidential campaign.

When Romney captured the GOP nomination and named Rep. Paul Ryan, a Roman Catholic, as his vice presidential running mate in August 2012, they became the first non-Protestant ticket in the Republican Party’s history. But as the primaries gave way to the general election campaign, the subject of religion subsided in the media, in part because neither Romney nor President Barack Obama made much effort to raise it. Fewer than one-in-seven religion-related stories in the campaign (13%) resulted from statements or actions by either candidate.

Rather than focusing on the religious beliefs and practices of the candidates, media coverage of religion during the 2012 campaign frequently centered on the political clout of white evangelicals and their electoral choices – a topic that accounted for 29% of religion-related coverage overall. Talking about evangelicals became a way for the media to address the question of what impact Romney’s Mormon faith could have on the race, confronting religion as a tactical “horse-race” concern.

Romney was the subject of about twice as much religion-related coverage as Obama, and 45% of all religion-related stories in the campaign took the horse-race approach, dealing with how religion might impact the vote. In all, 34% of the religion coverage focused on faith as a character issue, or mentioned it in passing as part of a candidate’s identity. There was far less coverage (16%) of how religion might impact policymaking or governance.

These are among the key findings of the new study conducted by the Project for Excellence in Journalism (PEJ) and the Pew Forum, both of which are part of the Pew Research Center. The study examined nearly 800 religion-related stories from cable television, network broadcast television, radio, newspaper front pages and the most popular news websites in the country between August 2011 and Election Day (Nov. 6, 2012). In addition, the study involved a sample of specialized religious publications and an analysis of hundreds of thousands of messages about the candidates’ faith on Twitter and Facebook; the social media analysis relied on technology developed by Crimson Hexagon. (For more details on how the study was conducted, see the Methodology.)

By the end of the campaign, about two-thirds of U.S. adults (65%) were aware that Romney is a Mormon. But the vast majority of Americans (82%) said they had learned “not very much” or “nothing at all” about the Mormon religion, according to a Pew Research Center poll conducted shortly after the election.

Other findings of the PEJ media analysis include:

• The overall level of religion coverage was about the same as in 2008. Just as in the 2008 presidential campaign, religion accounted for a tiny portion of the total coverage devoted to the 2012 election by major news outlets. Of all the election stories studied, 1% were specifically about religion, the same portion as in 2008. Just 6% referred to religion in any way. That level of attention was fairly even across all platforms studied – with the exception of newspapers, whose front-page stories mentioned religion 16% of the time.

• More religion coverage focused on Romney than on Obama. Romney received twice as much religion coverage as Obama. Thirty-five percent of the religion-related stories focused on Romney, often raising questions about how his faith would be received by voters. Obama’s coverage, at 17%, often focused on incidents in which his Christian faith was challenged, including rumors that he is a Muslim.

• The two nominees rarely sought to bring religion into the campaign narrative. Just 8% of the religion stories during the 2012 race were prompted by statements or actions from the Romney campaign. For the Obama campaign, the comparable figure was even lower (5%). The GOP primary candidates were much more vocal about religion, prompting 22% of religion references in the media during the campaign. Nearly half of all religion stories (47%) resulted from analysis, opinion or enterprise reporting by members of the news media, highlighting the degree to which neither of the candidates wanted to raise religion as an issue.

• Religion coverage was heavily focused on the horse-race angle. Nearly half of all the religion-related stories studied (45%) dealt with how religion might impact the race. Many of these horse-race stories revolved around which candidate was winning among particular religious groups, such as white evangelical Protestants in Iowa. Other horse-race stories examined how the candidates used religion as part of their campaign strategy.

• The second biggest element of religion coverage dealt with the candidates’ beliefs and values. In all, 34% of the religion coverage during the presidential race focused on faith as a character issue or mentioned it in passing as part of a candidate’s biography. There was far less coverage -16% – of how religion might impact policymaking or governance.

• In social media, the tone of conversation about the candidates and religion tended to be negative. For Romney, negative assertions about his faith on Twitter and Facebook outnumbered positive ones by more than 3-to-1. And allegations that Obama is a Muslim appeared twice as often as messages seeking to refute those allegations.

• Within the Mormon community, media became a voice of caution to a group suddenly in the spotlight. An analysis of several publications aimed at members of the Church of Jesus Christ of Latter-day Saints (LDS) finds that during the campaign, LDS media often sought to prepare Mormons for increased scrutiny while maintaining distance from politics. For example, Church News, an official news outlet of the church, issued a statement that acknowledged Romney’s candidacy but firmly reiterated the church’s political neutrality.

In the end, the basic contours of religion in U.S. politics remained unchanged in the 2012 election, according to a Pew Forum analysis of exit poll results. In particular, white evangelical Protestants voted as overwhelmingly for Romney (79%) as they did for Republican candidates John McCain in 2008 (73%) and George W. Bush in 2004 (79%). Indeed, white evangelicals voted as strongly for Romney as Mormons did (78%), according to the Pew Forum analysis of exit poll data.

Four years ago, in the 2008 presidential election, media coverage of religion focused on a pivotal, defining moment for each candidate. In Romney’s case, the coverage peaked in December 2007 when he delivered a major speech about faith in public life in an effort to reduce public unease with Mormonism; many media reports likened Romney’s speech to John F. Kennedy’s famous address about his Catholicism to the Greater Houston Ministerial Association in September 1960.

In Obama’s case, the peak in religion-related coverage came in early 2008 when a video surfaced of his former pastor at Chicago’s Trinity United Church of Christ, the Rev. Jeremiah Wright, making controversial statements about race in America. Obama denounced Wright’s remarks, gave a major speech on the subject of race and eventually resigned from Wright’s congregation. Many media covered the controversy as a story about race or politics rather than, or in addition to, a story about Obama’s religion. In the 2008 primary season, the Wright saga was the No. 1 campaign story overall, accounting for 6% of media coverage of the campaign in the first four months of that year.

In the 2012 presidential campaign, the overall level of religion coverage was about the same as in 2008: Religion was the central topic in about 1% of campaign stories studied. But in 2012 the media covered religion less as a defining issue about the identity of the candidates and more as a tactical matter. The tactical coverage focused on such questions as whether Romney’s Mormon faith would hurt him with the GOP’s conservative Christian base, whether Santorum’s fervent Catholicism would help him win the GOP nomination and whether Obama’s health care legislation would alienate Catholics who felt that the law infringed on their institutions’ religious freedom.

During the 15-month period studied, the volume of religion coverage formed an arc that peaked during the primary season, in the early months of 2012. Nearly half (41%) of all mentions of religion in major media outlets appeared during January and February, around the time of the Iowa caucuses and the early primaries in New Hampshire, South Carolina and elsewhere. This was also the period when Santorum, who was particularly vocal about the role of faith in his political decision-making, briefly surged in the polls.

But as the campaign became a contest between Obama and Romney, the media’s attention to religion’s role in the campaign waned, save for a brief period when Islam became part of the media narrative following an attack on the U.S. diplomatic mission in Benghazi, Libya, that killed the ambassador and three other Americans. Between June and Election Day, religion was mentioned in just 4% of campaign stories – about a third as many as in January and February, during the early primaries (13%).  

In contrast with the 2008 presidential campaign, moments in which either of the candidates was challenged on his religious identity, or forced to defend a controversial religious position, were muted and short-lived. Among the incidents that rose to the surface, at least momentarily, was when a Baptist minister from Texas described Romney’s Mormon faith as a cult. The Rev. Robert Jeffress made the comments at the October 2011 Values Voter Summit in Washington, D.C., as he was introducing Rick Perry. Romney responded only briefly to the comments – “I just don’t believe that that kind of divisiveness based on religion has a place in this country,” he said – and the story faded after about a week, amounting to 5% of all religion coverage during the period studied.

Another modest flurry of religion coverage began in January 2012 when the Obama administration announced that most health insurance plans must cover women’s contraception, provoking criticism from some religious institutions and leaders. These critics included the U.S. Conference of Catholic Bishops, which described the legislation as a threat to religious freedom. The ensuing debate became a campaign issue as GOP candidates, including Romney, challenged Obama – a rare instance in 2012 in which the candidates themselves initiated faith-related news. Yet it accounted for just 4% of all religion-related coverage in the campaign.

Other religion-related events that garnered media coverage in the 2012 campaign included a recycling of the Rev. Wright controversy from four years earlier, a YouTube video about the Islamic Prophet Muhammad that sparked protests in several countries, a gathering in Texas of prominent evangelical leaders to decide which candidate to support, and continuing allegations or rumors that Obama is a Muslim.

While just 1% of all the election coverage studied was squarely about religion, in the 2012 campaign PEJ also analyzed the number of stories that mentioned religion, even in passing. By that measure, 6% of campaign stories in 2012 contained at least some reference to religion. (Because this was a new measure, no comparable figure is available for 2008.)

Overall, most sectors of the mainstream media – broadcast television, cable television, radio and major news websites – all mentioned religion in about the same proportion of 2012 campaign stories (5% to 8%). The one exception was newspaper front pages, where the percentage was at least twice as high: Fully 16% of front-page newspaper stories about the campaign contained at least a passing reference to religion.

Why the disparity between print and other sectors? One possible explanation is that the relative lack of major events or controversies tied to religion may have dampened interest among cable television and talk radio outlets. Web and network television coverage also tended to be somewhat more event-driven. Many of the newspaper stories, by contrast, tended to probe religion even when there was no specific news event triggering the story.

Indeed, more than half (58%) of front-page religion coverage in newspapers was initiated by reporters choosing to raise the issue of religion, rather than reacting to statements by  candidates, comments by religious figures, polls or other outside triggers. That figure is noticeably higher than the share of all religion-related campaign stories in major media as a whole that were self-generated by a reporter, anchor or media commentator (47%).

The candidates themselves were responsible for triggering 35% of religion references in news reports. That figure includes actions and statements by nine presidential candidates in the primaries and general election as well as the two vice presidential candidates. Romney triggered only 8% of religion references, and Obama just 5%. The seven GOP primary candidates who ultimately withdrew from the race and Paul Ryan, Romney’s running mate, triggered 22% of the religion references.

Other references to religion in campaign coverage were triggered by a mix of polls (7%), events (8%) and statements made by religious figures (7%).

An examination of the main themes, or ongoing storylines, in coverage of religion during the presidential race suggests that faith was often viewed through the lens of politics. Throughout the 2012 campaign, major media routinely examined which candidate was winning or losing ground among various religious groups as well as how the candidates and their supporters were using religion as a political tool in the campaign.

A plurality of all stories containing references to religion (45%) focused on what political commentators call “the horse race,” or the tactics of the campaigns and the ups and downs of the candidates in the polls, as opposed to the substance of the candidates’ positions on key issues, their personal histories and their deeper motivations or personalities. The largest portion of horse-race stories (36%) examined how various religious groups, especially evangelicals, would vote. An additional 9% examined how the candidates were using religion as a tactical tool to position themselves or appeal to particular constituencies.

Among the 54% of stories that took a more substantive approach, the largest category (34%) dealt with the candidates’ biographies, including the place of religion in their lives. An additional 16% examined how their faith might impact policy and governance, and 4% explored the religious makeup of the electorate

Another way of looking at the media coverage is to assess which religious faith was featured in each story studied. The data underscore just how prominently evangelicalism featured in the media narrative about American faith and politics. Evangelical Protestants received more coverage in the campaign than did the religion of either candidate – Mormonism and mainline Protestantism (or even Christianity in general since the president has not publicly joined a congregation since leaving Trinity).

Nearly a third of all coverage (31%) focused on evangelicals. Mormonism was a distant second at 18%. Other Christian groups, including Catholics and Black Protestants, accounted for an additional 28%. Judaism (4%) and Islam (6%) were given minor supporting roles.

The heavy scrutiny by the media of white evangelical Protestants – their beliefs, values and voting behavior – can be explained in part because of the size of this group. White evangelicals made up a third (34%) of all Republican and Republican-leaning registered voters, according to Pew Research Center surveys conducted in January-July 2012. And they ended up accounting for 23% of the electorate in 2012 (and also in 2008), according to a Pew Forum analysis of exit poll data. Despite all the speculation about whether evangelicals would stay home on Election Day rather than vote for a Mormon, however, the exit polls indicate that they voted for Romney by an overwhelming margin and with no less enthusiasm than Romney’s other supporters.

Taken together, Romney and Obama were the subject of a slight majority (52%) of religion coverage over the 15-month period studied. The remainder of the coverage was spread among their running mates and the seven Republican primary candidates who ultimately withdrew.

However, Romney was the subject of twice as much religion-related coverage as Obama, with 35% of the media mentions focusing on his Mormon faith. Coverage of Obama’s religion (17% of all religion coverage) more often veered into rumor and innuendo.

Romney

The Romney campaign approached the subject of his religion with caution and, to a considerable extent, so did the mainstream media.

Though he often made generic references to God or faith on the campaign trail, Romney was more reticent about his personal religious beliefs and practices. Just 14% of the religion coverage focusing on Romney was triggered by something that he or his campaign did or said. Reporters, pundits and talk show hosts, whether through investigations or analysis, initiated 48% of religion coverage of Romney. Other candidates – including other GOP contenders in the primaries – largely stayed away from the issue, triggering just 5% of the religion coverage focused on Romney.

There were instances, though, when Romney opened up to the press and the public about his faith. One Sunday in August 2012, Romney’s campaign aides allowed a reporter to attend church services with him near Romney’s vacation home in New Hampshire. During the Republican National Convention later that month, several speakers took the stage to help flesh out Romney’s religious background for viewers, sharing personal stories of Romney’s acts of service in the Mormon community. And Romney himself made reference to his faith tradition, by name, in his nomination acceptance speech.

Another instance of Romney asserting himself in the public debate about religion came after the Obama administration’s Affordable Care Act was introduced. Romney drew coverage for suggesting that Obama was eroding religious liberty by requiring religious institutions to provide coverage for contraception; those remarks accounted for 4% of Romney’s religion-related coverage in the campaign.

The biggest theme, or running storyline, in the Romney-focused religion coverage was his relationship with evangelical Christian voters (33%). While the media were less inclined to question, as they did in 2007, whether the country was ready for a Mormon president, they did report and comment extensively on evangelicals’ discomfort with Romney’s religion.

This line of coverage included a live analysis on Feb. 2, 2012, by Fox News Channel’s Carl Cameron, who told viewers that “it’s worth noting that Mr. Romney’s Mormonism isn’t popular with evangelical Christians in the early voting states.” Similarly, the Rev. Franklin Graham, son of evangelist Billy Graham, told CBS Early Show viewers on Feb. 17, 2012, that “it’s going to be difficult for Romney, as a Mormon, with the evangelical community.”

The issue of whether conservative Christians (also referred to in media accounts by such terms as “values voters,” the “Christian Right” and the GOP’s “Christian base”) as well as other religious groups would support a Mormon candidate accounted for nearly half of Romney’s coverage, at 43%.

Still, fully a third of Romney’s coverage (30%) examined his religious beliefs and background. This coverage ranged from deep profile pieces – such as an October 2011 New York Times exploration of Romney’s time as a Mormon bishop in Belmont, Mass. – to merely parenthetical phrases mentioning that Romney is Mormon. Another 12% of coverage focused on how Romney’s faith could impact how he would govern.

Though no single event dominated Romney’s religion coverage, the comments about Mormonism made by Jeffress stood out, accounting for 10% of all stories focused on Romney’s faith. After Jeffress publicly called Mormonism a cult, cable and broadcast television brought significant exposure to the pastor. During a series of intense questions by CNN host Anderson Cooper, for example, Jeffress insisted that Mormons “do not embrace historical Christianity,” though he also said he thought that Romney would be a better choice than Obama for Christian voters.

Obama

While there are some similarities in how Obama’s faith was covered by the media, there are two ways in which it differed from Romney’s coverage. First, the media focused less on support for Obama among particular religious groups (20% of stories), keying in more on Obama’s religious beliefs (39%) and the intersection of faith and governance (34%). Second, much of Obama’s religion coverage focused on accusations, rumors or misperceptions challenging the truthfulness of his statements about his Christian faith.

The biggest single storyline in Obama’s religion coverage focused on the negative reactions from religious groups, especially Catholic bishops, to aspects of his signature health care law (14%). This provoked claims from some Republican candidates and conservative leaders that Obama was waging a war on religious liberty.

Two instances in which conservatives questioned the sincerity of Obama’s faith made the news as well, accounting for 9% of Obama’s religion coverage. In one instance, the Rev. Franklin Graham said in a Jan. 31, 2012, interview that one would “have to ask President Obama” in response to a question of whether Graham believed the president was a Christian. And Santorum told a group in Ohio on Feb. 18, 2012, that Obama’s agenda was based on “some phony theology.”

An additional 13% of Obama’s religion coverage returned to the Rev. Wright controversy of 2008, as media figures on the right felt compelled to unearth Obama’s past ties to the minister, who espoused a brand of liberation theology that conservatives found unpalatable and unpatriotic.

And another issue that arose in the 2008 campaign, rumors that Obama is a Muslim, also reappeared this time around, accounting for 7% of Obama’s religion coverage. That was in addition to another Islam-related storyline in Obama’s coverage – one that occurred late in the campaign when the attack on the U.S. diplomatic mission in Benghazi prompted Obama to address tensions between Muslims and other faiths. That accounted for 12% of Obama’s religion coverage.

In additional to the main analysis, researchers also examined two alternative streams of information: social media and niche publications directed at the Mormon community. Each revealed a tenor quite different from the mainstream media as well as from each other. An additional analysis of the conversation about religion and the two major candidates between August 2011 and November 2012 reveals just how far social media departed from the more reserved treatment of both Obama and Romney by the legacy media. And an additional analysis of a group of digital niche publications directed at the Mormon community highlights the ambivalence that community felt about being in the spotlight during the campaign.

Social Media Discussion of Romney

In social media, the conversation about religion and the two major candidates was far from the reserved approach of the mainstream media.  On both  Twitter and Facebook, the majority of the discussion about Romney’s Mormonism was highly negative, while suspicions of Obama’s relationship to Islam came through more strongly than statements refuting them .

Twitter discussion about Romney’s Mormon faith during the primaries and general election put the social media platform’s characteristics on full display. The micro-blogging format lends itself to spontaneity and rawness, and as such, the tone of the conversation was often irreverent.

More than half of the assertions about Romney’s faith (57%) were negative in nature, while only 14% were positive.

[i]

Others were more conspiratorial, reflecting their sense that Romney and his faith are secretive: “Wonder if Romney’s taxes highlight his leadership positions in the Mormon church and he doesn’t want people to ask about it,” tweeted TheRitaPhD on Aug. 11, 2012.

A significant portion of the negative conversation on Twitter came in the form of jokes – 11% of the entire conversation and 19% of the negative assertions. The high volume of jokes is a finding that has surfaced as a pattern in other research on social media (see a May 2011 PEJ study about social media reaction to Osama bin Laden’s death and an August 2011 study about the reaction to the death of Steve Jobs).

Even the positive assertions were often a reminder of the hurdle Romney faces with certain voters. “I just don’t understand why people wouldn’t vote for Romney just because he’s Mormon. #ignorant,” tweeted Tbarnes34 on Aug. 30, 2012.

Another 28% of the assertions were neutral, which often came in the form of a user simply mentioning that Romney is affiliated with the Mormon faith.

On Facebook, the tone of conversation about Romney’s Mormon faith was generally similar to what was found on Twitter, though slightly more negative, with 63% of the conversation negative, 22% neutral and 14% positive. (Because Facebook is not nearly as prone to humor as Twitter, at least on this subject, the portion of the conversation devoted to jokes was not measured.)

s on Twitter, the relatively few positive assertions about Romney’s faith on Facebook often were responding to criticisms. One user wrote on Aug. 29, 2012, “I should not dignify this nonsense with a post, but I just can’t let it go. This guy is making a LEAP by calling Romney a racist because he’s Mormon. Huh?”

And the negative discussion was often harsh and suspicious. “Before anyone supports Mormon Mitt Romney for President, they need to know the extremely unique Mormon ‘End of the Age‘ viewpoint and especially the ‘White Horse Prophecy‘! Once properly understood, you will realize that no genuine Christian could ever support Mitt Romney for President,” wrote another Facebook user on June 11, 2012.

Social Media Discussion of Obama

Though Obama identifies as a Christian, the supposition that he is secretly a Muslim has existed since before his election in 2008. The controversy reached a crescendo in 2008 but has never fully disappeared. On social media during the 2012 campaign, this allegation or rumor was espoused about twice as often as it was refuted.

On Twitter, the biggest portion of the conversation about Obama’s relationship to Islam was not contending that he is a practicing Muslim but rather that he has sympathy for the Muslim world (52% of the conversation). Yet 27% of the conversation espoused the belief that he is a Muslim, while 12% refuted that idea. The remaining 9% of the Twitter conversation consisted of general descriptions of the Muslim rumors, neither espousing nor refuting them.

Those who espoused the rumor often used “Muslim” as an epithet: “He just needs to be removed for being a fraud and basicly obama is nothing less then a Muslim Radical Terrorist,” wrote one user on Dec. 18, 2011.

Those who accused Obama of having sympathies with the Muslim world often referred to his administration’s actions in Egypt and Libya: “Obama praises the Muslim Brotherhood takeover of #Egypt. Is anyone surprised? I’m not,” said one Twitter user on June 25, 2012.

As with the social media discussion of Romney’s faith, the Twitter and Facebook conversations about Obama and religion were largely aligned. On Facebook, 53% of the conversation consisted of arguments that the president unduly sympathizes with the Muslim world. Another 26% consisted of rumors that he is Muslim, more than twice the portion (11%) refuting that rumor. And 9% consisted of neutral descriptions of those rumors.

Footnote:

[i]

This report on media coverage of religion in the 2012 presidential campaign uses data derived from two different methodologies. Data regarding the coverage in the mainstream press were derived from the Project for Excellence in Journalism’s in-house coding operation. (Click here for details on how that project, also known as PEJ’s News Coverage Index, is conducted.)

Data regarding the tone of conversation on social media (Twitter and Facebook) were derived from a combination of PEJ’s traditional media research methods, based on long-standing rules regarding content analysis, along with computer coding software developed by Crimson Hexagon. That software is able to analyze the textual content from millions of posts on social media platforms. Crimson Hexagon (CH) classifies online content by identifying statistical patterns in words.

Human Coding of Mainstream Media

Sample Design

The mainstream media content was based on coverage originally captured as part of PEJ’s weekly News Coverage Index (NCI). 

Each week, the NCI examines the coverage from 52 outlets in five media sectors, including newspapers, online news, network TV, cable TV, and radio. Following a system of rotation, between 25 and 28 outlets each weekday are studied as well as 3 newspapers each Sunday.

In total, the 52 media outlets examined for this campaign study were as follows:

Newspapers (Eleven in all)

Coded two out of these four every weekday; one on Sunday 
The New York Times 
Los Angeles Times 
USA Today 
The Wall Street Journal 

Coded two out of these four every weekday; one on Sunday
The Washington Post
The Denver Post
Houston Chronicle
Orlando Sentinel 

Coded one out of these three every weekday and Sunday
Traverse City Record-Eagle (MI)
The Daily Herald (WA)
The Eagle-Tribune (MA)

Web sites (Coded six of twelve each weekday)

Yahoo News
MSNBC.com 
CNN.com
NYTimes.com 
Google News 
FoxNews.com

ABCNews.com 
USAToday.com 
WashingtonPost.com 
LATimes.com
HuffingtonPost.com 
Wall Street Journal Online

Network TV (Seven in all, Monday-Friday)

Morning shows – coded one or two every weekday
ABC – Good Morning America 
CBS – Early Show 
NBC – Today

Evening news – coded two of three every weekday 
ABC – World News Tonight 
CBS – CBS Evening News 
NBC – NBC Nightly News

Coded two consecutive days, then skip one
PBS – NewsHour

Cable TV (Fifteen in all, Monday-Friday)

Daytime (2:00 to 2:30 pm) coded two out of three every weekday
CNN 
Fox News 
MSNBC

Nighttime CNN – coded one or two out of the four every day

Situation Room (5 pm)
Situation Room (6 pm) 
Erin Burnett OutFront
Anderson Cooper 360

Nighttime Fox News – coded two out of the four every day 
Special Report w/ Bret Baier 
Fox Report w/ Shepard Smith 
O’Reilly Factor 
Hannity

Nighttime MSNBC – coded one or two out of the four every day 
PoliticsNation 
Hardball (7 pm) 
The Rachel Maddow Show 
The Ed Show

Radio (Seven in all, Monday-Friday)

NPR – Coded one of the two every weekday

Morning Edition
All Things Considered

Talk Radio
Rotate between:

Rush Limbaugh
Sean Hannity

Coded ever other day
Ed Schultz

Radio News 
ABC Headlines
CBS Headlines

From that sample, the study included all relevant stories:

• On the front page of newspapers
• In the entirety of commercial network evening newscasts and radio headline segments
• In the first 30 minutes of network morning news and all cable programs
• The first 30 minutes of talk radio programs
• A 30-minute segment of NPR’s broadcasts or PBS’ NewsHour (rotated between the first and second half of the programs)
• The top 5 stories on each website at the time of capture

Click here for the full methodology regarding the News Coverage Index and the justification for the choices of outlets studied.

Sample Selection

To arrive at the sample for this particular study of campaign coverage, we first gathered all stories from August 13, 2011-November 6, 2012, that were coded as campaign stories, meaning that 50% or more of the story was devoted to discussion of the ongoing presidential campaign.

This process resulted in a sample of 12,726 stories. That sample was then further narrowed to include only campaign stories that contained at least a reference to religion. This resulted in 793 stories.

Coding of Mainstream Press Religion-Related Campaign Stories

The baseline data in this study derived from PEJ’s regular Index coding was created by a team of seven experienced coders. We have tested all of the variables derived from the regular weekly Index coding and all the variables reached a level of agreement of 80% or higher. For specific information about those tests, see the methodology section for the NCI.

Unit of Analysis

The unit of analysis for this study was the religion reference. Anytime a story contained even a passing reference to religion-a word, a sentence, a paragraph or a section of the story-it was coded using five variables, unique to this study.

An additional set of five variables, focused on religion, were designed as follows: 

Trigger variable

This variable designates the action, event or editorial decision that caused religion to become news in any particular campaign story. Possible triggers include reporters, political candidates, religious figures, and poll releases, among others.

Theme variable

This variable determines the type of broad subject matter addressed by the discussion of religion in a campaign story, such as religious voter support of candidates, candidate beliefs, or impact of faith on policy and governance.

Religion events variable

This variable tracks whether a reference to faith in a campaign story touches on an event or isolated incident, as opposed to religion generically. Examples include the Jeremiah Wright controversy, the gathering of religious leaders in Texas, or Robert Jeffress’ comments about Mormonism being a cult.

Religion newsmaker variable

This variable designates which candidate, surrogate, figure or part of the electorate is the primary focus of the religion reference in a campaign story.

Religious faith focus variable

This variable designates the religious faith tradition that is the focus of the religion reference in a campaign story. If two faiths were discussed in the story, coders selected the one that received more space or time.

Coding Process

Testing of all variables used to determine campaign stories has shown levels of agreement of 80% or higher. For specific information about those tests, see the methodology on intercoder testing.

During coder training for this particular study, intercoder reliability tests were conducted for all the religion-specific variables.

For this study, each of the three coders were trained on the tone coding methodology and then were given a set of 30 stories to code for the five religion-specific variables. The rate of intercoder reliability for each variable was as follows:

• Trigger variable: 83%
• Theme variable: 80%
• Religion events variable: 85%
• Religion newsmaker variable: 83%
• Religious faith focus variable: 80%

Coding of the Conversation in Social Media Using a Computer Algorithm

The section of this report that dealt with the social media discussion of the candidates and religion employed media research methods that combine PEJ’s content analysis rules developed over more than a decade with computer coding software developed by Crimson Hexagon. The analysis was based on examination of nearly 670,000 tweets and 76,000 Facebook posts about Obama and nearly 400,000 tweets and 29,000 Facebook posts about Romney.

Crimson Hexagon is a software platform that identifies statistical patterns in words used in online texts. Researchers enter key terms using Boolean search logic so the software can identify relevant material to analyze. PEJ draws its analysis samples from all public Twitter posts and a random sample of publicly available Facebook posts (and for other PEJ studies, blog posts). Then a researcher trains the software to classify documents using examples from those collected posts. Finally, the software classifies the rest of the online content according to the patterns derived during the training.  

According to Crimson Hexagon: “Our technology analyzes the entire social internet (blog posts, forum messages, Tweets, etc.) by identifying statistical patterns in the words used to express opinions on different topics.”  Information on the tool itself can be found at http://www.crimsonhexagon.com/ and the in-depth methodologies can be found herehttp://www.crimsonhexagon.com/products/whitepapers/.

Crimson Hexagon measures text in the aggregate and the unit of measure is the ‘statement’ or assertion, not the post or Tweet. One post or Tweet can contain more than one statement if multiple ideas are expressed. The results are determined as a percentage of the overall conversation.

Monitor Creation and Training

Each individual study or query related to a set of variables is referred to as a “monitor.”

The process of creating a new monitor consists of four steps. There were four monitors created for this study – two for Obama (Twitter and Facebook) and two for Romney (Twitter and Facebook).

First, PEJ researchers decide what timeframe and universe of content to examine. The timeframe for this study was August 23, 2011-November 6, 2012. PEJ only includes English-language content.

Second, the researchers enter key terms using Boolean search logic so the software can identify the universe of posts to analyze. The following terms were used:

• Obama Facebook monitor: Obama AND (Muslim OR Moslem OR Islam OR Islamic OR Islamist)
• Obama Twitter monitor: Obama AND (Muslim OR Moslem OR Islam OR Islamic OR Islamist)
• Romney Facebook monitor: Romney AND (Mormon OR Mormonism OR LDS OR “Latter-Day Saints”)
• Romney Twitter monitor: Romney AND (Mormon OR Mormonism OR LDS OR “Latter-Day Saints”)

Next, researchers define categories appropriate to the parameters of the study. The categories were as follows:

• Obama Facebook monitor: Obama is a Muslim; General description of ‘Muslim’ rumors; Obama is not a Muslim; Obama has Muslim sympathies
• Obama Twitter monitor: Obama is a Muslim; General description of ‘Muslim’ rumors; Obama is not a Muslim; Obama has Muslim sympathies
• Romney Facebook monitor: Positive; Neutral; Negative
• Romney Twitter monitor: Positive; Neutral; Negative (general); Negative (jokes)

Fourth, researchers “train” the CH platform to analyze content according to specific parameters they want to study. The PEJ researchers in this role have gone through in-depth training at two different levels. They are professional content analysts fully versed in PEJ’s existing content analysis operation and methodology. They then undergo specific training on the CH platform including multiple rounds of reliability testing.

The monitor training itself is done with a random selection of posts collected by the technology. One at a time, the software displays posts and a human coder determines which category each example best fits into. In categorizing the content, PEJ staff follows coding rules created over the many years that PEJ has been content analyzing the news media. If an example does not fit easily into a category, that specific post is skipped. The goal of this training is to feed the software with clear examples for every category.

For each new monitor, human coders categorize at least 250 distinct posts. Typically, each individual category includes 20 or more posts before the training is complete. To validate the training, PEJ has conducted numerous intercoder reliability tests (see below) and the training of every monitor is examined by a second coder in order to discover errors.

The training process consists of researchers showing the algorithm stories in their entirety that are unambiguous in tone. Once the training is complete, the algorithm analyzes content at the assertion level, to ensure that the meaning is similarly unambiguous. This makes it possible to analyze and proportion content that contains assertions of differing tone. This classification is done by applying statistical word patterns derived from posts categorized by human coders during the training process.

The monitors are then reviewed by a second coder to ensure there is agreement. Any questionable posts are removed from the sample.

How the Algorithm Works

To understand how the software recognizes and uses patterns of words to interpret texts, consider a simplified example regarding an examination of the tone of coverage regarding Mitt Romney. As a result of the example stories categorized by a human coder during the training, the CH monitor might recognize that portions of a story with the words “Romney,” “faithful” and “committed” near each other are likely positive for Romney. However, a section that includes the words “Romney,” “secretive” and “corrupt” is likely to be negative for Romney.

Unlike most human coding, CH monitors do not measure each story as a unit, but examine the entire discussion in the aggregate. To do that, the algorithm breaks up all relevant texts into subsections. Rather than dividing each story, paragraph, sentence or word, CH treats the “assertion” as the unit of measurement. Thus, posts are divided up by the computer algorithm. If 40% of a post fits into one category, and 60% fits into another, the software will divide the text accordingly. Consequently, the results are not expressed in percent of newshole or percent of posts. Instead, the results are the percent of assertions out of the entire body of stories identified by the original Boolean search terms. We refer to the entire collection of assertions as the “conversation.”

Testing and Validity

Extensive testing by Crimson Hexagon has demonstrated that the tool is 97% reliable, that is, in 97% of cases analyzed, the technology’s coding has been shown to match human coding. PEJ spent more than 12 months testing CH, and our own tests comparing coding by humans and the software came up with similar results.

In addition to validity tests of the platform itself, PEJ conducted separate examinations of human intercoder reliability to show that the training process for complex concepts is replicable. The first test had five researchers each code the same 30 stories which resulted in an agreement of 85%.

A second test had each of the five researchers build their own separate monitors to see how the results compared. This test involved not only testing coder agreement, but also how the algorithm handles various examinations of the same content when different human trainers are working on the same subject. The five separate monitors came up with results that were within 85% of each other.

Unlike polling data, the results from the CH tool do not have a sampling margin of error since there is no sampling involved. For the algorithmic tool, reliability tested at 97% meets the highest standards of academic rigor.

Please click here for a PDF of topline data tables.