How the Discussion on Twitter Varies from Blogs and News Coverage And Ron Paul’s Twitter Triumph

A detailed examination of more than 20 million Tweets about the race for president finds that the political discussion on Twitter is measurably different than the one found in the blogosphere-more voluminous, more fluid and even less neutral.

But both forms of social media differ markedly from the political narrative that Americans receive from news coverage, according to a new study by the Pew Research Center’s Project for Excellence in Journalism, which examines campaign coverage and the online conversation from May 2-November 27.

One distinguishing factor about the campaign discourse on Twitter is that it is more intensely opinionated, and less neutral, than in both blogs and news. Tweets contain a smaller percentage of statements about candidates that are simply factual in nature without reflecting positively or negatively on a candidate.

In general, that means the discourse on Twitter about the candidates has also been more negative.

The political discussion on Twitter has also fluctuated with events more than it has in the blogosphere, where the authors seem to have made up their minds and where the tone about candidates shifts relatively little. On Twitter, the conversation about a candidate sometimes changed markedly from week to week, shifting from positive to negative and vice versa.

Finally the new study found that the candidate conversation on Twitter is tremendously active-indeed the number of statements about candidates on Twitter vastly outnumber those offered in blogs by a factor of more than 9 to 1.

While it is impossible to know whether the blog sample is capturing the entirety of the political blog conversation, as it is captured on Twitter, it does include all the blogs that enjoy broad traffic and that are tracked by the major blog monitoring services. Data from the Pew Internet & American Life Project shows that a slightly higher percentage of adult internet users say they blog (14%) than use Twitter (13%). And according to the survey, Twitter use is slightly bigger among male internet users than females, most popular among those 18-29 and more widely used by non-white internet users than white ones.

If the difference in volume between Twitter and blogs is indicative of something about the volume of the discourse in those two universes, it suggests that tweeting-with its trim  140-character format that readily invites the  instantaneous observation-is a more frequent activity than blogging.

The Twitter sample for this study is derived from “the Twitter Firehose data feed,” which includes all public tweets from the Twitter system, some 140 million posted every day. From that large universe, the analysis examined those tweets that directly addressed the 2012 presidential campaign. (For blogs, the universe is broad, hundreds of thousands of blogs each day, as tracked by the company Crimson Hexagon.)

And in that campaign discussion on Twitter, one candidate has fared better than anyone else. Congressman Ron Paul has enjoyed the most favorable tone on Twitter of all candidates examined. From May through November, fully 55% of the assertions about the Texas Republican on Twitter have been positive-the highest of any candidate-while 15% have been negative-the lowest percentage of any candidate. That is a differential for Paul of 40 points on the positive side.

Paul is also the most favorably discussed candidate in blogs. While he trails significantly in the polls, and has received less coverage than every Republican candidate except Rick Santorum from news outlets, Paul seems to have struck a chord with some cohort in social media.

This treatment of Paul stands in contrast to that of most of the GOP field, for whom Twitter has been a tough neighborhood. Five of Paul’s seven GOP rivals have had negative opinions on Twitter outstrip positive ones by roughly 2-1 or more.

One exception was businessman Herman Cain-about whom Twitter had been up and then down before he suspended his campaign on December 3 in the wake of allegations of sexual harassment and of a long-term affair. Former Utah Governor Jon Huntsman received more positive than negative assertions though the volume of conversation about him is small.

On the other side of the political spectrum, President Obama’s negative evaluations on Twitter outweighed his positive ones by 3-1. But overall on Twitter, as is true in blogs, other candidates have received rougher treatment than the president. Obama’s most sharply negative assessment has come from the news media, not social media.

These are some of the findings of the study, which also updates the tone and amount of narrative of each candidate in news coverage overall, in a sub-sample of national news outlets, and in the political blogosphere.

The work is part of a new ongoing analysis of the race for president conducted by PEJ that will continue through the election, tracking the amount of attention paid to the candidates in different media platforms and the tone of that attention. The research combines PEJ’s traditional ongoing weekly content analysis conducted by human researchers with computer algorithmic technology developed by the company Crimson Hexagon.

This report, which tracks from May 2 through November 27, captures in close to entirety the first phase of the 2012 campaign, that period before voters are formally involved, the time that might be called The Media Primary. Among its findings is that Twitter and blogs differ enough that the concept of social media as a single form of communication is probably an oversimplification.

The findings also suggest that neither Twitter nor blogs function in general as a form of vox populi that either reflects or anticipates changes in public mood as expressed in representative samples of the population in polling. Sometimes these social media move with polls, but often they do not.

When it comes to news coverage, this research of tone is not a study of media fairness or bias. Rather, it offers a comprehensive, quantitative analysis of whether the messages Americans receive about a candidate in the news media are positive, negative or neutral. The work examines and quantifies all the assertions about a candidate in news stories, editorial and commentaries, whether they come from journalists, supporters, opponents, citizens, newsmakers, pundits, polling data or other sources. The goal is to understand the overall narrative about that candidate. When a candidate is widely criticized by rivals, for instance, Americans are hearing negative statements. When a candidate begins to surge in the polls, and his or her candidacy begins to look more viable, Americans are receiving positive statements about that candidate. In blogs and Tweets, similarly, all assertions are examined.

Among the study findings:

• While they differ, Twitter and blogs both produced harsher narratives overall about the candidates than did news coverage. On Twitter, negative assertions outstripped positive ones for six of the eight GOP candidates, and in five of those cases, by a wide margin. The same was true in blogs. In the news media, only one GOP candidate had a clearly negative narrative, and for four GOP hopefuls, the narrative was more positive than negative.

• Since October, there have been some notable shifts in the tone of the Twitter conversation for several GOP hopefuls. Three of them-Mitt Romney, Herman Cain and Rick Perry-experienced their most negative assessments on Twitter in the past eight weeks. And one candidate, Newt Gingrich, enjoyed his first week of more positive than negative assessments starting October 24.

• In news coverage, the most covered candidate in the last seven weeks was Herman Cain, and that heavy attention was not favorable. Before he suspended his campaign, Cain suffered six straight weeks in which negative assertions in the news outnumbered positive ones-a period of scrutiny that began before allegations about sexual misconduct surfaced and came after nine weeks of positive coverage from August through mid-October. Thus Cain’s candidacy involved three distinct periods in news coverage- limited attention for two and a half months, a period of rising coverage for three months, and six weeks of intensifying and ultimately decisive scrutiny.

• While he has enjoyed the biggest surge in polling numbers in the last month, former House Speaker Newt Gingrich has experienced only a minimal improvement in the tone of his news coverage. It has gone from very heavily negative to only modestly so. For the full seven months studied, Gingrich has had the most unflattering narrative in news coverage of any GOP contender-17% positive, 33% negative and 50% neutral.

• In the blogosphere, since May only one candidate other than Ron Paul-Cain-has received more positive than negative coverage, and that by the razor thin margin (32% positive and 30% negative). The most discussed GOP contender in the blogosphere has been Romney, but the tone has been mixed, with 33% of the conversation positive and 35% negative. Yet that is a much better result than Romney has had in Twitter.

One distinguishing characteristic about the campaign conversation on Twitter is the sheer volume of opinions or assertions carried in the 140-character format.

Consider that there were more than 15 million assessments offered about President Obama between May 2 and November 27, almost 2.2 million about Cain, and almost 1.6 million about Romney. Gingrich was the subject of far fewer, just under 600,000, which puts him ahead of only Huntsman and Rick Santorum.

Another characteristic of the conversation on Twitter is the intensity of the discourse. In most cases, whatever the tone of the conversation about a candidate was in blogs, it was amplified on Twitter-in both the negative and positive directions.

In the blogosphere, for instance, negative opinions about Bachmann outnumbered positive opinions by 26 percentage points. On Twitter, that gap leaped to 51 percentage points. The same pattern was equally clear in the case of Perry. The discourse on blogs was tough enough-negative assertions exceeded positive ones by 25 percentage points. But on Twitter, that margin grew to 40 points. Similarly, in varying degrees, three other candidates whose negatives outweighed their positives on blogs-Romney, Obama and Santorum-also saw that gap grow on Twitter.

For Paul, the converse was true. He enjoyed the most flattering conversation on blogs, where positive assertions outnumbered negative ones by 32 percentage points. But that gap stretched even further-to 40 percentage points-on Twitter.

That intensity gap was connected to another difference about Twitter. The data find that it is less neutral-far less so than news coverage, but even less neutral than blogs. Only two candidates, Romney and Gingrich, had a higher percentage of neutral assessments on Twitter (40% for Romney, 38% for Gingrich) than in blogs (32% for Romney, 37% for Gingrich).

For most candidates, however, the percentage of neutral statements was markedly smaller on Twitter than in blogs and much smaller than in the news coverage, where neutral assessments comprised the largest component of each candidate’s coverage.

There may be several explanations for why Twitter content is more voluminous, more intense and less neutral. One is brevity. Capped at 140 characters, there isn’t much room for qualification and nuance. Another is the ease with which a provocative statement can be passed on. One Tweet can be amplified and multiplied many times over by having Twitter users retweet the original message.

When it came to the volatility of a candidate’s coverage-how much the tone of that coverage varied from week to week-the news media produced the narrative that was most likely to change and bounce between positive and negative.

In blogs the narrative changed relatively little regardless of what was occurring in the campaign.

On Twitter, there was considerably more volatility within candidate narratives than in blogs.

Perry, for example, had negative opinions ranging from 27% one week to 70% in another. In blogs, the biggest weekly differential in his negative attention was 29 percentage points. On Twitter Romney saw his negatives vacillate from as low as 23% one week to as much as 53% in another. On blogs, the biggest weekly differential was nine percentage points.

Even Paul, who enjoyed a consistently positive narrative on Twitter, saw the percentage of positive assertions range from 43% one week to 64% in another-a difference of 21 points. On blogs the biggest spread in the percentage of positive assertions from one week to another was eight points.

Perhaps nobody had a wilder ride on Twitter than Gingrich, whose negative coverage ranged from 19% one week to 70% another-a differential of 51 points. In blogs, his negatives varied much less, 25 points.

Another characteristic of the campaign discourse on Twitter was its sometimes viral nature. Some weeks, traffic about a candidate was driven by pithy (and often pointed) quips-sometimes from celebrities with large followings on Twitter-that would rapidly spread and reach enormous volume. Late night talk host Conan O’Brien generated plenty of attention when he tweeted, in an unflattering reference to Newt Gingrich’s appearance, that the former Speaker “is the #1 candidate in the ‘Could be Related to Bilbo Baggins’ category.” (Baggins was a character in J.R.R. Tolkien’s book, The Hobbit). That week, 59% of the Twitter opinions about Gingrich were negative compared with only 17% positive.

In that same vein, Perry’s reference to Social Security as a Ponzi scheme during a September 7 presidential debate generated a pun-ish response comparing the candidate’s coiffure to that of a 1970’s sitcom character: “Rick Perry’s hair is a Fonzi scheme.” That week, Perry had a tough narrative on Twitter, with negative assertions exceeding positive ones by 44 percentage points.

Pop culture references were popular in the Twitter campaign discourse, as well. After one Obama critic tweeted that he wouldn’t mind if this administration had an “it was all a dream” ending-a reference to the famous final episode of the 1980’s sitcom “Newhart”-it triggered this tweet from conservative commentator Jonah Goldberg:

“I prefer a Scooby Doo ending where Obama’s mask is torn off and he blames those meddling kids.”

One other characteristic of the social media platform is that comments were sometimes very personal and pungent and even profane in nature, using language and leveling allegations that would be off limits  in more traditional news coverage and considerably less likely to show up in the blogosphere.

Mitt Romney

The former Massachusetts governor-who has been at or near the front of the GOP pack for much of the campaign-has generated plenty of attention on Twitter. But the conversation about him there was far harsher than in blogs or news coverage.

From May 2-November 27, Romney was the second-most discussed Republican candidate on Twitter, the subject of almost 1.6 million Twitter assertions. He trailed only Herman Cain. That is largely in keeping with his rank in terms of attention on blogs-where he was the most discussed GOP candidate-and in the news coverage, where he was the No. 3 Republican newsmaker.

Five of the seven busiest weeks for Romney on Twitter occurred in the final weeks examined in this report, October 10-Nov. 27, the same time frame in which the tone of the conversation has turned more negative.

The overall tone for Romney on Twitter was far more negative than positive-19% positive, 40% negative and 41% neutral. That is distinctly different from and less flattering than the mixed assessments he generated both in blogs (33% positive, 35% negative,32% neutral) and in the news media (25% positive, 28% negative, 47% neutral).

In 20 of the 30 weeks included in this study, the negative assertions about Romney on Twitter exceeded positive ones by at least 15 percentage points. And a good deal of the negative assessments about him stemmed from the familiar theme that the former governor is too moderate for the conservative Republican base.

Given its brevity, certain words tend to show up on Twitter often as political shorthand. In the case of Romney, for instance, a certain “R word,” surfaced frequently on Twitter, as in this Tweet declaring simply that “Romney is a RINO.”  RINO is an acronym for “Republican in name only.”

Some of Romney’s better weeks, when the negative conversation exceeded the positive by only single digits, occurred in a three-week period from August 15-September 4 when his negative numbers diminished and the neutral component grew dramatically. But his toughest stretch in the Twitter conversation occurred in the last month, when negative assertions outstripped positive ones by more than 30 percentage points in three of the four weeks.

One of the tweets making the rounds in mid-November was this message from author and humorist Andy Borowitz, alluding to the candidate’s evolving position on some issues: “Romney Admits He is Flip-flopper, Then Denies It.”

Newt Gingrich

One candidate whose Twitter narrative has changed dramatically in recent weeks is Newt Gingrich-as he has moved from negative into positive territory and most recently, into mixed.

Overall, the amount of attention paid to him on Twitter has been relatively modest, but that has also changed recently. Outside of the period surrounding his announcement in May, Gingrich’s three biggest weeks of attention on Twitter occurred from November 7-27-when he surged past Cain and even Romney in some polls.  In November, only Romney and Cain (the most debated candidate of all) were the subjects of more discussion on Twitter.

People are not only talking about Gingrich more in tweets in recent weeks, they are also talking more positively. Indeed, after 25 straight weeks in which the discussion about Gingrich on Twitter was more negative than positive-usually by double-digit differentials sometimes as large as 57 points-the Twitter consensus has shifted with Gingrich’s new poll numbers. He enjoyed two clearly positive weeks in late October, though that eased to a more mixed view in November.

Much of that changed narrative can be attributed to Gingrich’s brighter horserace outlook.  One November 13 tweet, “Gingrich Becomes TEA Party’s Top Choice,” linked to the results of a poll showing Gingrich, at 21%, as the favorite GOP candidate among self-described Tea Party supporters.

For the seven months studied, negative assertions about Gingrich (40%) have outnumbered the positive (21%) by almost 2-1, with another 38% of them neutral.  That is somewhat better than he fared on blogs (12% positive, 51% negative, 37% neutral). It is closer to the tone of his narrative in the news coverage (17% positive, 33% negative, 50% neutral). Yet with his surge, the tone on Twitter is still not markedly positive.

Herman Cain

The Twitter conversation about Herman Cain reflected the frantic trajectory of his campaign. Attention to him spiked dramatically in the fall after he emerged as a surprise frontrunner. But the tone of that narrative took a significantly negative turn in the last month of his candidacy when sexual harassment allegations, followed by another woman’s assertion that she had a long term affair with Cain, finally culminated in his December 3 announcement.  (In news coverage, the tougher scrutiny began two weeks earlier and initially involved his policy proposals and professional record.)

Driven by an explosion of interest in October and November, Cain also generated more attention in the seven-month period than any other GOP contender on Twitter, with more than two million opinions offered about him. That is considerably more attention than he received in blogs, where he was the fourth-most discussed GOP candidate. The huge increase in recent attention to Cain also made him the No. 1 newsmaker in the news coverage from May 2-November 27.

And nearly three quarters of those Twitter opinions occurred in the period-since October 3-that included his dramatic jump to the top of the presidential polls and the allegations of sexual harassment that broke on October 30 and presaged a subsequent and significant drop in the those same polls.

By the numbers over seven months, Cain fared reasonably well in the Twitter narrative, trailing only Paul and Huntsman in overall tone. On Twitter, 34% of the assertions about Cain were positive, 35% were negative and 31% were neutral. That is generally in sync with the mixed tone of his news coverage (29% positive, 34% negative, 37% neutral) and on blogs (32% positive, 30% negative, 37% neutral).  But that only tells part of the story.

Until recently, the Twitter conversation about Cain had been relatively stable from week to week, with his positive assertions outstripping negative ones by anywhere between 10 and 16 percentage points in 22 of the 26 weeks. However, there was a notable turn toward the negative starting in mid-October and he had his four worst weeks from October 31-November 27, with negative opinions outstripping positive ones by at least 14 percentage points in each week.

A number of problems for the Cain candidacy helped drive that negative narrative on Twitter, including the sexual harassment allegations, a viral video of Cain stumbling over a question about Libya and his drop in the polls, where he had been a frontrunner.

Some of the conversation focused on polls. One of the items making the rounds on Twitter was a link to a November 11 Politico story reporting on a McClatchy-Marist poll that had Newt Gingrich vaulting into second place (19%) and Cain dropping to third (17%).

And some tracked the appearance of different women making new allegations. A week later, one of the tweets circulating about Cain said simply, “Woman accuses Cain of bold sexual advance.”

Ron Paul

The Texas congressman is a social media phenomenon.

In many respects, the conversations about Paul on Twitter and blogs were similar-and both were different from his news coverage.  On Twitter, there were more than 1.1 million assertions about Paul, ranking him fifth among the GOP candidates. He was also the No. 5 newsmaker on blogs. But in the news media, Paul finished next to last in the Republican field in terms of quantity of coverage-ahead of only Santorum.

Paul had several major spikes in attention in the past seven months, including the two weeks in August surrounding his second-place finish in the Iowa straw poll, the week in mid-September when the CNN debate in Florida occurred and the week of October 17-23, when another CNN debate took place. But he had his busiest week on Twitter from November 21-27, which included speculation that Paul would have a strong showing in the January 3 Iowa caucus.

And as was the case in blogs, Paul is the clear winner of the Twitter media primary when it comes to tone. A whopping 55% of the assertions about him were positive, only 15% were negative and 30% were neutral.  His closest competitor in terms of positive attention was Cain, well back at 34%. And the candidate with the next lowest percentage of negative assertions was Huntsman, also well off the pace at 24%.

Indeed, Paul is the only candidate who generated more positive than negative assertions on Twitter, blogs and in the news coverage.

On blogs, the assertions about Paul were overwhelmingly positive as well (47% positive, 15% negative and 38% neutral).  In news coverage, however, they were more mixed (23% positive, 16% negative and 61% neutral) and the attention to him there was scant.

In none of the 30 weeks included in this report did the margin between positive and negative assertions about Paul on Twitter fall below 29 percentage points. And he enjoyed six weeks where the differential was at least 50 percentage points.

While he has never shown up among the frontrunners in national GOP polls-and most news coverage discounts his chances of winning the nomination-Paul commands an energized online following, as the tone of his narrative on both Twitter and blogs indicates. Indeed, a number of the tweets examined by PEJ offered the simple message: “Vote for Ron Paul.”

Rick Perry

Much like Paul, Perry is a good illustration of some of the basic similarities between Twitter and blogs. Both in terms of amount of coverage and tone, the assessment of the Texas Governor in social media is far different from that in news coverage.

The Twitter conversation reinforces the idea that Perry was less of a hot topic in social media than in more traditional news coverage. He generated over 1.25 million assertions on Twitter from May 2-November 27, ranking him fourth among Republicans-behind Bachmann, Romney and Cain. That was slightly below his rank in the blogosphere, where he was the third most discussed Republican. In the competition for news coverage, Perry trailed only Herman Cain.

Two of the weeks when Perry received his most attention on Twitter-the period from August 8-21-coincided with his August 13 entry into the race and his instant rise, at that point, to the top of the GOP polls.

The overall tone of the Twitter assessments of Perry was overwhelmingly negative-only 15% positive compared with 55% negative and 29% neutral. (Only Bachmann had a higher percentage of negative opinions offered about her on Twitter during the 30 weeks studied.)

The Perry assessment on Twitter is fairly similar to the one for him in blogs (19% positive, 44% negative, 37% neutral), but very different from the news coverage, where he received more positive coverage (29%) than negative (25%), with 46% neutral.

For the most part, Perry’s narrative on Twitter was relentlessly negative. In 25 of the 30 weeks, the negative conversation exceeded the positive by at least 25 percentage points. In only one week, May 9-15, three months before he announced his candidacy, did that gap narrow to less than 10 points.

The Twitter conversation about Perry has also grown more critical over time as he has slipped from the ranks of the Republican frontrunners. In each of the eight weeks between October 3 and November 27, negative assertions overwhelmed positive ones by at least 39 percentage points-a punishing narrative that happened just once in the previous 22 weeks.

A number of these unflattering tweets took aim at Perry’s difficulties during debates, such as this example-“Rick Perry: Always one question behind.”-that was tweeted during the November debate in South Carolina.

Michele Bachmann

Michele Bachmann has generated plenty of attention on Twitter, but she also experienced the harshest assessment of anyone running, including Obama. 

The Minnesota congresswoman was the subject of nearly a million and a half opinions on Twitter, ranking her No. 3 behind Romney and Cain. She was the second-biggest newsmaker among Republicans in blogs and No. 5 in news coverage.

Bachmann’s busiest weeks on Twitter corresponded to the major events in her campaign-the announcement of her candidacy on June 27 and the two weeks in August surrounding her first-place finish in the August 13 Ames Iowa straw poll. Indeed, almost one-third of all the assertions about her on Twitter occurred in those three weeks.

But the Twitter discourse about Bachmann so far has been overwhelmingly negative. Only 12% of the assertions about her on Twitter were positive compared with 63% negative and 25% neutral. That is a differential of 51 points in the negative direction and represents the second-lowest number of positive assertions and the highest level of negative assertions of any candidate studied.

Those numbers also stand in stark contrast to the tone about Bachmann seen in news coverage, which was more positive overall than negative (29% positive, 24% negative and 47% neutral).

The Twitter discourse about Bachmann was also much harsher than in blogs, where the discussion about her was 18% positive, 44% negative and 38% neutral-a 26 point differential toward in the negative, roughly half what it was in Twitter.

It is difficult to find a bright period in the Bachmann Twitter narrative. In 27 of the 30 weeks from May 2-November 27, negative assertions about her campaign exceeded positive assertions by 40 or more percentage points. While some candidates saw volatility in the tone of the conversation about them, it is clear that Bachmann was unable to surmount the doubts of her critics on Twitter.

Jon Huntsman

On Twitter, Jon Huntsman was pretty well liked, but not often discussed. Over the seven months since May 2, there have been about 350,000 assertions offered about him. That puts Huntsman ahead of only Rick Santorum of GOP candidates studied on Twitter. (In blogs, he was the least discussed candidate of all and in news coverage; only Santorum and Paul got less attention.)  

Huntsman’s biggest spike in coverage on Twitter surrounded his June 21 announcement. And the lack of attention in all three media platforms seems to reflect his standing in the polls, where he has never moved above low single digits.

Huntsman is unusual among Republican hopefuls in that the tone of the conversation about him on Twitter more closely resembled the narrative of his news coverage than in the blogosphere.

He was one of only two candidates to generate more positive attention on Twitter (30%) than negative (24%) with the plurality (46%) being neutral. That resembles the overall tone of his news coverage (21% positive, 17% and 62% neutral), but is far more favorable than the reception he got on blogs-16% positive, 29% negative and 55% neutral.

Still, given Huntsman’s status as a lagging newsmaker in all three platforms, it is questionable whether the positive assessments of him on Twitter and in the news coverage have much of an impact on public opinion.

And much of the positive conversation about Huntsman came early. From May through early July, positive assertions exceeded negative ones by at least 10 percentage points in eight straight weeks. In the remaining four months, he had only three weeks when his coverage was that positive.

Two of those weeks, however, were from November 14-27, when there was a moderate surge of positive opinions about him.

Rick Santorum

There was little enthusiasm on Twitter for the former Pennsylvania senator, either in volume or tone.

Santorum received less attention on Twitter than any other Republican (325,000 assertions in seven months). He received the most attention (about 31,000 opinions) from October 17-23, the week of the CNN debate in Las Vegas.

The attention he generated on Twitter also wasn’t generally flattering. He had the smallest percentage of positive assertions (11%) of any GOP candidate studied. And only Bachmann and Perry had a higher percentage of negative assertions than the 52% that Santorum achieved. (Another 37% of the assertions on Twitter were neutral.)

The Twitter narrative for Santorum was very similar to that on blogs-10% positive, 48% negative and 42% neutral. But the conversation on Twitter and in blogs stand in stark contrast to Santorum’s generally mixed narrative in the news coverage of 23% positive, 25% negative and 52% neutral.

And the discussion about Santorum on Twitter was relatively unchanging. In only one of the 30 weeks studied did his negative coverage exceed positive by less than 24 percentage points.

Barack Obama

If presidential elections are referendums on the incumbent, a look at Twitter suggests that is clearly the case with President Obama. There were more assertions on Twitter about the sitting president than all the Republican contenders for his office combined. During the seven months examined, the study identified and analyzed more than 15 million such assertions.

That volume also remained relatively steady most weeks. The President had his busiest week by far (almost 1.9 million opinions) from May 2-8, in the immediate aftermath of the raid that killed Osama bin Laden.

The other distinction about the Twitter discussion of the president is that it was not more strikingly negative than found in news coverage. Indeed, the discourse in blogs, news and tweets about the president bear some similarities.

The narrative is negative. Overall, 17% of the assertions about him have been positive, 51% negative and 33% neutral. In blogs, it was somewhat less critical, 14% positive, 38% negative and 48% neutral. But the ratio of negative to positive assertions was higher, almost 4-to-1 in news coverage (at 9% positive, 35% negative and 56% neutral). It is in news, indeed, that the President received the lowest percentage of positive statements.

There are some structural reasons that help explain the negative attention to the president in all three platforms, including his association with a steady barrage of unsettling news about the U.S. economy and the fact that he and his policies are regularly targeted by his potential presidential rivals, opponents on Capitol Hill and some disgruntled Democrats.

On Twitter, the president never came close to having a week in which his positive assertions matched his negative ones. But in one sign of Twitter’s tendency to react to events, he had his best week, relatively speaking, from May 2-8, when he hit his high water mark in positive assertions (28%) and tied his low water mark in negative assertions (47%).

That was the week after the death of bin Laden when some tweeters were sending around the congratulatory message:  “Obama got Osama.”

On October 17, PEJ issued a detailed report assessing the tone and level of coverage of the presidential candidates from a wide swath of news outlets and blogs from May 2 through October 9.

This report updates that data through November 27.

In the last seven weeks, two former frontrunners (Cain and Perry) saw the tone of their news coverage become sharply negative. The coverage of the ever present second candidate, Romney, also became more critical. And the new frontrunner, Gingrich, has enjoyed only a modest improvement in tone.

Indeed, as he has surged in public opinion polls, the narrative about Gingrich in the news has changed in only the following way: it has moved from a double digit negative differential to something in single digits.

Overall, in the 30 weeks studied since May, Gingrich has enjoyed only one week in which coverage was clearly more positive than negative, September 19-25, and then just modestly, with positive assertions outnumbering negative by four points.

Since then, his coverage has either been mixed (within three points on either side) or negative. In November, as his poll numbers rose, Gingrich has seen three clearly negative weeks of coverage and one that was mixed. One other factor is that in recent weeks, the amount of coverage of Gingrich has grown dramatically.

As Gingrich has risen in the polls, two others saw their narratives in the news become negative. One of those candidates, Cain, was the subject of intense coverage. The other, Perry, saw attention wane.

In the run-up to his decision to suspend his campaign on December 3 Cain experienced six consecutive weeks in which his negative coverage outstripped positive. That downturn in the tone of Cain’s coverage coincided to a large degree with a huge spike in attention to him in the news media. In October and November as Cain emerged as a surprise frontrunner and then was faced with multiple allegations of sexual harassment, he registered as a primary newsmaker in 35% of all the presidential campaign stories studied and as a significant newsmaker in 45%. His closest competitor for coverage in that two-month period, Mitt Romney, was a dominant newsmaker in 17% of the stories and a significant one in 28%.

Thus Cain’s departure from the race not only coincided with a sustained period of negative attention. That attention (in social media as well as news media) was also intense in volume.

(To register as a primary newsmaker, someone must be featured in at least 50% of a story. To be a significant newsmaker, someone must be included in at least 25%.)

Perry has seen an even more negative portrayal in the last two months, but that has coincided with the news media paying him less attention. He has endured eight straight weeks of more negative than positive coverage as he stumbled on the campaign trail and slumped in the polls.

The candidate  who had been the most positively portrayed of any presidential hopeful between May 2 and October 9 has for the last four weeks been the most negatively portrayed except for President Obama. Perry’s coverage during November has been more negative than Cain’s.

In that month, when the narrative turned more negative, media attention to Perry also diminished dramatically as he generated only about one-third as much news coverage as he had in October.

The tone of President Obama’s coverage, which has run about 4-1 negative over positive, has remained virtually unchanged over the past seven weeks and stands at 9% positive, 35% negative and 56% neutral.

Overall, four candidates have enjoyed coverage that has been more positive than negative in the seven months from May 2-November 27. They are Bachmann (29% positive, 24% negative, 47% neutral), Huntsman (21% positive, 17% negative, 62% neutral), Paul (23% positive, 16% negative, 61% neutral) and Perry (29% positive, 25% negative, 46% neutral.)

On November 9, PEJ released a report that looked at a sub-sample of news outlets that mirrored the roughly four dozen news organizations we track in our weekly News Coverage Index coding measuring the topics in the news. This sub-sample represents the major news outlets from print, online, radio, and network broadcast and cable TV news and might be described as an “elite” sample based on audience size.

That Nov. 9 report, which covered May 2 to Nov. 6, found little variation in the overall tone of candidate news coverage between the broader news sample of more than 11,500 outlets and the subset of 47 elite media outlets. There were instances over those six months in which elite media tended to move faster when the narrative about a candidate was changing, such as when the tone of elite media coverage of Rick Perry shifted to more negative than positive in mid-September after some controversial debate performances. It would be three weeks later, following further Perry stumbles, before the tone turned in the broader news media sample.

Coverage of the president hardly varied between the two media samples.

In this update, looking back at the last several weeks of coverage, we again found only modest, but interesting distinctions. There was little difference in the coverage of the two candidates now most prominent in the polls, Romney and Gingrich.

In the case of Gingrich, in both media samples coverage in November has been somewhat more negative than positive. And in both, the amount of both positive and negative assertions about Gingrich has been rising while the number of strictly neutral or factual ones has been falling.

In the case of Romney, the coverage in both samples has also been more negative than positive in the last month, in contrast to many weeks prior of a fairly mixed narrative.

Where we find differences between the so-called elite media outlets and the broader swath of news media studied is in the coverage of the two candidates whose campaigns appeared to be in the most trouble, Cain and Perry.

For Perry, the elite media in the last month has tended to be less positive and to some lesser degree more negative.

In the case of Cain, the narrative was more sharply drawn in the elite media. In the four weeks since allegations first surfaced in Politico about his sexual conduct, the elite media had a smaller percentage of positive assertions about Cain and a higher percentage of negative ones than has the broad media sample. As Cain responded to the allegations from several women, generating criticism for his handling of this crisis, the elite media portrayal of the state and viability of Cain’s candidacy had grown more skeptical than that seen elsewhere. By early December, Cain’s candidacy was over.

If the last seven weeks have been a period in which news coverage was generally more negative for most candidates, the conversation on blogs shows the same basic trends. There, Perry and Cain have been the subject of increasingly negative attention, and Gingrich has seen some modest gains in the number of positive statements about him.

Gingrich is the candidate who has received the highest percentage of negative opinions in the blogosphere so far in this campaign, and those negative statements about him have outnumbered positive ones by more than for any other candidate. That situation has not materially changed in the last six weeks since he began to rise in the polls.

If anything, Gingrich has had a better stretch in news coverage during this period than in blogs. The best Gingrich can say is that in each of the last four weeks, negative statements by bloggers about him outnumbered positive ones by less than 30 points, something that happened only once in the previous 26 weeks.

If Gingrich is going to win the nomination, he will either do it despite the overall narrative about him in blogs, or he will have to change that narrative.

By contrast, Perry experienced his most difficult stretch in the blogosphere in the last seven weeks. Each week negative statements about him exceeded positives by at least 31 points, a witheringly clear cut narrative.

For Cain, who had enjoyed 26 straight weeks in which bloggers were either more positive or mixed about him, the last month has been a different and difficult time. Negative statements among bloggers about the businessman politician have outnumbered positive every week by at least eight points. That trend started October 31, the day after Politico broke the first story containing allegations of sexual harassment.

While the old and new media behave differently, in other words, there are clear instances in which they interrelate and social media amplify the narrative of the news.

The blogosphere does not always follow news coverage. Ron Paul’s social media narrative stands apart from the one about him in the news.

For Cain, as with Perry, however, the news and the blogosphere were connected. Tough coverage in the news translated into a more difficult conversation about them in social media, both in blogs and in Twitter.

One significant difference among social media platforms is how blogs and Twitter have treated one other candidate, however-Romney. While the narrative on Twitter has been more negative than positive by a 2-1 margin, in blogs it is mixed (split among 33% positive, 35% negative, 32% neutral).

In the case of Romney, blogs and news coverage resemble each other, while Twitter is a different conversation.

The study, “Twitter and the Campaign” uses content analysis data from two sources.

Data regarding the quantity of coverage in the traditional press is derived from the Project for Excellence in Journalism’s in-house coding operation. (Click here for details on how that project, also known as PEJ’s News Coverage Index, is conducted.)

To arrive at the results regarding the quantity of coverage on blogs and Twitter, and the tone of coverage on all outlets, PEJ employed a combination of traditional media research methods, based on our long-standing rules regarding content analysis, along with computer coding software developed by Crimson Hexagon. That software is able to analyze the textual content from billions of messages on blogs, Twitter and web-based articles from news sites. Crimson Hexagon (CH) classifies online content by identifying statistical patterns in words.

Use of Crimson Hexagon’s Technology

The technology is rooted in an algorithm created by Gary King, a professor at Harvard University’s Institute for Quantitative Social Science. (Click here to view the study explaining the algorithm.)

The purpose of computer coding in general, and Crimson Hexagon specifically, is to “take as data a potentially large set of text documents, of which a small subset is hand coded into an investigator-chosen set of mutually exclusive and exhaustive categories. As output, the methods give approximately unbiased and statistically consistent estimates of the proportion of all documents in each category.”

Universe

Crimson Hexagon software examines online content provided by RSS feeds of millions of sites from the U.S. and around the world. This provides researchers with analysis of a much wider pool of content than conventional human coding can provide. CH maintains a database of all texts available so content can be investigated retroactively.

Twitter

Crimson Hexagon draws its universe of tweets from something called the “Twitter Firehose data feed.” That is a feed of all the tweets on the twitter platform that are public. According to the Twitter’s own blog, there are about 140 million tweets posted each day that are public on the Firehose feed. (The Firehose does not include private tweets. However, since private tweets are sent to individuals, much like emails, they are not part of public conversations.)

The volume of conversation in Twitter is referred to as “assertions.” The number of assertions refers to the quantity of statements or opinions focused on each person.

Because CH examines text in the aggregate, it is not enough to simply count the number of tweets where a person’s name shows up to gauge how often a candidate is being discussed. Some tweets may include multiple statements or opinions, while others may use the same word as a candidate’s name without referring to the candidate. For example, a tweet that refers to the Huntsman Center at the University of Utah is likely unrelated to the presidential campaign of Jon Huntsman, and is therefore discarded from the sample studied in this report.

Therefore, the number of assertions is a more accurate measure because it includes the relevant statements about the subjects in the race without extraneous information.

Blogs

Crimson Hexagon’s sample includes hundreds of thousands of blogs from the U.S. and around the world. While the blogosphere is growing every day, it is impossible to create a sample that includes all possible blogs. However, CH’s universe is one of the largest blog samples available and includes all of the most popular English-language political and news blogs.

As with Twitter, the volume of conversation is referred to as “assertions.” The number of assertions refers to the quantity of statements or opinions focused on each person. Since blog posts can range in size, many posts include multiple assertions. At the same time, other long blog posts focused on a non-campaign subject may mention a candidate in a single sentence, and the software learns to include only that relevant statement while exempting the rest from the sample.

Traditional Press

The monitors focused on the traditional press are based on more than 11,500 news websites.

While the software collects and analyzes online content, the database includes many news sites produced by television and radio outlets. Most stations do not offer exact transcripts of their broadcast content on their sites and RSS feeds, however, those sites often include text stories that are very similar to reports that were aired. For example, even though the television programs from Fox News are not in the sample directly, content from Fox News is present through the stories published on FoxNews.com.

The universe includes content from the websites of all the major television networks, such as ABC, NBC, MSNBC and CNN, along with thousands of local television and radio stations. Two notable television sources, CBS and PBS’ NewsHour, do offer transcripts of their television news programs, and those texts are included in the sample.

Elite vs. Broad Sample in the Traditional Press

For this report, PEJ examined two different samples using Crimson Hexagon’s database of news outlets. Most of the discussion is focused on the universe described as the “broad” sample which includes all of the more than 11,500 news sites available. Not all of these outlets contain campaign stories on a regular basis, but any time they do, those stories are included in the sample. For instance, local television newscasts may not offer much coverage of the presidential campaign. However, the sample will include any relevant reports that do appear.

The universe entitled the “elite” sample is made up of a smaller collection of news sites that provide a focused cross section of national media based in large part on audience numbers. This elite sample is based on the 52 different outlets included in PEJ’s weekly News Coverage Index (NCI) which includes print, cable, radio, online and broadcast.

Of the 52 outlets found in the NCI, 47 are included in the elite sample. For technical reasons, five sources cannot be represented. Three radio talk shows (Rush Limbaugh, Sean Hannity and Ed Schultz) do not have accompanying RSS feeds. The Wall Street Journal is not included in the algorithmic tone coding due to its paywall. Google News, which does not produce original material but rather pulls stories from other sources, is not included because the same material is coded in the other outlets where it appears.

The 47 outlets’ content is distributed through 21 unique URLs. This number is smaller because television websites often serve as umbrellas for web feeds from multiple programs. For example, the URL abcnews.go.com includes feeds from both Good Morning America and ABC’s World News with Diane Sawyer. Foxnews.com provides material from Fox News programs including Special Report with Bret Baier, Fox Report with Shepard Smith, the O’Reilly Factor and Hannity.

The list of URLs that are included in the elite sample are as follows:

• 1. http://www.msnbc.msn.com/
• 2. http://www.today.msnbc.msn.com/
• 3. http://www.ed.msnbc.msn.com/
• 4. http://www.cnn.com/
• 5. http://www.foxnews.com/
• 6. http://www.abcnews.go.com/
• 7. http://www.cbsnews.com/
• 8. http://www.npr.org/
• 9. www.pbs.org/newshour
• 10. http://www.nytimes.com/
• 11. http://www.washingtonpost.com/
• 12. http://www.usatoday.com/
• 13. http://www.ajc.com/
• 14. http://www.latimes.com/
• 15. http://www.toledoblade.com/
• 16. http://www.azcentral.com/
• 17. http://www.thehour.com/
• 18. http://www.spokesman.com/
• 19. http://www.joplinglobe.com/
• 20. http://www.news.yahoo.com/
• 21. http://www.huffingtonpost.com/

Monitor Creation and Training

Each individual study or query related to a set of variables is referred to as a “monitor.”

The process of creating a new monitor consists of four steps. (See below for an example of these steps in action.)

First, PEJ researchers decide what timeframe and universe of content to examine-general news stories, blogs, messages on the major social media sites Twitter and Facebook or some combination. For this study, the focus was solely on English-language tweets, blogs and news outlets.

Second, the researchers enter key terms using Boolean search logic so the software can identify the universe of posts to analyze.

Next, researchers define categories appropriate to the parameters of the study. If a monitor is measuring the tone of tweets or coverage for a specific politician, for example, there would be four categories: positive, neutral, negative, and irrelevant for posts that are off-topic in some way.

If a monitor is measuring media framing or storyline, the categories would be more extensive. For example, a monitor studying the framing of coverage about the death of Osama bin Laden might include nine categories: details of the raid, global reaction, political impact, impact on terrorism, role of Pakistan, straight account of events, impact on U.S. policy, the life of bin Laden, and a category off-topic posts.

Fourth, researchers “train” the CH platform to analyze content according to specific parameters they want to study. The PEJ researchers in this role have gone through in-depth training at two different levels. They are professional content analysts fully versed in PEJ’s existing content analysis operation and methodology. They then undergo specific training on the CH platform including multiple rounds of reliability testing.

The monitor training itself is done with a random selection of posts collected by the technology. One at a time, the software displays posts and a human coder determines which category each example best fits into. In categorizing the content, PEJ staff follows coding rules created over the many years that PEJ has been content analyzing news media. If an example does not fit easily into a category, that specific post is skipped. The goal of this training is to feed the software with clear examples for every category.

For each new monitor, human coders categorize at least 250 distinct posts. Typically, each individual category includes 20 or more posts before the training is complete. To validate the training, PEJ has conducted numerous intercoder reliability tests (see below) and the training of every monitor is examined by a second coder in order to discover errors.

The training process consists of researchers showing the algorithm stories in their entirety that are unambiguous in tone. Once the training is complete, the algorithm analyzes content at the assertion level, to ensure that the meaning is similarly unambiguous. This makes it possible to analyze and proportion content that contains assertions of differing tone. This classification is done by applying statistical word patterns derived from posts categorized by human coders during the training process.

How the Algorithm Works

To understand how the software recognizes and uses patterns of words to interpret texts, consider a simplified example. Imagine the study examining news and blog coverage regarding the death of Osama bin Laden that utilizes the nine categories listed above. As a result of the example stories categorized by a human coder during the training, the CH monitor might recognize that portions of a story with the words “Obama,” “poll” and “increase” near each other are likely about the political ramifications. However, a section that includes the words “Obama,” “compound” and “Navy” is likely to be about the details of the raid itself.

Unlike most human coding, CH monitors do not measure each story, blog or tweet as a unit, but examine the entire discussion in the aggregate. To do that, the algorithm breaks up all relevant texts into subsections. Rather than the dividing each story, paragraph, sentence or word, CH treats the “assertion” as the unit of measurement. (Because of the 140 character limit, most tweets consist of a single assertion but this is not a requirement; if a tweet contains two separate assertions, the monitor will assess each one individually.) Thus, posts are divided up by the computer algorithm. If 40% of a story fits into one category, and 60% fits into another, the software will divide the text accordingly. Consequently, the results are not expressed in percent of newshole or percent of stories. Instead, the results are the percent of assertions out of the entire body of stories identified by the original Boolean search terms. We refer to the entire collection of assertions as the “conversation.”

Testing and Validity

Extensive testing by Crimson Hexagon has demonstrated that the tool is 97% reliable, that is, in 97% of cases analyzed, the technology’s coding has been shown to match human coding. PEJ spent more than 12 months testing CH and its own tests comparing coding by humans and the software came up with similar results.

In addition to validity tests of the platform itself, PEJ conducted separate examinations of human intercoder reliability to show that the training process for complex concepts is replicable. The first test had five researchers each code the same 30 stories which resulted in an agreement of 85%.

A second test had each of the five researchers build their own separate monitors to see how the results compared. This test involved not only testing coder agreement, but also how the algorithm handles various examinations of the same content when different human trainers are working on the same subject. The five separate monitors came up with results that were within 85% of each other.

Unlike polling data, the results from the CH tool do not have a sampling margin of error since there is no sampling involved. For the algorithmic tool, reliability tested at 97% meets the highest standards of academic rigor.

Ongoing Monitors

In some instances, PEJ uses CH to study a given period of time, and then expand the monitor for additional time going forward. In order to accomplish this, researchers first create a monitor for the original timeframe according to the method described above.

Because the tenor and content of online conversation can change over time, additional training is necessary if the timeframe gets extended. Since the specific conversation about candidates evolves all the time, the CH monitor must be trained to understand how newer posts fit into the larger categories.

In those instances, researchers conduct additional training for the monitor with a focus on posts that occurred during the new time period. For every new week that is examined, at least 25 more posts are added to the monitor’s training. At that point, the monitor is run to come up with new results for the expanded time period which are added to results that were already derived in the original timeframe.

An Example

Since the use of computer-aided coding is a relatively new phenomenon, it will be helpful to demonstrate how the above procedure works by following a specific example measuring content on Twitter.

PEJ created a monitor to measure the tone of the conversation on Twitter for Republican candidate Newt Gingrich. First, we created a monitor with the following guidelines:

• 1. Source: All public Twitter messages
• 2. Original date range: May 2 to October 16, 2011
• 3. English-language content only
• 4. Keyword: Gingrich

We then created the four categories that are used for measuring tone:

• 1. Positive
• 2. Neutral
• 3. Negative
• 4. Off-topic/Irrelevant

Next, we trained the monitor by classifying documents. CH randomly selected entire tweets from the time period specified, and displayed them one by one. A PEJ researcher decided if each post is a clear example of one of the four categories, and if so, assigned that tweet into the appropriate category. If an example post is not clear in its meaning, or could fit into more than one category, such as a tweet with a mix of positive and negative assertions, the coder skipped the post. Since the goal is to find the clearest cases possible, coders will often skip many posts until they find good examples.

A tweet such as the following: “The ideal ticket…Gingrich to educate the American people about true conservatism,” would be a good example to put in the “positive” category for Gingrich. A different tweet that shows disapproval of Gingrich, such as: “Gingrich is the Newt of all evil,” would be put in the “negative” category. A post that is strictly factual, such as a tweet previewing a debate lineup that reads: “Speaker sked: Ron Paul, Michele Bachmann, Newt Gingrich, Rick Santorum, Rick Perry. Starts at 730. 10 mins each,” would be put in the “neutral” category. And a tweet that includes the word “Gingrich” but is not about the candidate at all, such as a story about a different person with the same last name, would go in the “off-topic” category.

The coder trained 250 documents in all. Each of the four categories had more than 20 posts in them.

At that point, the initial training was finished. For the sake of validity, PEJ has another coder check over all of our training and look for tweets that they would have categorized differently. Those tweets are removed from the training sample because the disagreement between coders shows that they are not clear, precise examples. In the case of the Gingrich monitor, there were six documents removed for this reason.

Finally, we “ran” the monitor. This means that the algorithm examined the word patterns derived from the monitor training, and applied those patterns to every tweet that was captured using the initial guidelines. Since the software studies the conversation in an aggregate as opposed to individual tweets, the algorithm divided up the overall conversation into percentages that fit into the four categories.

For the initial monitor, the algorithm examined over 320,000 assertions from thousands of tweets and determined that 15% of the conversation was positive, 35% neutral, and 50% negative. The assertions or statements that are off-topic were excluded from the results.

In order to extend the Gingrich monitor beyond October 16, coders added at least 25 new tweets of content to the training for each new week examined. This assures that any linguistic changes in the overall coverage or conversation regarding Gingrich in the new week are accounted for. We then run the monitor again each week, which now includes the original training of 250 posts plus 25 new ones, for the new week while leaving the earlier results in place.