{"id":191544,"date":"2024-11-18T09:45:27","date_gmt":"2024-11-18T14:46:20","guid":{"rendered":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/?p=191544"},"modified":"2025-11-04T10:23:47","modified_gmt":"2025-11-04T15:23:47","slug":"news-influencers-methodology","status":"publish","type":"post","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/journalism\/2024\/11\/18\/news-influencers-methodology\/","title":{"rendered":"Methodology"},"content":{"rendered":"\n

For our latest survey data about news influencers,\u00a0read the \u201cNews Influencers Fact Sheet.\u201d<\/a><\/em><\/p>\n\n\n\n

This report \u2013 a study of popular news influencers \u2013 uses three different research components and methodologies, including a nationally representative survey of U.S. adults conducted through Pew Research Center\u2019s American Trends Panel (ATP), as well as an analysis of news influencers and posts from those accounts from three distinct time periods in 2024.<\/p>\n\n\n\n

This is the latest report from the Pew-Knight Initiative, a research program funded jointly by The Pew Charitable Trusts and the John S. and James L. Knight Foundation. Find related reports online at https:\/\/alpha.pewresearch.org\/pewresearch-org\/pew-knight\/<\/a>.<\/p>\n\n\n\n

Survey<\/h3>\n\n\n\n

Overview<\/h4>\n\n\n\n

Data in this report comes from Wave 150 of the American Trends Panel (ATP), Pew Research Center\u2019s nationally representative panel of randomly selected U.S. adults. The survey was conducted from July 15 to Aug. 4, 2024. A total of 10,658 panelists responded out of 12,188 who were sampled, for a survey-level response rate of 87%.<\/p>\n\n\n\n

The cumulative response rate accounting for nonresponse to the recruitment surveys and attrition is 3%. The break-off rate among panelists who logged on to the survey and completed at least one item is less than 1%. The margin of sampling error for the full sample of 10,658 respondents is plus or minus 1.2 percentage points.<\/p>\n\n\n\n

SSRS conducted the survey for Pew Research Center via online (n=10,262) and live telephone (n=396) interviewing. Interviews were conducted in both English and Spanish.<\/p>\n\n\n\n

To learn more about the ATP, read \u201cAbout the American Trends Panel<\/a>.\u201d<\/p>\n\n\n\n

Panel recruitment<\/h4>\n\n\n\n

Since 2018, the ATP has used address-based sampling (ABS) for recruitment. A study cover letter and a pre-incentive are mailed to a stratified, random sample of households selected from the U.S. Postal Service\u2019s Computerized Delivery Sequence File. This Postal Service file has been estimated to cover 90% to 98% of the population.[3. numoffset=”3″ AAPOR Task Force on Address-based Sampling. 2016. \u201cAAPOR Report: Address-based Sampling<\/a>.\u201d] Within each sampled household, the adult with the next birthday is selected to participate. Other details of the ABS recruitment protocol have changed over time but are available upon request.[4. Email pewsurveys@pewresearch.org<\/a>.] Prior to 2018, the ATP was recruited using landline and cellphone random-digit-dial surveys administered in English and Spanish.<\/p>\n\n\n\n

A national sample of U.S. adults has been recruited to the ATP approximately once per year since 2014. In some years, the recruitment has included additional efforts (known as an \u201coversample\u201d) to improve the accuracy of data for underrepresented groups. For example, Hispanic adults, Black adults and Asian adults were oversampled in 2019, 2022 and 2023, respectively.<\/p>\n\n\n\n

Sample design<\/h4>\n\n\n\n

The overall target population for this survey was noninstitutionalized persons ages 18 and older living in the United States. All active panel members were invited to participate in this wave.<\/p>\n\n\n\n

Questionnaire development and testing<\/h4>\n\n\n\n

The questionnaire was developed by Pew Research Center in consultation with SSRS. The web program used for online respondents was rigorously tested on both PC and mobile devices by the SSRS project team and Pew Research Center researchers. The SSRS project team also populated test data that was analyzed in SPSS to ensure the logic and randomizations were working as intended before launching the survey.<\/p>\n\n\n\n

Incentives<\/h4>\n\n\n\n

All respondents were offered a post-paid incentive for their participation. Respondents could choose to receive the post-paid incentive in the form of a check or gift code to Amazon.com. Incentive amounts ranged from $5 to $20 depending on whether the respondent belongs to a part of the population that is harder or easier to reach. Differential incentive amounts were designed to increase panel survey participation among groups that traditionally have low survey response propensities.<\/p>\n\n\n\n

Data collection protocol<\/h4>\n\n\n\n

The data collection field period for this survey was July 15 to Aug. 4, 2024. Surveys were conducted via self-administered web survey or by live telephone interviewing. <\/p>\n\n\n\n

For panelists who take surveys online:<\/strong>[5. The ATP does not use routers or chains in any part of its online data collection protocol, nor are they used to direct respondents to additional surveys.] Postcard notifications were mailed to a subset on July 15.[6. Postcard notifications for web panelists are sent to 1) panelists who were recruited within the last two years and 2) panelists recruited prior to the last two years who opt to continue receiving postcard notifications.] Survey invitations were sent out in two separate launches: soft launch and full launch. Sixty panelists were included in the soft launch, which began with an initial invitation sent on July 15. All remaining English- and Spanish-speaking sampled online panelists were included in the full launch and were sent an invitation on July 16.<\/p>\n\n\n\n

\"Table<\/a><\/figure>\n\n\n\n

Panelists participating online were sent an email invitation and up to six email reminders if they did not respond to the survey. ATP panelists who consented to SMS messages were sent an SMS invitation with a link to the survey and up to six SMS reminders.<\/p>\n\n\n\n

For panelists who take surveys over the phone with a live interviewer:<\/strong> Prenotification postcards were mailed on July 10, and reminder postcards were mailed on July 15. Ten CATI panelists were included in the soft launch, for which dialing began on July 15. CATI panelists receive up to six calls from trained SSRS interviewers.<\/p>\n\n\n\n

Data quality checks<\/h4>\n\n\n\n

To ensure high-quality data, Center researchers performed data quality checks to identify any respondents showing patterns of satisficing. This includes checking for whether respondents left questions blank at very high rates or always selected the first or last answer presented. As a result of this checking, five ATP respondents were removed from the survey dataset prior to weighting and analysis.<\/p>\n\n\n\n

Weighting<\/h4>\n\n\n\n

The ATP data is weighted in a process that accounts for multiple stages of sampling and nonresponse that occur at different points in the panel survey process. First, each panelist begins with a base weight that reflects their probability of recruitment into the panel. These weights are then calibrated to align with the population benchmarks in the accompanying table to correct for nonresponse to recruitment surveys and panel attrition. If only a subsample of panelists was invited to participate in the wave, this weight is adjusted to account for any differential probabilities of selection.<\/p>\n\n\n\n

\"Table<\/a><\/figure>\n\n\n\n

Among the panelists who completed the survey, this weight is then calibrated again to align with the population benchmarks identified in the accompanying table. The weight is then trimmed at the 1st and 99th percentiles to reduce the loss in precision stemming from variance in the weights. Sampling errors and tests of statistical significance take into account the effect of weighting.<\/p>\n\n\n\n

The following table shows the unweighted sample size and the error attributable to sampling that would be expected at the 95% level of confidence for the full sample in the survey.<\/p>\n\n\n\n

\"Table<\/a><\/figure>\n\n\n\n

Sample sizes and sampling errors for other subgroups are available upon request. In addition to sampling error, one should bear in mind that question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of opinion polls.<\/p>\n\n\n\n

Dispositions and response rates<\/h4>\n\n\n\n
\"Table<\/a><\/figure>\n\n\n\n
\"Table<\/a><\/figure>\n\n\n\n

How family income tiers are calculated<\/h4>\n\n\n\n

Family income data reported in this study is adjusted for household size and cost-of-living differences by geography. Panelists then are assigned to income tiers that are based on the median adjusted family income of all American Trends Panel members. The process uses the following steps:<\/p>\n\n\n\n

    \n
  1. First, panelists are assigned to the midpoint of the income range they selected in a family income question that was measured on either the most recent annual profile survey or, for newly recruited panelists, their recruitment survey. This provides an approximate income value that can be used in calculations for the adjustment.<\/li>\n\n\n\n
  2. Next, these income values are adjusted for the cost of living in the geographic area where the panelist lives. This is calculated using price indexes published by the U.S. Bureau of Economic Analysis. These indexes, known as Regional Price Parities<\/a> (RPP), compare the prices of goods and services across all U.S. metropolitan statistical areas as well as non-metro areas with the national average prices for the same goods and services. The most recent available data at the time of the annual profile survey is from 2022. Those who fall outside of metropolitan statistical areas are assigned the overall RPP for their state\u2019s non-metropolitan area.<\/li>\n\n\n\n
  3. Family incomes are further adjusted for the number of people in a household using the methodology from Pew Research Center\u2019s previous work on the American middle class<\/a>. This is done because a four-person household with an income of say, $50,000, faces a tighter budget constraint than a two-person household with the same income.<\/li>\n\n\n\n
  4. Panelists are then assigned an income tier. \u201cMiddle-income\u201d adults are in families with adjusted family incomes that are between two-thirds and double the median adjusted family income for the full ATP at the time of the most recent annual profile survey. The median adjusted family income for the panel is roughly $74,100. Using this median income, the middle-income range is about $49,400 to $148,200. Lower-income families have adjusted incomes less than $49,400 and upper-income families have adjusted incomes greater than $148,200 (all figures expressed in 2023 dollars and scaled to a household size of three). If a panelist did not provide their income and\/or their household size, they are assigned \u201cno answer\u201d in the income tier variable.<\/li>\n<\/ol>\n\n\n\n

    Two examples of how a given area\u2019s cost-of-living adjustment was calculated are as follows: The Pine Bluff metropolitan area in Arkansas is a relatively inexpensive area, with a price level that is 19.1% less than the national average. The San Francisco-Oakland-Berkeley metropolitan area in California is one of the most expensive areas, with a price level that is 17.9% higher than the national average. Income in the sample is adjusted to make up for this difference. As a result, a family with an income of $40,400 in the Pine Bluff area is as well off financially as a family of the same size with an income of $58,900 in San Francisco. <\/p>\n\n\n\n

    Analysis of news influencers<\/h3>\n\n\n\n

    Overview<\/h4>\n\n\n\n

    To better understand the news influencers that Americans are getting news from on social media, researchers conducted an in-depth analysis of a sample of news influencers who posted about current events and civic issues on at least one of five major social media sites where news is often discussed: Facebook, Instagram, TikTok, X and YouTube. The analysis looked at personal characteristics of each news influencer as well as their online practices.<\/p>\n\n\n\n

    \"Infographic<\/a><\/figure>\n\n\n\n

    For the purposes of this research, news influencers are defined as people who regularly post about current events and civic issues on social media and have at least 100,000 followers on any of Facebook, Instagram, TikTok, X or YouTube. News influencers can be journalists who are affiliated with a news organization or independent content creators, but they must be people<\/em> and not institutions.<\/p>\n\n\n\n

    Account collection<\/h4>\n\n\n\n
    Overview<\/h5>\n\n\n\n

    There are very limited lists of news influencers, and few resources to verify those that do exist. Because of this, we compiled our own list of 2,058 news influencers by searching for accounts that regularly used news-related keywords on Facebook, Instagram, TikTok, X and YouTube.<\/p>\n\n\n\n

    Keyword development<\/h5>\n\n\n\n

    The first step in this process was developing a list of news-related keywords that are relevant to current events and civic issues in the United States. Throughout late 2023 and early 2024, four researchers with expertise in the news media brainstormed related keywords and grouped them into 45 categories ranging from affirmative action to free speech to immigration. Each category had multiple keywords \u2013 sometimes dozens.<\/p>\n\n\n\n

    In order to ensure we were not missing news events as they were occurring, in January 2024, we conducted daily reviews of transcripts of news on CNN, Fox News and MSNBC, as well as news highlights from NewsMax\u2019s YouTube page and a review of Breitbart and Vox\u2019s social media and homepage (to ensure a mix of the key points of discussion across the political spectrum). For the cable news channels, researchers downloaded transcripts from The Internet Archive\u2019s TV Archive<\/a> for each day in January 2024. Once downloaded, episodes were sent to OpenAI\u2019s GPT-4 model with the instructions:<\/p>\n\n\n\n

    \n

    \u201cYou are an AI assistant that helps extract information from text. You will be given a set of transcripts from cable news shows. For each episode, please tell us the main topics that are discussed.\u201d<\/p>\n<\/div>\n\n\n\n

    \n

    Each day in January 2024, this team of researchers reviewed each summary statement, along with a manual check of NewsMax\u2019s homepage, YouTube page, and social media; Vox\u2019s homepage and social media; and Breitbart\u2019s homepage and social media to discover major stories and potential related keywords. During the keyword brainstorming process, we searched for these keywords on social media sites to determine the extent to which they were being discussed on social media.<\/p>\n\n\n\n

    The resulting set of 45 keyword categories plus keywords developed through this daily news review were used on each site, although the keywords were sometimes changed to reflect common practices of the site (such as the tendency to use hashtags) and the tools we used.<\/p>\n\n\n\n

    Collecting accounts<\/h5>\n\n\n\n

    The general process for collecting accounts was similar across sites. Because automated search capabilities on these sites are limited, we used a set of social media monitoring tools to collect accounts. Once accounts were collected, we filtered for the number of times each account had a post that matched any keywords during the collection period because we wanted to limit the analysis to accounts that were regularly posting about news. The exceptions were TikTok and Instagram, as explained below.<\/p>\n<\/blockquote>\n\n\n\n

    To augment this automated collection, we added influencers who were mentioned in articles about news influencers by prominent news outlets.<\/p>\n\n\n\n

    This collection process resulted in 3,041,652 accounts across Facebook, Instagram, X and YouTube. <\/p>\n\n\n\n

    After collection, but before manually visiting any account profiles, we looked at the names of accounts that had terms shared with at least nine other accounts. Those terms included publishing terms like \u201cradio\u201d or \u201cTV\u201d that suggested a news organization, country (e.g., \u201cIndia\u201d) and place names (e.g., \u201cBoston\u201d), and large institutions like the United Nations (\u201cUN\u201d) or Congress (\u201cCongress\u201d). During this process, two researchers quickly evaluated whether a given account was obviously an organization and not an individual and met to discuss accounts that they disagreed about. A total of 4,566 accounts across Facebook, Instagram, X and YouTube were removed this way.<\/p>\n\n\n\n

    Facebook<\/h4>\n\n\n\n

    To develop a list of news influencers on Facebook, we turned to CrowdTangle, a social media marketing tool from Meta (Meta shut down CrowdTangle<\/a> in August 2024). Only public Facebook pages were considered, not groups or private pages. We added two filters to this collection inside CrowdTangle: The minimum number of page followers needed to be 100,000, and they needed to post primarily in English. Across keywords, we collected 358,186 potential news influencer accounts. We further filtered this to only those accounts that posted about any news-related topic at least 30 times in January 2024 (excluding duplicate posts). This resulted in 7,218 accounts, each of which was manually reviewed.<\/p>\n\n\n\n

    Instagram<\/h4>\n\n\n\n

    To develop a list of news influencers on Instagram, we first turned to CrowdTangle, a social media marketing tool from Meta (Meta shut down CrowdTangle<\/a> in August 2024). We added two filters to this collection inside CrowdTangle: The minimum number of followers needed to be 50,000 (during the consolidation process, that follower threshold was later raised to 100,000) and they needed to post primarily in English. Across keywords, we collected 181,850 potential news influencer accounts. We further filtered this to only those accounts that posted about any topic at least 15 times in January 2024 (excluding duplicate posts; we noticed in testing that the overall rate of posting was lower on Instagram than other sites, so we lowered this threshold). After removing accounts with organization names, we were left with 3,818 accounts, each of which was manually reviewed.<\/p>\n\n\n\n

    Upon inspection, we realized that several Instagram accounts that we had found in media coverage were not picked up by CrowdTangle. Out of concern for the comprehensiveness of this data, we augmented the list of influencers from CrowdTangle with those from two sources: first, 49 influencers we had found in news articles that CrowdTangle did not pick up, and second, using Modash.io, a social media marketing tool. Because we already had a list of potential news influencers (including the ones we found in news articles), we used Modash\u2019s account relevancy tool, which returns accounts that are similar to any provided account. For each account provided, we collected 45 relevant accounts (or three pages of results). Because there was some overlap between the sets of relevant accounts, this resulted in an additional 2,460 accounts, each of which was manually reviewed.<\/p>\n\n\n\n

    TikTok<\/h4>\n\n\n\n

    To develop a list of news influencers on TikTok we used Modash.io, a social media marketing tool. Whereas other similar tools provide lists of posts<\/em> that match a set of keywords, Modash\u2019s search function for TikTok lists accounts<\/em> that used any of the provided keywords in the last two months, resulting in 55,223 accounts. After filtering for accounts with at least 100,000 followers, we were left with 3,342 accounts, each of which was manually reviewed.<\/p>\n\n\n\n

    X (formerly Twitter)<\/h4>\n\n\n\n

    To develop a list of news influencers on X, we used Meltwater, a social media marketing and analysis tool that can search the entire X firehose<\/a>. Meltwater includes some advanced filters, which we used in our collection process, as well as some quirks in how it delivers data. We searched for all English-language posts from January 2024 that used our keywords (including in hashtags) and were from accounts with at least 100,000 followers.<\/p>\n\n\n\n

    Because Meltwater only lets you download 20,000 posts at a time from its dashboard, we removed retweets and accounts that were known to be politicians, and then limited the time and keyword scope of each search we conducted to be below that 20,000-post threshold whenever possible, and posts were sorted by engagement before downloading. After downloading each post, we used only the account information and the number of posts matching each keyword \u2013 none of the other post data was used in analysis. This resulted in 39,618 accounts.<\/p>\n\n\n\n

    We then limited our analysis to accounts that had at least 30 posts in January and that matched our keywords. After removing accounts with organization names, we were left with 7,803 accounts, each of which was manually reviewed.<\/p>\n\n\n\n

    YouTube<\/h4>\n\n\n\n

    To develop a list of news influencers on YouTube, we used Meltwater, a social media marketing and analysis tool. Meltwater searches the titles and descriptions of YouTube videos from the last 30 days to match keywords but cannot filter YouTube by language or follower count.<\/p>\n\n\n\n

    Searching YouTube through Meltwater for our keywords in January 2024 resulted in 2,461,998 accounts. We then limited this set to accounts that used those keywords at least 15 times (a lower number than other sites because we found that most accounts posted less frequently). Then, using the YouTube API<\/a>, we collected follower counts for each account and filtered out accounts with fewer than 100,000 followers. After removing accounts with organization names, we were left with 3,576 accounts, each of which was manually reviewed.<\/p>\n\n\n\n

    Account verification and consolidation<\/h4>\n\n\n\n

    After this initial sample was concluded, researchers examined all 28,266 accounts that met our criteria to determine if the account was connected to a news influencer. We operationalized the above definition using several criteria:<\/p>\n\n\n\n