{"id":111431,"date":"2021-03-04T15:06:00","date_gmt":"2021-03-04T20:06:00","guid":{"rendered":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/decoded\/\/\/what-different-survey-modes-and-question-types-can-tell-us-about-americans-views-of-china\/"},"modified":"2024-04-14T04:10:35","modified_gmt":"2024-04-14T09:10:35","slug":"what-different-survey-modes-and-question-types-can-tell-us-about-americans-views-of-china","status":"publish","type":"decoded","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/decoded\/2021\/03\/04\/what-different-survey-modes-and-question-types-can-tell-us-about-americans-views-of-china\/","title":{"rendered":"What different survey modes and question types can tell us about Americans\u2019 views of China"},"content":{"rendered":"\n
\"\"<\/a>
Pew Research Center illustration<\/figcaption><\/figure>\n\n\n\n

(Related post: <\/em>Trends are a cornerstone of public opinion research. How do we continue to track changes in public opinion when there\u2019s a shift in survey mode?<\/em><\/a>)<\/em><\/p>\n\n\n\n

Every year since 2002, Pew Research Center has polled people in the U.S. and around the world as part of a major, cross-national study known as the Global Attitudes Survey<\/a>. This year, for the first time, we conducted the U.S. component of the survey through our online American Trends Panel (ATP)<\/a> instead of by telephone. The change is part of a broader shift at the Center, which now conducts nearly all of its U.S. surveys<\/a> using the ATP instead of phone polls.<\/p>\n\n\n\n

As previously explained in this blog<\/a>, a change in survey mode can present problems for researchers trying to interpret trends in public opinion over time. To detect potential mode effects<\/a> ahead of transitioning the U.S. component of the Global Attitudes Survey from the telephone to an internet-based panel, we fielded concurrent surveys online and on the phone in 2020, asking the same questions in both formats. We discovered several notable differences in this experiment, including in the U.S. public\u2019s views of China. For example, the share of Americans with an unfavorable view of China was 13 percentage points higher on the ATP than on the phone (79% vs. 66%). Potential reasons for this disparity include the absence of a \u201cdon\u2019t know\u201d category on the ATP (7% of phone respondents in 2020 answered \u201cdon\u2019t know\u201d) and that self-administered online surveys may be less prone than phone polls carried out by live interviewers to social desirability bias<\/a> effects.<\/p>\n\n\n\n

In this post, we\u2019ll explore additional nuances we recently observed in Americans\u2019 views of China and conclusions we feel comfortable drawing from our Global Attitudes Survey trends in the U.S., given the challenges of a shift in survey mode.<\/p>\n\n\n\n

Background<\/h3>\n\n\n\n

Our U.S. phone surveys in recent years have revealed increasingly negative views of China, with the share of Americans with an unfavorable view rising 26 percentage points between 2018 and 2020<\/a>. As we developed our 2021 survey, we wanted to find out whether this trend has continued, particularly in the context of COVID-19<\/a>, tariffs<\/a>, human rights violations<\/a> and other important issues. To help answer that question, we designed the 2021 survey with two different questions about overall views of China, each one asked of half the sample.<\/p>\n\n\n\n

First, we included a traditional, four-point question we have previously fielded on the phone: \u201cPlease tell me if you have a very favorable, somewhat favorable, somewhat unfavorable or very unfavorable opinion of China.<\/em>\u201d Given the differences we observed in our concurrent web and phone surveys in 2020, we decided we would not be able to \u201ctrend\u201d this question; that is, we would not be able to look at change over time on this particular measure by comparing the 2021 ATP findings to past phone results.<\/p>\n\n\n\n

Second, we included a question that examined Americans\u2019 views of China using a different measure, one for which we do<\/em> have directly comparable ATP data from 2016 and 2018<\/a>. It reads: \u201cWe\u2019d like to get your feelings toward some different countries in the world on a \u2018feeling thermometer.\u2019 A rating of zero degrees means you feel as cold and negative as possible. A rating of 100 degrees means you feel as warm and positive as possible. You would rate the country at 50 degrees if you don\u2019t feel particularly positive or negative toward the country. How do you feel toward China?\u201d<\/em><\/p>\n\n\n\n

Regardless of mode or measure, most Americans have a negative view of China<\/h3>\n\n\n\n

As we examined the results for these two questions in our 2021 survey, a few things became clear.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

First, on both questions, a majority of Americans had negative views of China: 76% of U.S. adults who responded to the standard four-point question in the online panel had an unfavorable view, while 67% of those who were asked the thermometer measure said they had a \u201ccold\u201d feeling (anywhere from 0 to 49 on the scale).<\/p>\n\n\n\n

Second, Americans\u2019 views of China have indeed become substantially more negative in recent years. Looking at the thermometer measure, the share of adults with \u201ccold\u201d feelings toward China rose from 46% in 2018 to 67% in the new survey. The share with \u201cvery cold\u201d feelings (0 to 24 on the thermostat) roughly doubled from 23% to 47%. In our previous phone surveys, we observed a similar rise: In 2018, 47% expressed an unfavorable view of China, a share that climbed to 73% in 2020. Notably, in the ATP version of the four-point question \u2014 the only one for which we are able to make comparisons between 2020 and 2021 \u2014 there was little change in the share of Americans who said they had an unfavorable view of China (79% in 2020, 76% in 2021).<\/p>\n\n\n\n

Third, demographic differences were similar across the two measures, suggesting that the general patterns we found are consistent regardless of how the question is asked. For example, on both the four-point question and the feeling thermometer, men were more likely than women to hold very<\/em> negative views of China. The same was true for older Americans versus younger Americans as well as for Republicans and Republican-leaning independents versus Democrats and Democratic leaners.<\/p>\n\n\n\n

The differences between the two measures related to the \u2018neutral\u2019 option<\/h3>\n\n\n\n

Still, there were some differences between the feeling thermometer and the four-point scale. For example, the share of Americans who expressed a \u201ccold\u201d feeling on the thermometer was lower than the share who said they had an \u201cunfavorable\u201d view on the four-point scale (67% vs. 76%), even as the share of Americans expressing \u201cvery cold\u201d ratings was higher than the share expressing \u201cvery unfavorable\u201d views (47% vs. 39%). In addition, roughly one-in-five Americans rated their feelings as 50 on the thermometer, essentially choosing a \u201cneutral\u201d midpoint that was not available to those who received the four-point question.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

There are various reasons why our respondents might have chosen this neutral category. Some respondents may not have heard much about China and therefore didn\u2019t feel informed enough to answer the question. For some, this may have been an issue of \u201csatisficing\u201d \u2014 that is, choosing an answer that is easier cognitively instead of engaging with the question, just to keep the interview moving along. It\u2019s also of course possible that many respondents genuinely didn\u2019t feel positively or negatively about China.<\/p>\n\n\n\n

Indeed, the results of an open-ended question we asked on the same survey indicated a mix of all of these reasons. Among the responses given by those who chose the neutral option, 6% wrote something like \u201cI don\u2019t know much about China\u201d or suggested that they hadn\u2019t heard enough to write anything in detail. (This was distinct from simply skipping the question.) The other most common responses among this \u201cneutral\u201d group involved China\u2019s economy (including the idea that everything is \u201cmade in China\u201d), China\u2019s relationship with the U.S. (including comments about how the U.S. economy is impacted by China), China\u2019s politics and government, references to human rights abuses, mentions of COVID-19, the idea that China has a large population or is overpopulated, and pollution. (For more on the open-ended responses and the methodology, see \u201cIn their own words: What Americans think about China.\u201d)<\/p>\n\n\n\n

Notably, some demographic groups chose the neutral midpoint at higher rates than others. For example, women were more likely than men to say they felt neither warm nor cold toward China, as were Democrats compared with Republicans.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

The pros and cons of each measure<\/h3>\n\n\n\n

There are some key differences between the feeling thermometer and the four-point scale. The thermometer is administered using text box entry and 101 answer options \u2014 any number from 0 to 100. We then assign meaning to the 101 values by grouping them into categories, treating 50 as a neutral \u201cneither cold nor warm\u201d group. The four-point scale, in contrast, involves a conventional display where respondents choose one of four visible options.<\/p>\n\n\n\n

We often stay away from using scales with neutral midpoint options (unless the midpoint is itself a useful category<\/a>) because \u201csatisficing\u201d is a well-documented problem in survey research. When asked the four-point question, which does not have a neutral middle category, most respondents pick a side rather than skip the question altogether. We can then also see the degree of these feelings \u2014 \u201cvery\u201d or \u201csomewhat\u201d \u2014 which is valuable for understanding political and social attitudes. In some ways, these factors make our traditional four-point question preferable to the feeling thermometer.<\/p>\n\n\n\n

The four-point scale also leads to easier analysis and reporting. There are fewer categories, and readers can more easily grasp what \u201cfavorable\u201d and \u201cunfavorable\u201d mean. The thermometer, on the other hand, requires us to explain how terms relate to numerical codes (e.g., \u201ccold\u201d is 0 to 49). Additionally, Pew Research Center\u2019s international survey work is often conducted on the phone. While we typically include a visual prompt to assist with the thermometer question in our online polls, we can\u2019t do that over the phone, making the question more cumbersome for the respondent.<\/p>\n\n\n\n

Still, the thermometer does have some advantages. Respondents are given the opportunity to pick the position closest to how they feel with greater precision. In our 2021 survey, for instance, 32% of Americans said their views of China fell below 10 on the 0 to 100 scale, including 24% who said their feeling was a zero out of 100. Density around an endpoint with this level of detail is something we don\u2019t see on a four-point scale. Additionally, researchers have several choices of how they can analyze the data. We focused here on frequencies, but one could instead look at mean ratings. We concentrated most of our analysis on the 0\u201325 \u201cvery cold\u201d feelings discussed in our report, but researchers could also examine those who choose 0\u20139 or zero itself, offering a greater degree of flexibility.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Conclusion<\/h3>\n\n\n\n

We see that across modes and measures a majority of Americans have negative views of China, and that unfavorable views have increased in recent years. In addition, thanks to simultaneously asking a feeling thermometer question, we gained additional insight into the share of Americans who may not feel particularly warm or cold toward China as well as the intensity of the animosity some Americans feel.<\/p>\n\n\n\n

Given the challenge of effectively and efficiently deploying a feeling thermometer question by phone, we are unlikely to abandon our four-point measure of attitudes toward China anytime soon. But we now have knowledge about split samples and alternate measures that can aid the Center as we contemplate similar mode transitions in other countries.<\/p>\n","protected":false},"excerpt":{"rendered":"

Regardless of mode or measure, most Americans have a negative view of China \u2013 and opinions have become more unfavorable in recent years.<\/p>\n","protected":false},"author":377,"featured_media":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"sub_headline":"","sub_title":"","_crdt_document":"","_prc_public_revisions":[],"_ppp_expiration_hours":0,"_ppp_enabled":false,"ai_generated_summary":"","relatedPosts":[],"_prc_fork_parent":0,"_prc_fork_status":"","_prc_active_fork":0,"datacite_doi":"","datacite_doi_citation":"","_prc_seo_qr_attachment_id":0,"spoken_article_player_enabled":true,"footnotes":""},"categories":[357],"bylines":[686,954],"collection":[],"_post_visibility":[],"decoded-category":[537,532],"formats":[454],"_fund_pool":[],"languages":[],"regions-countries":[],"research-teams":[524],"class_list":["post-111431","decoded","type-decoded","status-publish","hentry","category-survey-methods","bylines-jerry-joseph-moncus","bylines-laura-silver","decoded-category-international-surveys","decoded-category-survey-methods","formats-decoded","research-teams-decoded"],"label":"Decoded","post_parent":0,"word_count":1744,"canonical_url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/decoded\/2021\/03\/04\/what-different-survey-modes-and-question-types-can-tell-us-about-americans-views-of-china\/","art_direction":{"A1":{"id":125868,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/2021.03.04_decoded_featured.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/2021.03.04_decoded_featured.png?w=564&h=317&crop=1","width":564,"height":317,"caption":"","chartArt":false},"A2":{"id":125868,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/2021.03.04_decoded_featured.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/2021.03.04_decoded_featured.png?w=268&h=151&crop=1","width":268,"height":151,"caption":"","chartArt":false},"A3":{"id":125868,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/2021.03.04_decoded_featured.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/2021.03.04_decoded_featured.png?w=194&h=110&crop=1","width":194,"height":110,"caption":"","chartArt":false},"A4":{"id":125868,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/2021.03.04_decoded_featured.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/2021.03.04_decoded_featured.png?w=268&h=151&crop=1","width":268,"height":151,"caption":"","chartArt":false},"XL":{"id":125868,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/2021.03.04_decoded_featured.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/2021.03.04_decoded_featured.png?w=720&h=405&crop=1","width":720,"height":405,"caption":"","chartArt":false},"social":{"id":125868,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/2021.03.04_decoded_featured.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/2021.03.04_decoded_featured.png?w=1200&h=628&crop=1","width":1200,"height":628,"caption":"","chartArt":false}},"_embeds":[],"table_of_contents":[],"datacite_doi":"","prc_seo_data":{"title":"What different survey modes and question types can tell us about Americans\u2019 views of China","description":"Regardless of mode or measure, most Americans have a negative view of China \u2013 and opinions have become more unfavorable in recent years.","og_title":"What different survey modes and question types can tell us about Americans\u2019 views of China","og_description":"Regardless of mode or measure, most Americans have a negative view of China \u2013 and opinions have become more unfavorable in recent years.","schema_type":"Article","noindex":false,"canonical_url":"","primary_terms":{"category":43},"custom_schema":[],"og_image":125868,"indexnow_submitted_at":null,"gsc_index_status":null},"jetpack_sharing_enabled":true,"relatedPostsOrdered":[],"bylinesOrdered":[{"key":"_1l0ki331e","termId":686},{"key":"_4g87pflec","termId":954}],"acknowledgementsOrdered":[],"_links":{"self":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/decoded\/111431","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/decoded"}],"about":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/types\/decoded"}],"author":[{"embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/users\/377"}],"replies":[{"embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/comments?post=111431"}],"version-history":[{"count":2,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/decoded\/111431\/revisions"}],"predecessor-version":[{"id":138516,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/decoded\/111431\/revisions\/138516"}],"wp:attachment":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/media?parent=111431"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/categories?post=111431"},{"taxonomy":"bylines","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/bylines?post=111431"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/collection?post=111431"},{"taxonomy":"_post_visibility","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_post_visibility?post=111431"},{"taxonomy":"decoded-category","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/decoded-category?post=111431"},{"taxonomy":"formats","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/formats?post=111431"},{"taxonomy":"_fund_pool","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_fund_pool?post=111431"},{"taxonomy":"languages","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/languages?post=111431"},{"taxonomy":"regions-countries","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/regions-countries?post=111431"},{"taxonomy":"research-teams","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/research-teams?post=111431"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}