{"id":11113,"date":"2018-12-06T10:01:13","date_gmt":"2018-12-06T15:01:13","guid":{"rendered":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/short-reads\/%year%\/%monthnum%\/%day%\/how-asking-about-your-sleep-smoking-or-yoga-habits-can-help-pollsters-verify-their-findings\/"},"modified":"2024-04-14T03:14:54","modified_gmt":"2024-04-14T08:14:54","slug":"how-asking-about-your-sleep-smoking-or-yoga-habits-can-help-pollsters-verify-their-findings","status":"publish","type":"short-read","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/short-reads\/2018\/12\/06\/how-asking-about-your-sleep-smoking-or-yoga-habits-can-help-pollsters-verify-their-findings\/","title":{"rendered":"How asking about your sleep, smoking or yoga habits can help pollsters verify their findings"},"content":{"rendered":"
\"Typical<\/a><\/p>\n

\"\"
Acharaporn Kamornboonyarush\/EyeEm, Patrick Foto, Jupiterimages (all via Getty)<\/figcaption><\/figure><\/figure>\n\n

Have you smoked at least 100 cigarettes in your life? Have you practiced yoga in the past year? On average, how many hours of sleep do you get in a day?<\/p>\n\n

At face value, these questions are not directly related to the topics Pew Research Center is most committed to studying. Yet our researchers have been periodically asking questions like these for years. Why? These are examples of benchmarking questions, which the Center uses as a check to ensure that our surveys are accurate.<\/p>\n\n

Why and how we use benchmarking questions<\/strong><\/p>\n\n

Determining the accuracy of a survey requires some sort of objective standard against which the survey can be compared. In election polls and other measures of voting intent, the standard is the outcome of the election. But for surveys that don\u2019t ask about elections or voting intent, researchers need to find another way to benchmark their findings. This is often done with the help of other surveys \u2013 usually large, expensive, government surveys conducted with great attention to data quality.<\/p>\n\n

Pew Research Center surveys occasionally include questions about economic, demographic and lifestyle characteristics for which government statistics are available as a benchmark. This not only helps us check the accuracy of our findings, it also helps us study how surveys themselves can be better conducted.<\/p>\n\n

Take, for example, a Pew Research Center study from last year that examined what low response rates<\/a> \u2013 many potential respondents being contacted but far fewer of them participating \u2013 mean for the accuracy of telephone surveys. To help answer this question, the study compared the results of a telephone survey by the Center with those of high-response, benchmark surveys by the federal government to see what, if any, differences existed.<\/p>\n\n

\"On<\/figure>\n\n

The report found that Pew Research Center surveys were closely aligned with federal surveys on key demographic and lifestyle benchmarks. Across 14 questions about personal traits, the average difference between the government estimate and the Center\u2019s telephone survey estimate was 3 percentage points. Differences on individual questions ranged from 0 to 8 points. The largest was on a measure asking respondents about their health status: The government found that 59% of people rated their health as very good or excellent, while the Center\u2019s telephone survey found 51% doing so.<\/p>\n\n

<\/p>\n\n

The other 13 items were quite close to the benchmarks, most with differences of 3 percentage points or less, which was generally within the margin of error.\u00a0These questions included measures of family income, employment status, household size, citizenship, health insurance, length of residence at current\u00a0address, marital and parenthood status, smoking frequency, place of birth (among Hispanics) and having a driver\u2019s license. In other words, on these measures, the low-response telephone survey provided results quite comparable to those of the high-response government survey used as a benchmark.<\/p>\n\n

\"Reports<\/figure>\n\n

Overall, the report showed that bias introduced into surveys due to low response rates remains limited in scope. And, critically, telephone poll estimates for party affiliation, political ideology and religious affiliation continue to track well with estimates from high-response-rate benchmark surveys.<\/p>\n\n

However, as the Center and other survey researchers have discussed extensively<\/a>, telephone surveys continue to yield large biases on measures of civic and, to a lesser extent, political engagement. This discrepancy is probably because of nonresponse bias<\/a> \u2013 in which the kinds of people agreeing to participate in surveys are systematically different from those who can\u2019t be contacted or refuse to participate. As found in previous work<\/a>, the people who answer surveys are likely to be the same people involved in community life \u2013 they are joiners, and participating in surveys is a kind of pro-social behavior<\/a> related to other kinds of behaviors such as volunteering. Fortunately for pollsters, civic engagement is not strongly correlated<\/a> with political attitudes or most other measures researchers study in surveys.<\/p>\n\n

Caveats about benchmarks<\/strong><\/p>\n\n

\"Politically<\/figure>\n\n

Although large government surveys are generally considered to have high data quality, they\u2019re not immune to some of the same problems every survey researcher faces. For example, while government surveys tend to have very high response rates (on the order of 60% or more) compared with opinion polls conducted by other organizations, the risk of nonresponse bias<\/a> still exists.<\/p>\n\n

Government surveys, while carefully developed and tested, are also still subject to measurement error<\/a>, which can arise from the way in which questions are asked (such as what questions come immediately before a particular question, whether the survey was conducted on the phone or online, etc.). Pew Research Center questionnaires that include benchmarking questions do not replicate the exact context in which the original questions were asked, particularly because the Center tends to focus on topics that are different from those in benchmark surveys. Benchmarks also are generally unavailable for questions about attitudes and behaviors that the government does not study.<\/p>\n\n

All surveys can also face response bias issues, including social desirability bias, where respondents may modify answers to certain questions to present themselves more favorably. This is especially a risk when an interviewer asks sensitive questions: Respondents may, for example, overstate their voting frequency.<\/p>\n\n

All of these factors can affect the comparability of seemingly identical questions asked on different surveys, including government surveys. That said, benchmarking questions continue to be a valuable tool for survey researchers checking and assessing accuracy. They are especially vital for the Center\u2019s studies on survey methodology<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"

Pew Research Center uses benchmarking questions to ensure our surveys are accurate. Learn why and how we use these questions.<\/p>\n","protected":false},"author":386,"featured_media":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"sub_headline":"","sub_title":"","_crdt_document":"","_prc_public_revisions":[],"_ppp_expiration_hours":0,"_ppp_enabled":false,"ai_generated_summary":"","apple_news_api_created_at":"","apple_news_api_id":"","apple_news_api_modified_at":"","apple_news_api_revision":"","apple_news_api_share_url":"","apple_news_cover_media_provider":"image","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_cover_video_id":0,"apple_news_cover_video_url":"","apple_news_cover_embedwebvideo_url":"","apple_news_is_hidden":"","apple_news_is_paid":"","apple_news_is_preview":"","apple_news_is_sponsored":"","apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_sections":[],"apple_news_suppress_video_url":false,"apple_news_use_image_component":false,"relatedPosts":[],"_prc_fork_parent":0,"_prc_fork_status":"","_prc_active_fork":0,"datacite_doi":"","datacite_doi_citation":"","_prc_seo_qr_attachment_id":0,"spoken_article_player_enabled":true,"bylines":[{"key":"282c96d5ec4e3965393ba0b6104c4d25","termId":828}],"acknowledgements":[],"displayBylines":true,"footnotes":"","prc_watchers":[]},"categories":[36,401,357],"bylines":[828],"collection":[],"datasets":[],"_post_visibility":[],"formats":[467],"_fund_pool":[],"languages":[],"regions-countries":[515],"research-teams":[528],"workflow-status":[],"class_list":["post-11113","short-read","type-short-read","status-publish","hentry","category-methodological-research","category-survey-basics","category-survey-methods","bylines-kristen-bialik","formats-short-read","regions-countries-united-states","research-teams-methods"],"label":"Short Read","post_parent":0,"word_count":886,"canonical_url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/short-reads\/2018\/12\/06\/how-asking-about-your-sleep-smoking-or-yoga-habits-can-help-pollsters-verify-their-findings\/","art_direction":{"A1":{"id":21509,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/12\/FT_18.11.20_Benchmarking_feature.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/12\/FT_18.11.20_Benchmarking_feature.png?w=564&h=317&crop=1","width":564,"height":317,"chartArt":false},"A2":{"id":21509,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/12\/FT_18.11.20_Benchmarking_feature.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/12\/FT_18.11.20_Benchmarking_feature.png?w=268&h=151&crop=1","width":268,"height":151,"chartArt":false},"A3":{"id":21509,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/12\/FT_18.11.20_Benchmarking_feature.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/12\/FT_18.11.20_Benchmarking_feature.png?w=194&h=110&crop=1","width":194,"height":110,"chartArt":false},"A4":{"id":21509,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/12\/FT_18.11.20_Benchmarking_feature.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/12\/FT_18.11.20_Benchmarking_feature.png?w=268&h=151&crop=1","width":268,"height":151,"chartArt":false},"XL":{"id":21509,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/12\/FT_18.11.20_Benchmarking_feature.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/12\/FT_18.11.20_Benchmarking_feature.png?w=640&h=360&crop=1","width":640,"height":360,"chartArt":false},"social":{"id":21509,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/12\/FT_18.11.20_Benchmarking_feature.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/12\/FT_18.11.20_Benchmarking_feature.png?w=640&h=360&crop=1","width":640,"height":360,"chartArt":false}},"_embeds":[],"watchers":[],"table_of_contents":[],"datacite_doi":"","prc_seo_data":{"title":"Benchmarking questions: A way to check a survey's accuracy","description":"Pew Research Center uses benchmarking questions to ensure our surveys are accurate. Learn why and how we use these questions.","og_title":"How asking about your sleep, smoking or yoga habits can help pollsters verify their findings","og_description":"Pew Research Center uses benchmarking questions to ensure our surveys are accurate. Learn why and how we use these questions.","schema_type":"Article","noindex":false,"canonical_url":"","primary_terms":[],"custom_schema":[],"twitter_title":"How asking about your sleep, smoking or yoga habits can help pollsters verify their findings","twitter_description":"Pew Research Center uses benchmarking questions to ensure our surveys are accurate. Learn why and how we use these questions.","og_image":21509,"indexnow_submitted_at":null,"gsc_index_status":null},"prepublish_checks":{},"apple_news_notices":[],"jetpack_sharing_enabled":true,"relatedPostsOrdered":[],"bylinesOrdered":[{"key":"282c96d5ec4e3965393ba0b6104c4d25","termId":828}],"acknowledgementsOrdered":[],"_links":{"self":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/short-read\/11113","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/short-read"}],"about":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/types\/short-read"}],"author":[{"embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/users\/386"}],"replies":[{"embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/comments?post=11113"}],"version-history":[{"count":2,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/short-read\/11113\/revisions"}],"predecessor-version":[{"id":101867,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/short-read\/11113\/revisions\/101867"}],"wp:attachment":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/media?parent=11113"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/categories?post=11113"},{"taxonomy":"bylines","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/bylines?post=11113"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/collection?post=11113"},{"taxonomy":"datasets","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/datasets?post=11113"},{"taxonomy":"_post_visibility","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_post_visibility?post=11113"},{"taxonomy":"formats","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/formats?post=11113"},{"taxonomy":"_fund_pool","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_fund_pool?post=11113"},{"taxonomy":"languages","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/languages?post=11113"},{"taxonomy":"regions-countries","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/regions-countries?post=11113"},{"taxonomy":"research-teams","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/research-teams?post=11113"},{"taxonomy":"workflow-status","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/workflow-status?post=11113"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}