{"id":101127,"date":"2023-09-07T17:53:21","date_gmt":"2023-09-07T22:53:21","guid":{"rendered":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/2023\/09\/07\/assessing-the-accuracy-of-estimates-for-u-s-adults\/"},"modified":"2024-04-14T04:02:42","modified_gmt":"2024-04-14T09:02:42","slug":"assessing-the-accuracy-of-estimates-for-u-s-adults","status":"publish","type":"post","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/assessing-the-accuracy-of-estimates-for-u-s-adults\/","title":{"rendered":"1. Assessing the accuracy of estimates for U.S. adults"},"content":{"rendered":"<div style=\"border-width:1px;padding-bottom:var(--wp--preset--spacing--20);padding-left:var(--wp--preset--spacing--30);padding-right:var(--wp--preset--spacing--30);padding-top:var(--wp--preset--spacing--20);--block-gap: inherit\" class=\"wp-block-prc-block-collapsible has-background has-ui-beige-very-light-background-color has-border-color has-ui-beige-dark-border-color\" id=\"terminology\" data-wp-interactive=\"{&quot;namespace&quot;:&quot;prc-block\\\/collapsible&quot;}\" data-wp-context=\"{&quot;collapsibleId&quot;:&quot;terminology&quot;,&quot;isOpen&quot;:false}\" data-wp-class--is-open=\"context.isOpen\" data-wp-init--scroll-into-view=\"callbacks.onInitScrollIntoView\"><div class=\"wp-block-prc-block-collapsible__title\" data-wp-on--click=\"actions.onClick\"><div>Terminology<\/div><button class=\"wp-block-prc-block-collapsible__icon\"><span data-wp-bind--hidden=\"context.isOpen\"><i class=\"icon icon-library__light icon__circle-plus\"><svg style=\"width: 1em; height: 1em;\"><use xlink:href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/plugins\/prc-icon-library\/build\/icons\/sprites\/light.svg#circle-plus\"><\/use><\/svg><\/i><\/span><span data-wp-bind--hidden=\"!context.isOpen\" hidden><i class=\"icon icon-library__light icon__circle-minus\"><svg style=\"width: 1em; height: 1em;\"><use xlink:href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/plugins\/prc-icon-library\/build\/icons\/sprites\/light.svg#circle-minus\"><\/use><\/svg><\/i><\/span><\/button><\/div><div class=\"wp-block-prc-block-collapsible__content\">\n\n<p class=\"wp-block-paragraph\"><strong>Probability-based panel:<\/strong> This refers to a national survey panel recruited using random sampling from a database that includes most people in the population. Today, most such panels in the United States recruit by drawing random samples of residential addresses or telephone numbers. Typically, data collection with these panels is done online. However, some of these panels interview a small fraction of respondents (usually about 5% or fewer) using an offline mode such as live telephone. These panels are \u201cprobability-based\u201d because the chance that each address or phone number is selected is known. However, the chance that each selected person will join the panel or take surveys after joining is not known.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Online opt-in sample:<\/strong> These samples are recruited using a variety of methods that are sometimes referred to as \u201cconvenience sampling.\u201d Respondents are not selected randomly from the population but are recruited from a variety of online sources such as ads on social media or search engines, websites offering rewards in exchange for survey participation, or self-enrollment in an opt-in panel. Some opt-in samples are sourced from a panel (or multiple panels), while others rely on intercept techniques where respondents are invited to take a one-off survey.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Benchmark:<\/strong> These are \u201cground-truth\u201d measures used to assess the <a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/short-reads\/2018\/12\/06\/how-asking-about-your-sleep-smoking-or-yoga-habits-can-help-pollsters-verify-their-findings\/\">accuracy of survey estimates<\/a>. For example, survey-based estimates for the share of voters who voted for each candidate in the 2020 presidential election are compared to a benchmark based on an official tally by the Federal Election Commission (FEC). Survey estimates are deemed more accurate the closer they are to the benchmark value. In this study, the benchmarks come from high-quality federal surveys such as the American Community Survey or administrative records like the FEC vote tally. Although these benchmarks come from \u201cgold-standard\u201d data sources, they are not entirely free from error. As such, they are not \u201ctrue\u201d population values but rather the best available approximations.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Error:<\/strong> This is the difference between an individual survey estimate and its corresponding benchmark value. Error can be either positive or negative depending on whether the survey estimate is higher or lower than the benchmark. For example, the FEC benchmark for the share of voters who voted for Donald Trump in the 2020 presidential election is 47%. If a survey estimated that share to be 42%, the error would be -5 percentage points because it came in 5 points below the benchmark. If the estimate were 49%, the error would be +2 points.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Absolute error:<\/strong> This is the absolute value of the error for a survey estimate. It describes the <em>size<\/em> of the error irrespective of its direction (positive or negative). For example, two estimates that have error of +5 and -5 percentage points, respectively, both have an absolute error of 5 points.<\/p>\n\n\n\n<figure class=\"wp-block-image alignright size-640-wide is-resized\"><a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/?attachment_id=2360\"><img decoding=\"async\" src=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/wp-content\/uploads\/sites\/10\/2023\/08\/pm_09.07.23_benchmarking-00-02.png?w=620\" alt=\"A table showing an example of measuring error for a benchmark variable.\" class=\"wp-image-2360\" style=\"width:310px\" width=\"310\" \/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Average absolute error (for a benchmark variable):<\/strong> This is a measure that summarizes the average size of errors across all the categories within a single benchmark variable. For example, the smoking status variable has four categories: 1) Smoke every day, 2) Smoke some days, 3) Do not now smoke and 4) Never smoked 100 cigarettes. A survey\u2019s estimates for each category will have different levels of error, both positive and negative. For a given survey, the absolute error for the smoking status variable is equal to the sum of the absolute errors for each category divided by the number of categories.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Average absolute error (for a sample):<\/strong> Average absolute error can also be used to summarize the overall level of error across many different benchmarks within a single sample. When used in this context, the average absolute error for a sample is equal to the sum of the average absolute errors for each benchmark variable divided by the total number of benchmark variables.<\/p>\n\n<\/div><\/div>\n\n\n<p class=\"wp-block-paragraph\">To gauge each sample\u2019s accuracy on general population estimates for all U.S. adults, we calculated the weighted percentage of adults belonging to 77 categories across 28 different variables and compared them to corresponding benchmarks derived from high-quality government data sources. The benchmarks covered a variety of topics, including voting, health, and respondents\u2019 work, family and living situations. (Refer to <a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/wp-content\/uploads\/sites\/10\/2023\/09\/pm_09.07.23_benchmarking-appendix.pdf\">the appendix<\/a> for the full list of benchmarks and their sources.)<\/p>\n\n\n\n<figure class=\"wp-block-image alignright size-640-wide is-resized\"><a rel=\"attachment wp-att-120920\" href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/assessing-the-accuracy-of-estimates-for-u-s-adults\/pm_09-07-23_benchmarking-01-01-png\/\"><img data-dominant-color=\"dde6e6\" data-has-transparency=\"false\" loading=\"lazy\" decoding=\"async\"  srcset=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-01.png?resize=480,446 480w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-01.png?resize=782,727 782w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-01.png?resize=840,781 840w\" sizes=\"(max-width: 480px) 480px, (max-width: 782px) 782px, 640px\" height=\"595\" src=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-01.png?w=640\" alt=\"A bar chart showing the average error on online opt-in samples was twice that of probability-based panels.\" class=\"wp-image-120920 not-transparent\" style=\"--dominant-color: #dde6e6; width:420px\" width=\"640\" \/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Because many of the benchmark variables included more than one category, we calculated each variable\u2019s average absolute error \u2013 that is, the average of the absolute differences between the survey estimate and a corresponding benchmark value for each category \u2013 to compare the relative accuracy of variables that have different numbers of categories. To facilitate more general comparisons between samples overall, we also calculated the average absolute error for each sample as the mean of the average absolute errors across all 28 benchmarks.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In any study of this kind, it is important to note that the performance of any given sample depends on numerous factors such as the <a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2018\/01\/26\/for-weighting-online-opt-in-samples-what-matters-most\/\">variables used in weighting<\/a> or the <a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2016\/05\/02\/evaluating-online-nonprobability-surveys\/\">specific topics<\/a> included in the benchmarks. It is possible that the relative accuracy of each sample might differ if we had used a different weighting scheme or chosen a different set of benchmarks for comparison. Furthermore, not even \u201cgold-standard\u201d government statistics are entirely free from error. Consequently, the measures of error discussed in this report should be considered approximate.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Across all 28 benchmarks combined, the probability-based panels had a mean average absolute error of 2.6 percentage points for estimates among all U.S. adults. The error for individual probability-based panels ranged from a low of 2.3 points for probability panel 1 to a high of 3.0 points for probability panel 2. The average absolute error for the opt-in samples combined was about twice as large at 5.8 points. Of these, opt-in sample 3 had the lowest average error at 5.0 points. Opt-in samples 1 and 2 exhibited higher error on average with 6.4 and 6.1 points, respectively. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For ease of explanation, individual benchmark variables whose average absolute error was less than 2 percentage points were classified as having \u201clow\u201d error. Variables with more than 5 points of average absolute error were defined as having \u201chigh\u201d error, and the remainder were coded as having \u201cmedium\u201d error. These particular groupings were chosen because they each contain about one-third of all benchmarks from all six samples. It is important to note that these designations of low, medium or high error are relative to the specific benchmarks and samples included in this study.<\/p>\n\n\n\n<figure class=\"wp-block-image size-640-wide\"><a rel=\"attachment wp-att-120922\" href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/assessing-the-accuracy-of-estimates-for-u-s-adults\/pm_09-07-23_benchmarking-01-02-png\/\"><img data-dominant-color=\"f5f5f4\" data-has-transparency=\"false\" style=\"--dominant-color: #f5f5f4;\" loading=\"lazy\" decoding=\"async\"  srcset=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-02.png?resize=480,413 480w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-02.png?resize=782,672 782w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-02.png?resize=960,825 960w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-02.png?resize=1200,1031 1200w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-02.png?resize=1280,1100 1280w\" sizes=\"(max-width: 480px) 480px, (max-width: 782px) 782px, 640px\" height=\"550\" width=\"640\" src=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-02.png?w=640\" alt=\"A chart showing that opt-in samples had many more benchmarks with large errors than probability-based panels.\" class=\"wp-image-120922 not-transparent\" \/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Collectively, about half of all benchmarks on the probability-based panels fell into the low error category (44 out of 84) while 11% were classified as high error (9 out of 84). The only variable with consistently high error on the probability-based panels was voter turnout in the 2020 presidential election, for which all three samples overestimated the benchmark by +8 or +9 percentage points. This is consistent with our <a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2016\/05\/02\/evaluating-online-nonprobability-surveys\/\">2016 study<\/a>, which found civically engaged adults to be overrepresented on the American Trends Panel, and suggests that the problem afflicts ABS-recruited panels more generally.<\/p>\n\n\n\n<figure class=\"wp-block-image alignright size-640-wide is-resized\"><a rel=\"attachment wp-att-120926\" href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/assessing-the-accuracy-of-estimates-for-u-s-adults\/pm_09-07-23_benchmarking-01-03-png\/\"><img data-dominant-color=\"efefef\" data-has-transparency=\"false\" loading=\"lazy\" decoding=\"async\"  srcset=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-03.png?resize=480,439 480w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-03.png?resize=782,715 782w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-03.png?resize=840,768 840w\" sizes=\"(max-width: 480px) 480px, (max-width: 782px) 782px, 640px\" height=\"585\" src=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-03.png?w=640\" alt=\"A table showing the level of error on benchmark variables.\" class=\"wp-image-120926 not-transparent\" style=\"--dominant-color: #efefef; width:420px\" width=\"640\" \/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">This overall pattern was reversed for the opt-in samples, on which 51% of benchmarks fell in the high error category, compared with 20% in the low error category. There were 10 variables with consistently high error on the opt-in samples. Four involved the receipt of certain government benefits in the prior year. On average, the opt-in samples overestimated the shares of adults receiving food stamps (+16 points), Social Security (+15 points), unemployment compensation (+10 points) and workers\u2019 compensation (+9 points).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Another three variables with consistently high error related to employment, with the opt-in samples underestimating the share of all adults who worked for pay in the prior week by -12 points on average and the share of adults who worked at a job or business at any time in the prior year by -11 points on average. The opt-in samples also overstated the share of adults who were employed but had been unable to work in the past month because of COVID-19 by an average of +7 points.<\/p>\n\n\n\n<figure class=\"wp-block-image size-640-wide is-resized\"><a rel=\"attachment wp-att-120929\" href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/assessing-the-accuracy-of-estimates-for-u-s-adults\/pm_09-07-23_benchmarking-01-04-png\/\"><img data-dominant-color=\"f7f8f7\" data-has-transparency=\"false\" loading=\"lazy\" decoding=\"async\"  srcset=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-04.png?resize=480,509 480w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-04.png?resize=782,830 782w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-04.png?resize=960,1019 960w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-04.png?resize=1200,1273 1200w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-04.png?resize=1280,1358 1280w\" sizes=\"(max-width: 480px) 480px, (max-width: 782px) 782px, 640px\" height=\"679\" src=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-01-04.png?w=640\" alt=\"A dot plot that shows samples of the same type generally had similar levels of error on individual benchmarks.\" class=\"wp-image-120929 not-transparent\" style=\"--dominant-color: #f7f8f7; width:640px\" width=\"640\" \/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Two health-related benchmarks also saw consistently high error on the opt-in samples. Specifically, all of the opt-in samples exceeded the national benchmark for the share of adults with a food allergy (9%) by +6 points on average. They also understated the share of adults who have never tried vaping by -12 points and overstated the share who currently vape some days or every day by +8 and +5 points on average, respectively. Finally, all three opt-in samples overstated the share of adults who live in carless or single-car households by an average of +6 and +15 points, respectively.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">At the other end of the spectrum, there were seven variables with consistently low error on all three probability-based panels. These included parental status, number of children in the household, marital status, housing tenure, smoking status, English language proficiency and candidate vote share in the 2020 presidential election. Two of these, English proficiency and 2020 vote share, also had consistently low error on all three opt-in samples. Citizenship status also had consistently low error on the opt-in samples.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>To gauge each sample\u2019s accuracy on general population estimates for all U.S. adults, we calculated the weighted percentage of adults belonging to 77 categories across 28 different variables and compared them to corresponding benchmarks derived from high-quality government data sources. The benchmarks covered a variety of topics, including voting, health, and respondents\u2019 work, family and [&hellip;]<\/p>\n","protected":false},"author":658,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"sub_headline":"","sub_title":"","_prc_public_revisions":[],"_ppp_expiration_hours":0,"_ppp_enabled":false,"ai_generated_summary":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"apple_news_api_id":"","apple_news_api_revision":"","apple_news_api_created_at":"","apple_news_api_modified_at":"","apple_news_api_share_url":"","apple_news_api_pending":"","apple_news_is_preview":false,"apple_news_is_hidden":false,"relatedPosts":[],"reportMaterials":[],"multiSectionReport":[],"package_parts__enabled":false,"package_parts":[],"datacite_doi":"","datacite_doi_citation":"","_prc_seo_qr_attachment_id":0,"spoken_article_player_enabled":true,"displayBylines":true,"footnotes":"","prc_watchers":[],"_prc_fork_parent":0,"_prc_fork_status":"","_prc_active_fork":0,"jetpack_post_was_ever_published":false},"categories":[],"tags":[],"bylines":[],"collection":[],"datasets":[],"level_of_effort":[],"primary_audience":[],"information_type":[],"_post_visibility":[],"formats":[458],"_fund_pool":[],"languages":[],"regions-countries":[],"research-teams":[528],"workflow-status":[],"class_list":["post-101127","post","type-post","status-publish","format-standard","hentry","formats-report","research-teams-methods"],"label":false,"post_parent":101111,"word_count":1434,"canonical_url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/assessing-the-accuracy-of-estimates-for-u-s-adults\/","art_direction":{"A1":{"id":120894,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-topic.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-topic.png?w=564&h=317&crop=1","width":564,"height":317,"caption":"","chartArt":false},"A2":{"id":120898,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-crop.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-crop.png?w=268&h=151&crop=1","width":268,"height":151,"caption":"","chartArt":true},"A3":{"id":120898,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-crop.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-crop.png?w=194&h=110&crop=1","width":194,"height":110,"caption":"","chartArt":true},"A4":{"id":120898,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-crop.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-crop.png?w=268&h=151&crop=1","width":268,"height":151,"caption":"","chartArt":true},"XL":{"id":120894,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-topic.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-topic.png?w=720&h=405&crop=1","width":720,"height":405,"caption":"","chartArt":false},"social":{"id":120901,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-featured.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2023\/09\/pm_09.07.23_benchmarking-featured.png?w=1200&h=628&crop=1","width":1200,"height":628,"caption":"","chartArt":false}},"_embeds":[],"watchers":[],"table_of_contents":[{"id":101111,"title":"Comparing Two Types of Online Survey Samples","slug":"comparing-two-types-of-online-survey-samples","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/comparing-two-types-of-online-survey-samples\/","is_active":false},{"id":101127,"title":"1. Assessing the accuracy of estimates for U.S. adults","slug":"assessing-the-accuracy-of-estimates-for-u-s-adults","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/assessing-the-accuracy-of-estimates-for-u-s-adults\/","is_active":true},{"id":101128,"title":"2. Assessing the accuracy of estimates among demographic subgroups","slug":"assessing-the-accuracy-of-estimates-among-demographic-subgroups","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/assessing-the-accuracy-of-estimates-among-demographic-subgroups\/","is_active":false},{"id":101126,"title":"Acknowledgments","slug":"benchmarking-acknowledgments","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/benchmarking-acknowledgments\/","is_active":false},{"id":101122,"title":"Methodology","slug":"benchmarking-methodology","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/benchmarking-methodology\/","is_active":false}],"report_materials":[{"key":"_3lajd9bm9","type":"report","attachmentId":2446,"url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/wp-content\/uploads\/sites\/10\/2023\/09\/pm_09.07.23_benchmarking-report.pdf","label":"","icon":""},{"key":"_38uhyofal","type":"questionnaire","attachmentId":2406,"url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/wp-content\/uploads\/sites\/10\/2023\/09\/pm_09.07.23_benchmarking_questionnaire.pdf","label":"","icon":""},{"key":"_37uovu6cc","type":"link","attachmentId":0,"url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/wp-content\/uploads\/sites\/10\/2023\/09\/pm_09.07.23_benchmarking-appendix.pdf","label":"Appendix: Benchmarks and measures of error by sample","icon":"supplemental"},{"type":"dataset","id":1992,"label":"2021 Benchmarking Study","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/dataset\/2021-benchmarking-study\/"}],"report_pagination":{"current_post":{"id":101127,"title":"1. Assessing the accuracy of estimates for U.S. adults","slug":"assessing-the-accuracy-of-estimates-for-u-s-adults","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/assessing-the-accuracy-of-estimates-for-u-s-adults\/","is_active":true,"page_num":2},"next_post":{"id":101128,"title":"2. Assessing the accuracy of estimates among demographic subgroups","slug":"assessing-the-accuracy-of-estimates-among-demographic-subgroups","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/assessing-the-accuracy-of-estimates-among-demographic-subgroups\/","is_active":false,"page_num":3},"previous_post":{"id":101111,"title":"Comparing Two Types of Online Survey Samples","slug":"comparing-two-types-of-online-survey-samples","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/comparing-two-types-of-online-survey-samples\/","is_active":false,"page_num":1},"pagination_items":[{"id":101111,"title":"Comparing Two Types of Online Survey Samples","slug":"comparing-two-types-of-online-survey-samples","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/comparing-two-types-of-online-survey-samples\/","is_active":false,"page_num":1},{"id":101127,"title":"1. Assessing the accuracy of estimates for U.S. adults","slug":"assessing-the-accuracy-of-estimates-for-u-s-adults","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/assessing-the-accuracy-of-estimates-for-u-s-adults\/","is_active":true,"page_num":2},{"id":101128,"title":"2. Assessing the accuracy of estimates among demographic subgroups","slug":"assessing-the-accuracy-of-estimates-among-demographic-subgroups","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/assessing-the-accuracy-of-estimates-among-demographic-subgroups\/","is_active":false,"page_num":3},{"id":101126,"title":"Acknowledgments","slug":"benchmarking-acknowledgments","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/benchmarking-acknowledgments\/","is_active":false,"page_num":4},{"id":101122,"title":"Methodology","slug":"benchmarking-methodology","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2023\/09\/07\/benchmarking-methodology\/","is_active":false,"page_num":5}]},"parent_info":{"parent_title":"Comparing Two Types of Online Survey Samples","parent_id":101111},"materialsOrdered":[],"chaptersOrdered":[],"partsOrdered":[],"partsEnabled":false,"datacite_doi":"","prc_seo_data":{"title":"Online survey samples: How accurate are estimates for US adults? %page%","description":"Probability-based panel: This refers to a national survey panel recruited using random sampling from a database that includes most people in the population. Today, most such panels in the United&hellip;","og_title":"1. Assessing the accuracy of estimates for U.S. adults","og_description":"","schema_type":"Article","noindex":false,"canonical_url":"","primary_terms":{"regions-countries":10814863,"research-teams":10813973},"custom_schema":[],"og_image":120901,"indexnow_submitted_at":null,"gsc_index_status":null},"prepublish_checks":{"prc-image-alt-text":{"status":"complete","message":"All images have alt text.","data":null},"prc-about-this-research":{"status":"incomplete","message":"Add an \"About this research\" details block.","data":null},"prc-paragraph-count":{"status":"complete","message":"Found 17 paragraphs.","data":{"count":17}},"prc-internal-link":{"status":"complete","message":"Found 10 internal links.","data":{"count":10}}},"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"relatedPostsOrdered":[],"bylinesOrdered":[],"acknowledgementsOrdered":[],"_links":{"self":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/posts\/101127","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/users\/658"}],"replies":[{"embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/comments?post=101127"}],"version-history":[{"count":2,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/posts\/101127\/revisions"}],"predecessor-version":[{"id":130839,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/posts\/101127\/revisions\/130839"}],"wp:attachment":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/media?parent=101127"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/categories?post=101127"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/tags?post=101127"},{"taxonomy":"bylines","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/bylines?post=101127"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/collection?post=101127"},{"taxonomy":"datasets","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/datasets?post=101127"},{"taxonomy":"level_of_effort","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/level_of_effort?post=101127"},{"taxonomy":"primary_audience","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/primary_audience?post=101127"},{"taxonomy":"information_type","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/information_type?post=101127"},{"taxonomy":"_post_visibility","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_post_visibility?post=101127"},{"taxonomy":"formats","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/formats?post=101127"},{"taxonomy":"_fund_pool","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_fund_pool?post=101127"},{"taxonomy":"languages","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/languages?post=101127"},{"taxonomy":"regions-countries","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/regions-countries?post=101127"},{"taxonomy":"research-teams","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/research-teams?post=101127"},{"taxonomy":"workflow-status","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/workflow-status?post=101127"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}