{"id":101180,"date":"2021-06-09T17:50:27","date_gmt":"2021-06-09T22:50:27","guid":{"rendered":""},"modified":"2025-04-23T23:56:50","modified_gmt":"2025-04-24T03:56:50","slug":"measuring-the-risks-of-panel-conditioning-in-survey-research","status":"publish","type":"post","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/measuring-the-risks-of-panel-conditioning-in-survey-research\/","title":{"rendered":"Measuring the Risks of Panel Conditioning in Survey Research"},"content":{"rendered":"<h2 class=\"has-text-align-left wp-block-prc-block-subtitle\" aria-level=\"2\">Conditioning does not contribute significant error to panel estimates<\/h2>\n\n\n<figure class=\"wp-block-image aligncenter size-640-wide is-resized\"><a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/measuring-the-risks-of-panel-conditioning-in-survey-research\/pm_21-06-01_panelconditioning_featured-jpg\/\" rel=\"attachment wp-att-120967\"><img decoding=\"async\" src=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_21.06.01_panelConditioning_featured.jpg\" alt=\"\" style=\"width:640px\" \/><\/a><figcaption class=\"wp-element-caption\">(Alexander Spatari\/Getty Images)<\/figcaption><\/figure>\n\n\n<div style=\"border-width:1px;padding-bottom:var(--wp--preset--spacing--20);padding-left:var(--wp--preset--spacing--30);padding-right:var(--wp--preset--spacing--30);padding-top:var(--wp--preset--spacing--20);--block-gap: inherit\" class=\"is-style-alternate wp-block-prc-block-collapsible has-background has-ui-beige-very-light-background-color has-border-color has-ui-beige-dark-border-color\" id=\"how-we-did-this\" data-wp-interactive=\"{&quot;namespace&quot;:&quot;prc-block\\\/collapsible&quot;}\" data-wp-context=\"{&quot;collapsibleId&quot;:&quot;how-we-did-this&quot;,&quot;isOpen&quot;:false}\" data-wp-class--is-open=\"context.isOpen\" data-wp-init--scroll-into-view=\"callbacks.onInitScrollIntoView\"><div class=\"wp-block-prc-block-collapsible__title\" data-wp-on--click=\"actions.onClick\"><div>How we did this<\/div><button class=\"wp-block-prc-block-collapsible__icon\"><span data-wp-bind--hidden=\"context.isOpen\"><i class=\"icon icon-library__light icon__circle-plus\"><svg style=\"width: 1em; height: 1em;\"><use xlink:href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/plugins\/prc-icon-library\/build\/icons\/sprites\/light.svg#circle-plus\"><\/use><\/svg><\/i><\/span><span data-wp-bind--hidden=\"!context.isOpen\" hidden><i class=\"icon icon-library__light icon__circle-minus\"><svg style=\"width: 1em; height: 1em;\"><use xlink:href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/plugins\/prc-icon-library\/build\/icons\/sprites\/light.svg#circle-minus\"><\/use><\/svg><\/i><\/span><\/button><\/div><div class=\"wp-block-prc-block-collapsible__content\">\n\n<p class=\"wp-block-paragraph\">Pew Research Center conducted a series of analyses exploring data quality in its U.S. surveys, specifically those conducted on the Center\u2019s online survey platform, the&nbsp;<a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2019\/02\/27\/growing-and-improving-pew-research-centers-american-trends-panel\/\">American Trends Panel (ATP)<\/a>. The goal was to determine whether participation in the panel changed respondents\u2019 true or reported behavior over time (either immediately or over a longer period), a phenomenon referred to as panel conditioning. Because panel conditioning can be difficult to isolate from other differences (e.g., true change over time, reliability), analyses to detect conditioning were conducted in three different ways. First, an experiment was conducted in 2019-2020 in which some panelists were asked to complete several more surveys than others to determine if repeated exposure introduced conditioning. Second, estimates from newly recruited cohorts were compared with estimates from existing panelists at different points in time to determine if the existing panelists had different behaviors due to conditioning. Third, researchers appended administrative data on voting behavior pre- and post-empanelment to determine whether individuals changed their voting behavior over time, a sign of conditioning. All analyses were conducted using specially designed weights to control for panel attrition over time, cohort-level differences and variations in sampling procedures.<\/p>\n\n<\/div><\/div>\n\n\n<p class=\"wp-block-paragraph\">As public opinion polling increasingly moves toward the use of online panels, one threat that pollsters face is the possibility that their data could be damaged by interviewing the same set of people over and over again. The concern is that repetitive interviewing may introduce&nbsp;<a href=\"https:\/\/dism.duke.edu\/files\/2020\/05\/bach.pdf\">panel conditioning<\/a>,&nbsp;a state in which panelists change their beliefs or behavior just by being exposed to and answering a variety of questions over time.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Panel conditioning can have a harmful effect on data quality if respondents change their original attitudes and behaviors because of the survey stimulus. For example, an individual respondent may not have heard of House Speaker Nancy Pelosi, her role, party affiliation or voting record. They would not be alone, as&nbsp;<a href=\"https:\/\/www.washingtonpost.com\/news\/the-fix\/wp\/2015\/10\/08\/how-many-americans-even-know-who-the-speaker-of-the-house-is\/\">41%<\/a>&nbsp;of U.S. adults could not name the speaker of the House, and more than 10% reported never having heard her name. Panel conditioning would occur if the mere act of asking panelists about political leaders (e.g., \u201cDo you approve or disapprove of the way Nancy Pelosi is handling her job as speaker of the House?\u201d) causes panelists to&nbsp;<a href=\"https:\/\/ojs.ub.uni-konstanz.de\/srm\/article\/view\/874\">seek out more information<\/a>&nbsp;about them, form new opinions, or become more politically engaged than they would otherwise have been had they not joined the panel.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Alternatively, panel conditioning can have a&nbsp;<a href=\"https:\/\/www.understandingsociety.ac.uk\/blog\/2019\/11\/13\/what-is-panel-conditioning-and-what-can-we-do-about-it\">beneficial<\/a>&nbsp;<a href=\"https:\/\/onlinelibrary.wiley.com\/doi\/full\/10.1111\/obes.12296\">effect<\/a>&nbsp;on data quality when participation elicits more accurate reporting over time. Panelists may become more reflective of their own behaviors and attitudes and be able to&nbsp;<a href=\"https:\/\/www.iser.essex.ac.uk\/research\/publications\/493340\">more accurately<\/a>&nbsp;report them. As the researcher and respondent build rapport, panelists may also become&nbsp;<a href=\"https:\/\/link.springer.com\/article\/10.1007\/s11135-011-9640-9?con=17690&amp;error=cookies_not_supported&amp;code=4d36809c-4678-4353-97a3-1e9662e5337e\">more willing to report<\/a>&nbsp;their true behaviors and attitudes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A new study explored potential risks and benefits from repeated interviewing of participants in Pew Research Center\u2019s American Trends Panel (ATP). Among the key findings:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>There was no evidence that conditioning has biased ATP estimates for news consumption, discussing politics, political partisanship or voting, though empanelment led to a slight uptick in voter registration.&nbsp;<\/strong>Multiple analyses conducted on variables deemed most susceptible to panel conditioning by Center staff failed to identify any change in&nbsp;respondents\u2019 media consumption and dialogue behaviors, party identification, or voting record.&nbsp;However, empanelment did have a small effect on voter registration. Analysis of differences between ATP cohorts suggested that panelists were slightly more likely to register to vote soon after joining the ATP.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/measuring-the-risks-of-panel-conditioning-in-survey-research\/pm_06-09-21_panel-conditioning-00-0-png\/\" rel=\"attachment wp-att-120924\"><img data-dominant-color=\"f1f1f0\" data-has-transparency=\"false\" loading=\"lazy\" decoding=\"async\"  srcset=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-0.png?resize=480,272 480w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-0.png?resize=782,443 782w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-0.png?resize=960,544 960w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-0.png?resize=1200,680 1200w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-0.png?resize=1278,724 1278w\" sizes=\"(max-width: 480px) 480px, (max-width: 782px) 782px, 640px\" height=\"580\" width=\"1024\" src=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-0.png?w=1024\" alt=\"Multiple analyses demonstrate minimal effects of panel conditioning on behavioral and attitudinal change\" class=\"wp-image-120924 not-transparent\" style=\"--dominant-color: #f1f1f0; width:640px\" title=\"Multiple analyses demonstrate minimal effects of panel conditioning on behavioral and attitudinal change\" \/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>There is some evidence that panelists may change their reporting post-recruitment, likely improving data quality.&nbsp;<\/strong>On average, empaneled members reported less media consumption and dialogue in three out of six analyses.&nbsp;Evidence was mixed on whether these changes occurred gradually or soon after empanelment. While lower reports of consumption and dialogue may indicate higher misreporting when some response choices prompt several follow-up questions, the ATP rarely includes this type of design. Lower reports are likely, but not&nbsp;conclusively, an indicator of more accurate reporting and higher data quality over time.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Conditioning effects are difficult to isolate from true change over time, differences in panel cohorts, panel attrition and changes in measurements due to methodological enhancements. To ensure that all findings were replicable and robust, researchers began by selecting the six variables hypothesized to be most susceptible to conditioning \u2013 three media consumption and dialogue variables and three political affiliation and activism variables. Researchers then conducted three different types of analyses on the chosen variables. These included:&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">1. A randomized experiment executed around the 2020 election in which some panelists were asked questions susceptible to conditioning less often and some more often over time.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">2. A comparison of newer cohorts (who have been asked these items less frequently as an effect of joining the panel more recently) and older cohorts across a five-year time frame.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">3. A comparison of voter turnout records pre- and post-empanelment.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For this analysis, increasing trends over time among those empaneled (all else being equal) were interpreted as signs of a harmful and less accurate conditioning effect. This would include becoming more likely to follow government affairs, follow the news, discuss current events and politics, register to vote, identify as a member of a primary political party or vote as time since joining the panel increases. This assumption is founded in the&nbsp;<a href=\"https:\/\/www.iser.essex.ac.uk\/research\/publications\/493340\">theory<\/a>&nbsp;that respondents\u2019 awareness is raised and their interest is piqued when asked about a subject, increasing the frequency of these behaviors.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Decreasing frequencies may be the result of harmful or helpful effects of conditioning. Respondents may try to&nbsp;<a href=\"https:\/\/academic.oup.com\/poq\/article-abstract\/78\/3\/721\/1843708?redirectedFrom=fulltext\">\u201cgame\u201d<\/a> the survey, answering in a manner that they believe will yield fewer follow-up questions and make the survey shorter. This can harm data quality. However, \u201cgaming\u201d is unlikely on the ATP since the length of the survey is rarely correlated with the response to a given question. Instead, decreasing frequencies are likely an indicator of improved data quality. These interpretations are consistent with the theory of&nbsp;<a href=\"https:\/\/onlinelibrary.wiley.com\/doi\/abs\/10.1002\/9781444316568.wiem02057\">social desirability bias<\/a>, the idea that respondents wish to be seen favorably by researchers so they generally overreport good behaviors (e.g., voting). Only after building rapport and trust with the researcher are they more likely to report honestly and accurately.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Given the consistency of the results across all three sets of analyses and the fact that the variables used in the analyses were considered the most likely offenders, current methods appear to be sufficient to stave off large-scale harmful effects of panel conditioning and may improve data quality over time.<\/p>\n\n\n\n<h4 class=\"wp-block-heading wp-block-prc-block-chapter\" id=\"theres-potential-for-panel-conditioning-on-the-atp\">There\u2019s potential for panel conditioning on the ATP<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">Pew Research Center collects over 15 million data points every year from over 13,000 American Trends Panel (ATP) panelists. The&nbsp;<a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/u-s-survey-research\/american-trends-panel\/\">ATP<\/a>&nbsp;is comprised of individuals who have been recruited to take about two surveys per month. Between its inception in 2014 and October 2020, 74 surveys were conducted using the ATP. Over the years, the Center has recruited individuals to the panel a total of six times (about once per year) to replace individuals who have opted out and to grow the size of the panel. This means that the 2,188 individuals who were recruited in 2014 and were still active panelists as of October 2020 could have participated in all 74 surveys, whereas the newest (2020) cohort of 1,277 active panelists had only had the opportunity to answer three surveys since they were recruited much more recently.&nbsp;<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/measuring-the-risks-of-panel-conditioning-in-survey-research\/pm_06-09-21_panel-conditioning-00-1-png\/\" rel=\"attachment wp-att-120927\"><img data-dominant-color=\"f1f3f4\" data-has-transparency=\"false\" loading=\"lazy\" decoding=\"async\"  srcset=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-1.png?resize=480,363 480w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-1.png?resize=782,591 782w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-1.png?resize=960,726 960w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-1.png?resize=1200,908 1200w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-1.png?resize=1280,968 1280w\" sizes=\"(max-width: 480px) 480px, (max-width: 782px) 782px, 640px\" height=\"774\" width=\"1024\" src=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-1.png?w=1024\" alt=\"American Trends Panel panelists recruited in 2014 via an RDD frame invited to up to 74 surveys \" class=\"wp-image-120927 not-transparent\" style=\"--dominant-color: #f1f3f4; width:640px\" \/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">However, not every survey is created equal. All panelists are not invited to all surveys, as some surveys only require a subsample. Some panelists, even if invited, opt not to participate in a given survey. While this means that it\u2019s unlikely that any panelist actually participated in all 74 surveys, individuals who have been empaneled the longest have still been exposed to numerous surveys and questions. Panelists recruited in 2014 have taken an average of 58 surveys as of October 2020.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Some questions and topics are more susceptible to panel conditioning than others. For example, a survey about religious beliefs is unlikely to change someone\u2019s perception of God. These attitudes and beliefs are likely&nbsp;<a href=\"https:\/\/ojs.ub.uni-konstanz.de\/srm\/article\/view\/874\">well-entrenched<\/a>&nbsp;and less susceptible to change over time. However, these types of questions may also be prone to beneficial effects of panel conditioning. A respondent may not feel comfortable disclosing his\/her religious identity to a relatively unknown actor. Over time, trust is developed, and a more honest report of religious identify may follow.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Regardless of the exact number of surveys or questions prone to conditioning, three things are evident. First, if panelists\u2019 attitudes and behaviors are changed after just one survey on a given topic, then even the newest (2020) cohort will be altered after just a few surveys. This is referred to as \u201cimmediate\u201d harmful conditioning. Second, if conditioning effects are ongoing or only become more likely after being asked similar content repeatedly over time, then the oldest (2014) cohort would elicit the most biased responses while our newest (2020) cohort may still be representative of the population (all else being equal). This is referred to as \u201cgradual\u201d harmful conditioning. Third, the reverse may be true. Respondents may change their&nbsp;<em>reporting<\/em>&nbsp;immediately or gradually as they become more experienced panelists. Panel familiarity may improve data quality as respondents are more willing to report accurately. This is considered a helpful effect.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Despite the risk of conditioning (harmful or helpful), other researchers have found little, if any, cause for concern for multi-topic panels (i.e., the topic varies from survey to survey). For example,&nbsp;<a href=\"https:\/\/www.aapor.org\/AAPOR_Main\/media\/AnnualMeetingProceedings\/2019\/Disentangling-the-Effects-of-Panel-Conditioning.pdf\">research into the Ipsos Knowledge Panel<\/a> has&nbsp;<a href=\"https:\/\/www.researchgate.net\/publication\/238292479_Panel_Bias_from_Attrition_and_Conditioning_A_Case_Study_of_the_Knowledge_Networks_Panel\">failed to find<\/a>&nbsp;systematic conditioning effects on measures ranging from political activism to media consumption to internet usage, and minimal differences have been identified between more- and less-tenured AmeriSpeak panelists. This study is the first effort Pew Research Center has made to explore this topic in its panel. Our results broadly comport with these prior studies.<\/p>\n\n\n\n<h4 class=\"wp-block-heading wp-block-prc-block-chapter\" id=\"an-experiment-on-the-risks-of-panel-conditioning-during-a-presidential-election\">An experiment on the risks of panel conditioning during a presidential election<\/h4>\n\n\n\n<figure class=\"wp-block-image alignright size-large is-resized\"><a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/measuring-the-risks-of-panel-conditioning-in-survey-research\/pm_06-09-21_panel-conditioning-00-2-png\/\" rel=\"attachment wp-att-120930\"><img data-dominant-color=\"e5e9e8\" data-has-transparency=\"false\" loading=\"lazy\" decoding=\"async\"  srcset=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-2.png?resize=480,1054 480w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-2.png?resize=682,1498 682w\" sizes=\"(max-width: 480px) 480px, (max-width: 782px) 782px, 640px\" height=\"1024\" width=\"466\" src=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-2.png?w=466\" alt=\"No evidence of large harmful effects of panel conditioning in 2019-2020 experiment \" class=\"wp-image-120930 not-transparent\" style=\"--dominant-color: #e5e9e8; width:310px\" \/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">One way to test for conditioning bias is using a randomized experiment. Researchers assign a random subset of panelists to receive certain survey content, while other panel members do not receive that content. Done properly, this can isolate the effect of conditioning on panelists.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In November 2019, the Center administered a survey about news and media consumption to all ATP panelists. After the survey, respondents were divided into two random groups. One group of 1,000 individuals was not asked any more questions about news and media for 11 months while the second group of 10,855 panelists was invited to participate in six more surveys about news and media over the same time period. Then, both groups, along with a brand new cohort (2020), were invited to participate in an August 2020 survey about news and media, including some of the same questions from November 2019.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If panel conditioning occurs immediately after a participant\u2019s first survey (for better or worse), existing panelists in the group receiving similar questions infrequently should report different levels of engagement (i.e., media consumption and dialogue and voter registration) and major political party affiliation than the new recruits for whom the August 2020 survey was their first ATP survey. Higher engagement or affiliation would indicate harmful conditioning among existing panelists while lower engagement or affiliation would indicate changes in reporting habits, likely a beneficial effect of conditioning.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Additionally, if panel conditioning is a gradual process based on the number of times a type of question is asked, the 10,855 respondents who were repeatedly asked about their news and media consumption should be different than the 1,000 existing panelists who received these types of questions less frequently. If panel conditioning has harmful effects (i.e., changes behavior), panelists asked about these topics frequently are expected to report higher consumption and engagement levels. By contrast, if conditioning changes how respondents report their behaviors and attitudes (i.e., helpful effects), panelists asked about these topics frequently are expected to report lower (presumably more honest) levels.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Overall, no evidence of harmful forms of conditioning was observed for any of the five variables analyzed. While the new cohort exhibited slightly lower estimates (compared with the infrequently surveyed existing panelists) in two of the five comparisons, the differences were small (2 percentage points or less) and not statistically significant. Similarly, none of the comparisons showed statistically higher estimates among the frequently surveyed existing panelists when compared with those less frequently surveyed.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Respondents who were asked more frequently about their media consumption and dialogue behaviors reported&nbsp;<em>lower<\/em>values than those asked less frequently. For example, 57% of panelists receiving fewer surveys reported following the government and current affairs \u201cmost of the time,\u201d compared with just 50% among those who received the questions more frequently. A similar trend was observed when asked about following the news. For these variables, this change in reporting was gradual over time.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A more immediate shift was observed when respondents were asked how often they discuss politics. While no significant difference was observed between the two groups of existing panelists, there was a 6 percentage point difference between the new cohort and existing panelists assigned to the less frequent surveys group (24% vs. 18%, respectively).&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">While it is possible that the lower values are due to behavioral change, there is no strong theoretical reason that empanelment should reduce consumption. It may also be a harmful reporting change; panelists may be fatigued, may wish to shorten the survey, or be rushing through the survey. However, the most likely explanation is one of improved reporting. Respondents are more honest and willing to report less desirable behaviors after building trust and rapport with the Center.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Because many factors affect the likelihood of conditioning, some individuals may be more susceptible to conditioning than others. To this end, comparisons were also made by age, education and gender.[1. Other subgroups could not be analyzed due to limited sample sizes.] The subgroup analyses were generally consistent with the overall findings. However, not all subgroups behaved similarly. For example, 18- to 49-year-olds were statistically unaffected by empanelment when it came to reporting the frequency with which they follow the government and current affairs, whereas older individuals (both 50- to 64-year-olds and those 65 and older) exhibited a significant decline in frequency upon joining the panel. Given the different sample sizes among different subgroups, it was possible that a difference between groups appears significant for one comparison and does not reach significance for the other. Overall, similar patterns of change among the media and consumption variables were identified across groups.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Looking at the political affiliation and activism variables, there were slight harmful trends in the overall measures, but these differences failed to reach statistical significance. Similar trends were identified among most subgroups. Only women were subject to a significant, immediate shift in party identification. Specifically, the 2020 female recruits were less likely to identify with a major political party than existing panelists who were less frequently surveyed.&nbsp;<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/measuring-the-risks-of-panel-conditioning-in-survey-research\/pm_06-09-21_panel-conditioning-00-3-png\/\" rel=\"attachment wp-att-120936\"><img data-dominant-color=\"e4eae9\" data-has-transparency=\"false\" loading=\"lazy\" decoding=\"async\"  srcset=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-3.png?resize=480,558 480w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-3.png?resize=782,910 782w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-3.png?resize=960,1117 960w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-3.png?resize=1200,1396 1200w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-3.png?resize=1272,1480 1272w\" sizes=\"(max-width: 480px) 480px, (max-width: 782px) 782px, 640px\" height=\"1024\" width=\"880\" src=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-3.png?w=880\" alt=\"Panel conditioning effects were relatively consistent across subgroups \" class=\"wp-image-120936 not-transparent\" style=\"--dominant-color: #e4eae9; width:640px\" \/><\/a><\/figure>\n\n\n\n<h4 class=\"wp-block-heading wp-block-prc-block-chapter\" id=\"testing-for-conditioning-in-newer-vs-older-cohorts-over-time\">Testing for conditioning in newer vs. older cohorts over time&nbsp;<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">The Center often reports on how society has changed over time. In the absence of panel conditioning (and with proper weights and attrition adjustments), any observed change over time may be considered true change. However, if panel conditioning is present, it may bias the interpretation of the data. If the conditioning effect is in the same direction as the true change, i.e., people are becoming more engaged, then change would be magnified. If the direction of conditioning and true change are opposed, the true change would be underestimated.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To determine if panel conditioning was affecting analyses of change over time, data were collected following each ATP recruitment between 2014 and 2020. If conditioning were encouraging real change among panelists, older cohorts would be expected to report higher levels of media consumption and political activism than newer cohorts at a given point in time. If this change occurred soon after recruitment, the difference could be observed between the newest cohort and all other cohorts. In other words, the newest cohort at any given time point would look like a low outlier on a graph or figure. Additionally, if conditioning changes behavior gradually over time, newer cohorts would consistently consume less media and be less engaged than older cohorts. If graphing each cohort at a given point in time, gradual behavioral change would look like a set of stairs on a column chart or an ordered line on a dot plot.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Alternatively, if conditioning causes reporting changes, improving data quality, measurement would likely be reversed. Media consumption and political activism would appear highest for the newest cohort and lowest for the oldest cohort at a given point in time. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Measurements for each cohort were compared with each other at multiple time points to assess whether any type of conditioning was occurring. This yielded 20-35 comparisons per variable across six cohorts and 4-5 time points.[2. Frequency of following the news was only measured at&nbsp;four time points, limiting the number of comparisons to 32.The most recent time point for political party affiliation and voter registration was not used in this analysis due to being part of the weighting, limiting the number of comparisons to 20 each.] The questions used in this analysis were the same as the analysis of the experiment. They included measures of frequency with which panelists follow the government, follow the news, and discuss politics as well as party identification and whether they are registered to vote.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Most variables exhibited patterns consistent with the absence of panel conditioning. For example, there was no evidence of panel conditioning affecting behavioral change in the frequency with which individuals follow the government and public affairs. If conditioning yielded behavioral change quickly, the newest cohort would produce the lowest frequency of consumption. The newest cohort was never the lowest estimate in any of the five time points. In fact, it produced the highest estimate in three of the time points (October 2015, February 2018 and August 2020). If conditioning was more gradual at creating behavioral change, the estimates should align in order of cohort for each time point (e.g., the 2020 cohort would have the lowest estimate in August 2020 followed by the 2019 cohort, 2018 cohort, 2017 cohort, 2015 cohort and 2014 cohort). This also did not happen. For example, in August 2020, 51% of the 2018 and 2019 cohorts reported following the government most of the time followed by the 2015 and 2017 cohorts at 54%, the 2014 cohort at 55% and the 2020 cohort at 60%.&nbsp;<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/measuring-the-risks-of-panel-conditioning-in-survey-research\/pm_06-09-21_panel-conditioning-00-4-png\/\" rel=\"attachment wp-att-120939\"><img data-dominant-color=\"f1f2f1\" data-has-transparency=\"false\" loading=\"lazy\" decoding=\"async\"  srcset=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-4.png?resize=480,318 480w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-4.png?resize=782,518 782w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-4.png?resize=960,636 960w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-4.png?resize=1200,796 1200w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-4.png?resize=1276,846 1276w\" sizes=\"(max-width: 480px) 480px, (max-width: 782px) 782px, 640px\" height=\"679\" width=\"1024\" src=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-4.png?w=1024\" alt=\"Longer-tenured ATP panelists do not report following what is going on in government and public affairs more frequently than new panelists \" class=\"wp-image-120939 not-transparent\" style=\"--dominant-color: #f1f2f1; width:640px\" \/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Not only did the cohorts fail to fall into an order indicative of harmful conditioning, most of the differences among cohorts at a given time point also failed to reach significance. Twenty-nine of the 35 comparisons conducted to measure the frequency of following the government failed to reach significance. Of the remaining, all but two suggested the older cohort was following the government less often than the newer cohort. While this could indicate a change (and&nbsp;improvement) in reporting behavior over time, the lack of a consistent trend suggests it is more likely true differences among cohorts that could not be accounted for with weights.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/measuring-the-risks-of-panel-conditioning-in-survey-research\/pm_06-09-21_panel-conditioning-00-5-png\/\" rel=\"attachment wp-att-120944\"><img data-dominant-color=\"f3f4f3\" data-has-transparency=\"false\" loading=\"lazy\" decoding=\"async\"  srcset=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-5.png?resize=480,290 480w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-5.png?resize=782,472 782w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-5.png?resize=960,579 960w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-5.png?resize=1200,724 1200w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-5.png?resize=1280,772 1280w\" sizes=\"(max-width: 480px) 480px, (max-width: 782px) 782px, 640px\" height=\"618\" width=\"1024\" src=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-5.png?w=1024\" alt=\"Longer-tenured ATP panelists do not report following the news more frequently than new panelists \" class=\"wp-image-120944 not-transparent\" style=\"--dominant-color: #f3f4f3; width:640px\" \/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Other consumption and dialogue variables \u2013 frequency of following the news and discussing politics \u2013 were relatively similar to the measure of following the government. The newest cohort for a given time point never produced the lowest estimate of following the news all or most of the time. While the 2018 cohort did yield the lowest estimate of discussing politics nearly every day in May 2019, it was not statistically different from the 2017 cohort at the same point in time, and other new cohorts did not produce similar patterns. Also consistent with the lack of harmful effects was the lack of ordered estimates for any point in time and the failure of most comparisons (29 of 32 and 26 of 35 for following the news and discussing politics, respectively) to achieve statistical significance. Of those comparisons that did produce statistically significant differences, most were small and half were in the direction that suggests reporting improvements. While any change may yield slightly moderated or exaggerated results (depending on the direction of the change) in analyses of change over time, these changes are small. Luckily, for estimates that the Center publishes using the ATP, these effects are further mitigated when all of the cohorts are used.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/measuring-the-risks-of-panel-conditioning-in-survey-research\/pm_06-09-21_panel-conditioning-00-6-png\/\" rel=\"attachment wp-att-120949\"><img data-dominant-color=\"f3f3f3\" data-has-transparency=\"false\" loading=\"lazy\" decoding=\"async\"  srcset=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-6.png?resize=480,311 480w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-6.png?resize=782,506 782w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-6.png?resize=960,621 960w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-6.png?resize=1200,777 1200w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-6.png?resize=1276,826 1276w\" sizes=\"(max-width: 480px) 480px, (max-width: 782px) 782px, 640px\" height=\"663\" width=\"1024\" src=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-6.png?w=1024\" alt=\"Longer-tenured ATP panelists do not report discussing government and politics with others more frequently than new panelists\" class=\"wp-image-120949 not-transparent\" style=\"--dominant-color: #f3f3f3; width:640px\" \/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Party identification was the least susceptible to any type of conditioning (harmful or helpful) among the variables investigated. Of the 20 comparisons made among cohorts at four different points in time, none reached statistical significance.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/measuring-the-risks-of-panel-conditioning-in-survey-research\/pm_06-09-21_panel-conditioning-00-7-png\/\" rel=\"attachment wp-att-120950\"><img data-dominant-color=\"f4f4f4\" data-has-transparency=\"false\" loading=\"lazy\" decoding=\"async\"  srcset=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-7.png?resize=480,297 480w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-7.png?resize=782,485 782w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-7.png?resize=960,595 960w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-7.png?resize=1200,744 1200w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-7.png?resize=1278,792 1278w\" sizes=\"(max-width: 480px) 480px, (max-width: 782px) 782px, 640px\" height=\"635\" width=\"1024\" src=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-7.png?w=1024\" alt=\"Longer-tenured ATP panelists are not more likely than new panelists to self-identify with a major political party \" class=\"wp-image-120950 not-transparent\" style=\"--dominant-color: #f4f4f4; width:640px\" \/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">In this set of analyses, empanelment appeared to be changing individuals\u2019 behavior in one way&nbsp;&#8211;&nbsp;it encouraged individuals to register to vote. The newest cohort consistently reported lower rates of voter registration compared with other cohorts at the same point in time. For example, in October 2018, the registration rate was 76% among panelists recruited that year, compared with 79% among the panelists recruited in 2015 and 2017 and 82% among 2014 recruits.[3. Only the difference between the 2014 and 2018 cohorts reached statistical significance in October 2018.] The panel may act as a reminder or nudge for panelists to register, causing an uptick in registration soon after empanelment. These effects are small (4-7 percentage points), and not all comparisons among the newest and older cohorts reached statistical significance. Moreover, the conditioning effects are muted when cohorts are combined to create overall estimates.&nbsp;Despite these mitigating factors, the consistency in pattern across all points in time for all cohorts suggests some presence of conditioning&nbsp;changing behavior.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/measuring-the-risks-of-panel-conditioning-in-survey-research\/pm_06-09-21_panel-conditioning-00-8-png\/\" rel=\"attachment wp-att-120956\"><img data-dominant-color=\"f5f5f5\" data-has-transparency=\"false\" loading=\"lazy\" decoding=\"async\"  srcset=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-8.png?resize=480,288 480w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-8.png?resize=782,469 782w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-8.png?resize=960,576 960w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-8.png?resize=1200,720 1200w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-8.png?resize=1280,768 1280w\" sizes=\"(max-width: 480px) 480px, (max-width: 782px) 782px, 640px\" height=\"614\" width=\"1024\" src=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-8.png?w=1024\" alt=\"Participation in the ATP modestly increases voter registration rates  \" class=\"wp-image-120956 not-transparent\" style=\"--dominant-color: #f5f5f5; width:640px\" \/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">In addition to the small but significant uptick in voter registration due to conditioning, these analyses also shed light on another bias in the data: the ATP overrepresents eligible voters. While the Census Bureau estimates that&nbsp;<a href=\"https:\/\/www.census.gov\/data\/tables\/time-series\/demo\/voting-and-registration\/p20-583.html\">67%<\/a>[4. Researchers agree that this calculation artificially inflates the percent of the population that is unregistered because it includes those who were not asked or did not answer the registration question in the Voting and Registration Supplement as being unregistered.] of citizens 18 years of age or older were registered to vote in 2018, 76%-82% of eligible ATP panelists were registered at the same point in time.&nbsp;This overrepresentation is not the result of conditioning. Instead, it is in part due to differential nonresponse. Of the people invited to participate in the panel, individuals who are registered to vote are more likely to respond and join the panel than eligible individuals who are not registered. The Center addresses this bias by weighting the data. However, as evidenced in these analyses, the weights do not eliminate the entire bias, and additional improvements are warranted.<\/p>\n\n\n\n<h4 class=\"wp-block-heading wp-block-prc-block-chapter\" id=\"testing-for-conditioning-with-registered-voter-records\">Testing for conditioning with registered voter records<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">All surveys suffer from some error, so comparisons between survey estimates can conflate panel conditioning with other types of differences (e.g., measurement error, differences in recruitment methods). Researchers compared administrative data of voting before and after empanelment to further isolate panel conditioning from other differences. Specifically, Center researchers examined panelists\u2019 voter turnout histories between 2012-2018 from two&nbsp;<a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2018\/02\/15\/commercial-voter-files-and-the-study-of-u-s-politics\/\">commercial voter files<\/a>. This provided information about panelists both before and after they joined the ATP, allowing for analyses to determine whether the panel changed their behavior.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><a href=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/measuring-the-risks-of-panel-conditioning-in-survey-research\/pm_06-09-21_panel-conditioning-00-9-png\/\" rel=\"attachment wp-att-120960\"><img data-dominant-color=\"f5f6f5\" data-has-transparency=\"false\" loading=\"lazy\" decoding=\"async\"  srcset=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-9.png?resize=480,373 480w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-9.png?resize=782,607 782w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-9.png?resize=960,746 960w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-9.png?resize=1200,932 1200w, https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-9.png?resize=1280,994 1280w\" sizes=\"(max-width: 480px) 480px, (max-width: 782px) 782px, 640px\" height=\"795\" width=\"1024\" src=\"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning-00-9.png?w=1024\" alt=\"No presence of conditioning in administrative voting records\" class=\"wp-image-120960 not-transparent\" style=\"--dominant-color: #f5f6f5; width:640px\" \/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">If panel conditioning changes behaviors among ATP panelists, the voter turnout among cohorts that have already joined the panel should be statistically higher than the voter turnout among cohorts that had yet to join the panel.[5. Data are weighted to maximize the similarity across cohorts for the 2012 election. This weighting approach helps isolate the effect of conditioning but also results in biased point estimates. In other words, the numbers reported here are not actual turnout rates among the panelists. Attention should only focus on the differences between cohorts at a given point in time.] If change happens immediately upon joining the panel, then the observed difference should appear immediately after empanelment and hold across years. If change is ongoing, then the difference between yet-to-be-empaneled cohorts and existing cohorts should grow over time. Since administrative records only measure behavior, this analysis cannot be used to assess whether participation in the panel changes reporting over time.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">With the exception of the 2014 cohort, no differences in voter turnout were observed among cohorts in any of the four observed elections. The 2014 cohort was the only cohort to have been empaneled at the time of the 2014 election. A total of 68% of the 2014 cohort voted in the 2014 election, compared with 62%-64% of the other cohorts. However, after the 2014 election, there does not appear to be any compelling evidence of panel conditioning on the ATP when measuring voter turnout. The 2014 cohort turned out to vote at similar rates to yet-to-be-empaneled cohorts in both the 2016 and 2018 elections, and no other significant differences were observed among the other cohorts.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A new evaluation of the Center&#8217;s national American Trends Panel finds little evidence that panel estimates are affected by errors associated with panel conditioning, a phenomenon that occurs when survey participation changes respondents\u2019 true or reported behavior over time.<\/p>\n","protected":false},"author":367,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"sub_headline":"Conditioning does not contribute significant error to panel estimates","sub_title":"Conditioning does not contribute significant error to panel estimates","_prc_public_revisions":[],"_ppp_expiration_hours":0,"_ppp_enabled":false,"ai_generated_summary":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"relatedPosts":[],"reportMaterials":[{"key":"_99w74lc79","type":"report","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning.FULL_.pdf","attachmentId":120984}],"multiSectionReport":[{"key":"_5t2q9ift3","postId":101177},{"key":"_ddxjteyz2","postId":101171},{"key":"_h58wne3h0","postId":101176}],"package_parts__enabled":false,"package_parts":[],"_prc_fork_parent":0,"_prc_fork_status":"","_prc_active_fork":0,"datacite_doi":"","datacite_doi_citation":"","_prc_seo_qr_attachment_id":0,"spoken_article_player_enabled":true,"displayBylines":true,"footnotes":"","prc_watchers":[],"jetpack_post_was_ever_published":false},"categories":[355,36,450,357],"tags":[],"bylines":[719,671,779],"collection":[],"datasets":[],"level_of_effort":[],"primary_audience":[],"information_type":[],"_post_visibility":[],"formats":[458],"_fund_pool":[],"languages":[],"regions-countries":[],"research-teams":[528],"workflow-status":[],"class_list":["post-101180","post","type-post","status-publish","format-standard","hentry","category-american-trends-panel","category-methodological-research","category-online-surveys","category-survey-methods","bylines-arnold-lau","bylines-ashley-amaya","bylines-nick-hatley","formats-report","research-teams-methods"],"label":false,"post_parent":0,"word_count":4103,"canonical_url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/measuring-the-risks-of-panel-conditioning-in-survey-research\/","art_direction":{"A2":{"id":120967,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_21.06.01_panelConditioning_featured-jpg.webp","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_21.06.01_panelConditioning_featured-jpg.webp?w=268&h=151&crop=1","width":268,"height":151,"chartArt":false},"XL":{"id":120967,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_21.06.01_panelConditioning_featured-jpg.webp","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_21.06.01_panelConditioning_featured-jpg.webp?w=720&h=405&crop=1","width":720,"height":405,"chartArt":false},"A3":{"id":120967,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_21.06.01_panelConditioning_featured-jpg.webp","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_21.06.01_panelConditioning_featured-jpg.webp?w=194&h=110&crop=1","width":194,"height":110,"chartArt":false},"A4":{"id":120967,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_21.06.01_panelConditioning_featured-jpg.webp","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_21.06.01_panelConditioning_featured-jpg.webp?w=268&h=151&crop=1","width":268,"height":151,"chartArt":false},"A1":{"id":120967,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_21.06.01_panelConditioning_featured-jpg.webp","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_21.06.01_panelConditioning_featured-jpg.webp?w=564&h=317&crop=1","width":564,"height":317,"chartArt":false},"social":{"id":120967,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_21.06.01_panelConditioning_featured-jpg.webp","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_21.06.01_panelConditioning_featured-jpg.webp?w=1200&h=628&crop=1","width":1200,"height":628,"chartArt":false}},"_embeds":[],"watchers":[],"table_of_contents":[{"id":101180,"title":"Measuring the Risks of Panel Conditioning in Survey Research","slug":"measuring-the-risks-of-panel-conditioning-in-survey-research","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/measuring-the-risks-of-panel-conditioning-in-survey-research\/","is_active":true},{"id":101177,"title":"Acknowledgments","slug":"acknowledgments-4-8","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/acknowledgments-4-8\/","is_active":false},{"id":101171,"title":"Methodology","slug":"methodology-13-8","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/methodology-13-8\/","is_active":false},{"id":101176,"title":"Appendix A: Weighting for panel conditioning analyses","slug":"appendix-a-weighting-for-panel-conditioning-analyses","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/appendix-a-weighting-for-panel-conditioning-analyses\/","is_active":false}],"report_materials":[{"key":"_99w74lc79","type":"report","label":"","icon":"","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning.FULL_.pdf","attachmentId":120984}],"report_pagination":{"current_post":{"id":101180,"title":"Measuring the Risks of Panel Conditioning in Survey Research","slug":"measuring-the-risks-of-panel-conditioning-in-survey-research","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/measuring-the-risks-of-panel-conditioning-in-survey-research\/","is_active":true,"page_num":1},"next_post":{"id":101177,"title":"Acknowledgments","slug":"acknowledgments-4-8","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/acknowledgments-4-8\/","is_active":false,"page_num":2},"previous_post":null,"pagination_items":[{"id":101180,"title":"Measuring the Risks of Panel Conditioning in Survey Research","slug":"measuring-the-risks-of-panel-conditioning-in-survey-research","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/measuring-the-risks-of-panel-conditioning-in-survey-research\/","is_active":true,"page_num":1},{"id":101177,"title":"Acknowledgments","slug":"acknowledgments-4-8","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/acknowledgments-4-8\/","is_active":false,"page_num":2},{"id":101171,"title":"Methodology","slug":"methodology-13-8","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/methodology-13-8\/","is_active":false,"page_num":3},{"id":101176,"title":"Appendix A: Weighting for panel conditioning analyses","slug":"appendix-a-weighting-for-panel-conditioning-analyses","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/methods\/2021\/06\/09\/appendix-a-weighting-for-panel-conditioning-analyses\/","is_active":false,"page_num":4}]},"parent_info":{"parent_title":"Measuring the Risks of Panel Conditioning in Survey Research","parent_id":101180},"materialsOrdered":[{"key":"_99w74lc79","type":"report","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2021\/06\/PM_06.09.21_panel.conditioning.FULL_.pdf","label":"","attachmentId":120984,"icon":""}],"chaptersOrdered":[{"key":"_5t2q9ift3","postId":101177},{"key":"_ddxjteyz2","postId":101171},{"key":"_h58wne3h0","postId":101176}],"partsOrdered":[],"partsEnabled":false,"datacite_doi":"","prc_seo_data":{"title":"Measuring the Risks of Panel Conditioning in Survey Research","description":"A new evaluation of the Center's national American Trends Panel finds little evidence that panel estimates are affected by errors associated with panel conditioning, a phenomenon that occurs when survey participation changes respondents\u2019 true or reported behavior over time.","og_title":"Measuring the Risks of Panel Conditioning in Survey Research","og_description":"A new evaluation of the Center's national American Trends Panel finds little evidence that panel estimates are affected by errors associated with panel conditioning, a phenomenon that occurs when survey participation changes respondents\u2019 true or reported behavior over time.","schema_type":"Article","noindex":false,"canonical_url":"","primary_terms":[],"custom_schema":[],"og_image":120967,"indexnow_submitted_at":null,"gsc_index_status":null},"prepublish_checks":{"prc-image-alt-text":{"status":"incomplete","message":"1 image is missing alt text.","data":{"count":1}},"prc-about-this-research":{"status":"incomplete","message":"Add an \"About this research\" details block.","data":null},"prc-paragraph-count":{"status":"complete","message":"Found 42 paragraphs.","data":{"count":42}},"prc-internal-link":{"status":"complete","message":"Found 14 internal links.","data":{"count":14}}},"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"relatedPostsOrdered":[],"bylinesOrdered":[{"key":"_m15ks31lg","termId":671},{"key":"_nhu211c4p","termId":779},{"key":"_dqmascphm","termId":719}],"acknowledgementsOrdered":[],"_links":{"self":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/posts\/101180","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/users\/367"}],"replies":[{"embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/comments?post=101180"}],"version-history":[{"count":4,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/posts\/101180\/revisions"}],"predecessor-version":[{"id":224112,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/posts\/101180\/revisions\/224112"}],"wp:attachment":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/media?parent=101180"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/categories?post=101180"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/tags?post=101180"},{"taxonomy":"bylines","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/bylines?post=101180"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/collection?post=101180"},{"taxonomy":"datasets","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/datasets?post=101180"},{"taxonomy":"level_of_effort","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/level_of_effort?post=101180"},{"taxonomy":"primary_audience","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/primary_audience?post=101180"},{"taxonomy":"information_type","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/information_type?post=101180"},{"taxonomy":"_post_visibility","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_post_visibility?post=101180"},{"taxonomy":"formats","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/formats?post=101180"},{"taxonomy":"_fund_pool","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_fund_pool?post=101180"},{"taxonomy":"languages","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/languages?post=101180"},{"taxonomy":"regions-countries","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/regions-countries?post=101180"},{"taxonomy":"research-teams","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/research-teams?post=101180"},{"taxonomy":"workflow-status","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/workflow-status?post=101180"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}