{"id":13994,"date":"2014-07-02T07:00:20","date_gmt":"2014-07-02T12:00:20","guid":{"rendered":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/short-reads\/%year%\/%monthnum%\/%day%\/facebooks-experiment-is-just-the-latest-to-manipulate-you-in-the-name-of-research\/"},"modified":"2024-04-14T03:21:17","modified_gmt":"2024-04-14T08:21:17","slug":"facebooks-experiment-is-just-the-latest-to-manipulate-you-in-the-name-of-research","status":"publish","type":"short-read","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/short-reads\/2014\/07\/02\/facebooks-experiment-is-just-the-latest-to-manipulate-you-in-the-name-of-research\/","title":{"rendered":"Facebook\u2019s experiment causes a lot of fuss for little result"},"content":{"rendered":"

A study <\/a>in which Facebook manipulated the news feeds to more than 600,000 users sent social media users into a cyber-swoon this week and spilled over into the mainstream media: \u201cFacebook Tinkers With Users\u2019 Emotions,\u201d began the headline<\/a>\u00a0on the New York Times website.<\/p>\n\n

But the controversy over what these researchers did may be overshadowing other important discussions, specifically conversations about what they really found\u2014not much, actually\u2014and the right and wrong way to think about and report findings based on statistical analyses of big data. (We\u2019ll get to the ethics of their experiment in a moment.)<\/p>\n\n

Because they\u00a0are so large, studies based on supersized samples can produce results that are statistically significant but at the same time are substantively trivial. It\u2019s simple math: The larger the sample size, the smaller any differences need to be to be statistically significant\u2014that is, highly likely to be truly different from each other. (In this study, the differences examined were between those who saw more and those who saw fewer emotion-laden posts compared with a control group whose news feeds were\u00a0not manipulated.)<\/p>\n\n

And when you have an enormous random sample of 689,003, as these researchers did, even tiny differences pass standard tests of significance. (For perspective, a typical sample size in a nationally representative public opinion poll is 1,000.)<\/p>\n\n

That\u2019s why generations of statistics teachers caution their students that \u201cstatistically significant\u201d doesn\u2019t necessarily mean \u201creally, really<\/em> important.\u201d<\/p>\n\n

\"Facebook<\/a><\/figure>\n\n

Consider the findings of the Facebook study in which they varied how many positive and negative posts from friends test subjects were allowed to see. Posts were determined to be positive or negative if they contained a single positive or negative word. Then, the test subject\u2019s own use of positive and negative words in their status updates was monitored for a week. In all, test subjects posted a total 122 million words, four million of which were positive and 1.8 million negative.<\/p>\n\n

As reported by the authors, the number of negative words used in status updates increased, on average, by 0.04% when their friends’ positive posts in news feeds were reduced. That means only about four more negative words for every 10,000 written by these study participants. At the same time, the number of positive words decreased by only 0.1%, or about one less word for every 1,000 words written. (As a point of reference, this post is a little more than 1,000 words long.)<\/p>\n\n

Conversely, when negative posts were reduced, seven fewer negative words were used per 10,000, and the number of positive words rose by about six per 10,000.<\/p>\n\n

Based on these results, the authors concluded in their published study that their \u201cresults indicate that emotions expressed by others on Facebook influence our own emotions, constituting experimental evidence for massive-scale contagion via social networks.\u201d<\/p>\n\n

But do these tiny shifts, even if they are real, constitute evidence of an alarming \u201cmassive-scale contagion\u201d? Of course, importance is in the eye of the beholder. For some, these miniscule changes may be cause for alarm. But for others, they\u2019re probably just meh.<\/em><\/p>\n\n

One of the authors seems to have had second thoughts about the language they used to describe their work. In a Facebook post<\/a> written in response to the controversy, Adam D. I. Kramer acknowledged, \u201cMy coauthors and I are very sorry for the way the paper described the research.\u201d<\/p>\n\n

He also suggested that, even with their huge sample, they did not find a particularly large effect. The results, he wrote, were based on the “minimal amount to statistically detect it — the result was that people produced an average of one fewer emotional word, per thousand words, over the following week.\u201d<\/p>\n\n

Critics have raised other questions, notably The Atlantic<\/a> and\u00a0Wired <\/a>magazine, which\u00a0 questioned whether reading positive\u00a0posts directly caused the Facebook user to use more positive\u00a0words in their subsequent updates.<\/p>\n\n

But is what Facebook\u00a0did ethical? There is a good amount of discussion about whether Facebook was transparent enough with its users about this kind of experimentation. They did not directly inform those in the study that they were going to be used as human lab rats. In academic research, that\u2019s called not obtaining \u201cinformed consent\u201d and is almost always a huge no-no. (Facebook claims that everyone who joins Facebook agrees as part of its user agreement to be included in such studies.)<\/p>\n\n

The question is now about how, sitting on troves of new social media and other digital data to mine for the same kind of behavioral analysis, the new rules will need to be written.<\/p>\n\n

Experimental research is rife with examples of how study participants have been manipulated, tricked or outright lied to in the name of social science. And while many of these practices have been curbed or banned in academe, they continue to be used in commercial and other types of research.<\/p>\n\n

Consider the case of the \u201cVerifacitor,\u201d<\/a> the world\u2019s newest and best lie detector\u2014or at least that\u2019s what some participants were told in this study conducted by researchers at the University of Chicago\u2019s National Opinion Research Center in the mid-1990s.<\/p>\n\n

The test subjects were divided into two groups. Members of the control group were asked to sit at a desk where an interviewer asked questions about\u00a0exercise habits, smoking, drug use, sexual practices and excessive drinking.<\/p>\n\n

The other test subjects answered the same questions while being hooked up by electrodes to the Verifacitor, described by the operator as a new type of lie detector. (In fact, it was just a collection of old computer components the researchers had lying around.)<\/p>\n\n

To further enhance truth-telling, each participant was told before the formal interview began that the operator needed to calibrate the machine. So the participant was told to lie randomly in response to demographic questions about themselves that had been asked earlier on a screening questionnaire. (Questions like: Are you married? Did you finish high school? etc.).<\/p>\n\n

Of course the interviewer had been slipped the correct answers so she immediately identified a bogus response, much to the amazement of the test subject.<\/p>\n\n

Well you can guess what happened. Fully 44% of those in the Verifacitor group acknowledged they had ever used cocaine compared with 26% in the control group. Fully twice the proportion reported using amphetamines (39% vs. 19%), using other drugs (39% vs 19%) and drinking more alcohol than they should (34% vs. 16%).<\/p>\n\n

In other words, social science research has a long history of manipulation. Will it learn from its past?<\/p>","protected":false},"excerpt":{"rendered":"

The controversy over what the Facebook researchers did may be overshadowing other important discussions, specifically conversations about what they really found\u2014not much, actually\u2014and the right and wrong way to think about and report findings based on statistical analyses of Big Data. <\/p>\n","protected":false},"author":67,"featured_media":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"sub_headline":null,"sub_title":"","_prc_public_revisions":[],"_ppp_expiration_hours":0,"_ppp_enabled":false,"ai_generated_summary":"","apple_news_api_created_at":"","apple_news_api_id":"","apple_news_api_modified_at":"","apple_news_api_revision":"","apple_news_api_share_url":"","apple_news_cover_media_provider":"image","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_cover_video_id":0,"apple_news_cover_video_url":"","apple_news_cover_embedwebvideo_url":"","apple_news_is_hidden":"","apple_news_is_paid":"","apple_news_is_preview":"","apple_news_is_sponsored":"","apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_sections":[],"apple_news_suppress_video_url":false,"apple_news_use_image_component":false,"relatedPosts":[],"_prc_fork_parent":0,"_prc_fork_status":"","_prc_active_fork":0,"datacite_doi":"","datacite_doi_citation":"","_prc_seo_qr_attachment_id":0,"spoken_article_player_enabled":true,"displayBylines":true,"footnotes":"","prc_watchers":[]},"categories":[204,311,36,307],"bylines":[941],"collection":[],"datasets":[],"_post_visibility":[],"formats":[467],"_fund_pool":[],"languages":[],"regions-countries":[515],"research-teams":[526],"workflow-status":[],"class_list":["post-13994","short-read","type-short-read","status-publish","hentry","category-comparison-of-generations","category-facebook","category-methodological-research","category-social-media","bylines-rich-morin","formats-short-read","regions-countries-united-states","research-teams-internet"],"label":"Short Read","post_parent":0,"word_count":1068,"canonical_url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/short-reads\/2014\/07\/02\/facebooks-experiment-is-just-the-latest-to-manipulate-you-in-the-name-of-research\/","art_direction":false,"_embeds":[],"watchers":[],"table_of_contents":[],"datacite_doi":"","prc_seo_data":{"title":"Facebook\u2019s experiment causes a lot of fuss for little result","description":"The controversy over what the Facebook researchers did may be overshadowing other important discussions, specifically conversations about what they really found\u2014not much, actually\u2014and the right and wrong way to think about and report findings based on statistical analyses of Big Data.","og_title":"Facebook\u2019s experiment causes a lot of fuss for little result","og_description":"The controversy over what the Facebook researchers did may be overshadowing other important discussions, specifically conversations about what they really found\u2014not much, actually\u2014and the right and wrong way to think about and report findings based on statistical analyses of Big Data. ","schema_type":"Article","noindex":false,"canonical_url":"","primary_terms":[],"custom_schema":[],"og_image":0,"indexnow_submitted_at":null,"gsc_index_status":null},"prepublish_checks":{},"apple_news_notices":[],"jetpack_sharing_enabled":true,"relatedPostsOrdered":[],"bylinesOrdered":[{"key":"182dfe0c148e8cc2a4a455a9c1717022","termId":941}],"acknowledgementsOrdered":[],"_links":{"self":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/short-read\/13994","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/short-read"}],"about":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/types\/short-read"}],"author":[{"embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/users\/67"}],"replies":[{"embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/comments?post=13994"}],"version-history":[{"count":2,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/short-read\/13994\/revisions"}],"predecessor-version":[{"id":103569,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/short-read\/13994\/revisions\/103569"}],"wp:attachment":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/media?parent=13994"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/categories?post=13994"},{"taxonomy":"bylines","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/bylines?post=13994"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/collection?post=13994"},{"taxonomy":"datasets","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/datasets?post=13994"},{"taxonomy":"_post_visibility","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_post_visibility?post=13994"},{"taxonomy":"formats","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/formats?post=13994"},{"taxonomy":"_fund_pool","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_fund_pool?post=13994"},{"taxonomy":"languages","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/languages?post=13994"},{"taxonomy":"regions-countries","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/regions-countries?post=13994"},{"taxonomy":"research-teams","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/research-teams?post=13994"},{"taxonomy":"workflow-status","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/workflow-status?post=13994"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}